JP2009049976A - Image processing apparatus, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus that compresses a plurality of regions, respectively, in accordance with their characters. <P>SOLUTION: An image processing apparatus comprises: a characteristic region detecting section that detects a plurality of characteristic regions in an image; a condition storing section that stores thereon assignment conditions differing in accordance with characters of characteristic regions, so that different compression strength is assigned in accordance with the characters of the characteristic regions; a compressing section that compresses a plurality of characteristic region images that are images of the plurality of characteristic regions; and a compression control section that controls compression strength at which the compressing section compresses the plurality of characteristic region images in accordance with characters of the plurality of characteristic regions, with reference to the conditions stored in the condition storing section. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program. The present invention particularly relates to an image processing apparatus and an image processing method for processing an image, and a program for the image processing apparatus.

By comparing the encoded screen with the screen to be encoded, the change area in the screen is extracted, the screen to be encoded is divided into a plurality of image blocks, and the image block including the change area is encoded. There is known a moving image encoding device (see, for example, Patent Document 1). In this moving image encoding apparatus, more information bits than the number of information bits assigned to an image block including another change area are compared with an image block including a more limited area among image blocks including a change area. Assign a number and encode. There is also known a digital image generation method for generating additional information related to the importance of a photographing subject of a digital image and a background area corresponding to the subject and storing the additional information in association with a compressed digital image obtained by compressing the digital image. (For example, refer to Patent Document 2).
Japanese Patent No. 2828977 JP 2003-250132 A

  However, with the technique described in the above patent document, each of the plurality of regions cannot be compressed according to the feature.

  In order to solve the above-described problem, according to the first aspect of the present invention, an image processing device includes a feature region detection unit that detects a plurality of feature regions from an image, and different compression depending on the features of the feature regions. A condition storage unit that stores different assignment conditions according to the features of the feature region, and a compression unit that compresses each of a plurality of feature region images that are images of the plurality of feature regions, and a condition storage unit. The compression unit includes a compression control unit that controls the compression strength for compressing each of the plurality of feature region images according to the feature of the feature region based on the stored conditions.

  The feature region detection unit may detect a plurality of feature regions from a plurality of moving image constituent images included in a moving image that is an image. The compression control unit may control the compression strength at which the compression unit compresses each of the plurality of feature region images based on the conditions stored in the condition storage unit according to the features of the feature region.

  The condition storage unit may store different conditions depending on the type of feature region. Then, the compression control unit may control the compression strength at which the compression unit compresses each of the plurality of feature region images based on the condition stored in the condition storage unit according to the type of the feature region.

  The feature region detection unit detects a plurality of feature regions having different types of features from a plurality of moving image configuration images, and the compression unit includes a plurality of feature region images having the same type of features in the plurality of moving image configuration images. Each of the feature area videos may be compressed. Then, the compression control unit, based on the condition stored in the condition storage unit, the compression strength of the compression unit compressing each of the plurality of feature region moving images of the feature region image included in each of the plurality of feature region moving images. You may control according to the kind of feature, respectively.

  The condition storage unit stores different image quality reduction amounts according to the types of feature regions, and the compression unit stores each of a plurality of feature region moving images including a plurality of feature region images having the same type of features in a plurality of moving image constituent images. An image quality reduction unit that reduces the image quality, and the compression control unit converts the image quality reduction amount that the image quality reduction unit reduces the image quality based on the image quality reduction amount stored in the condition storage unit into a plurality of feature region moving images. You may control according to the kind of characteristic of the characteristic area image which each contains.

  According to a second aspect of the present invention, there is provided an image processing method, wherein a feature region detection step for detecting a plurality of feature regions from an image, and a feature region feature for assigning different compression strengths according to the feature region features Based on the condition storage stage for storing different allocation conditions depending on the condition, the compression stage for compressing each of the plurality of feature area images that are the images of the plurality of feature areas, and the conditions stored in the condition storage stage, A compression control step of controlling the compression strength at which each of the plurality of feature region images is compressed in the compression step according to the feature of the feature region.

  According to a third aspect of the present invention, there is provided a program for an image processing device, wherein the image processing device includes a feature region detection unit that detects a plurality of feature regions from an image, and a compression strength that varies depending on the features of the feature regions. A condition storage unit that stores different allocation conditions according to the features of the feature region, a compression unit that compresses each of a plurality of feature region images that are images of the plurality of feature regions, and a condition storage unit The compression unit functions as a compression control unit that controls the compression strength for compressing each of the plurality of feature region images according to the characteristics of the feature region.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of an image processing system 10 according to an embodiment. An object of the image processing system 10 is to reduce the amount of image data while maintaining a high quality image of a characteristic subject.

  The image processing system 10 includes a plurality of imaging devices 100a-c (hereinafter collectively referred to as imaging devices 100) that image the monitoring target space 150, and a plurality of image processing devices 120a-c (hereinafter referred to as image processing devices) that process images. 120), an image processing device 170, a communication network 110, an image DB 175, and a plurality of display devices 180a-c (hereinafter collectively referred to as display device 180).

  The image processing device 120a is connected to the imaging device 100a. Further, the image processing device 120b is connected to the imaging device 100b. Further, the image processing device 120c is connected to the imaging device 100c. Note that the image processing device 170 and the display device 180 are provided in a space 160 different from the monitoring target space 150.

  Hereinafter, operations of the imaging device 100a, the image processing device 120a, the image processing device 170, and the display device 180a will be described. The imaging apparatus 100a MPEG-encodes a captured moving image obtained by imaging the monitoring target space 150, generates captured moving image data, and outputs the captured moving image data to the image processing apparatus 120a to which the imaging apparatus 100a is connected.

  Specifically, the imaging device 100a includes an imaging unit 102a and a captured moving image compression unit 104a. The imaging unit 102a generates a plurality of moving image constituent images included in the captured moving image by imaging the monitoring target space 150. Note that the imaging unit 102a may generate a RAW video structure image. The captured moving image compression unit 104a synchronizes the RAW format moving image configuration images generated by the imaging unit 102a, and compresses the captured moving image including a plurality of moving image configuration images obtained by the synchronization by MPEG encoding or the like. Then, imaged moving image data is generated.

  The image processing device 120a acquires the captured moving image data generated by the imaging device 100a. The image processing device 120 decodes the captured moving image data acquired from the imaging device 100 to generate a captured moving image, and a plurality of features having different types of features such as a person 130 and a moving object 140 such as a vehicle from the generated captured moving image. Detect areas. Then, the image processing device 120a generates a plurality of feature area moving images by generating a moving image in which the characteristic region has a higher image quality than the other regions for each type of feature from the captured moving image. Further, the image processing apparatus 120a generates a background area moving image that is a moving image of a background area other than the characteristic area and has a lower image quality than the characteristic area moving image.

  Then, the image processing device 120a generates a plurality of feature region moving image data and background region moving image data by encoding the generated plurality of feature region moving images and background region moving images, respectively. At the same time, the image processing device 120a associates a plurality of characteristic area moving image data and background region moving image data obtained by encoding with each other and transmits them to the image processing device 170 through the communication network 110.

  The image processing device 170 acquires a plurality of feature region moving images and a background region moving image by decoding the associated plurality of feature region moving image data and background region moving image data received from the image processing device 120a, respectively. Then, the image processing apparatus 170 generates a single combined moving image by combining the plurality of feature region moving images and the background region moving image, and supplies the generated combined moving image to the display device 180a. The display device 180a displays the moving image supplied from the image processing device 170.

  Note that the image processing apparatus 170 may record the generated combined moving image or captured moving image data acquired from the image processing device 120a in the image DB 175. Then, the image processing device 170 may supply the composite moving image recorded in the image DB 175 to the display device 180a in response to a request from the display device 180a. Further, the image processing apparatus 170 may decode the captured moving image data recorded in the image DB 175 as described above and supply it to the display apparatus 180a in response to a request from the display apparatus 180a. Note that the image DB 175 may include a non-volatile recording medium such as a hard disk as an example, and may record the composite moving image supplied from the image processing apparatus 170 on the recording medium.

  Note that the imaging device 100b and the imaging device 100c have components having the same functions as the components of the imaging unit 102a. The functions and operations of the image capturing apparatus 100b and the image capturing apparatus 100c are the same as the functions and operations of the image capturing apparatus 100a except for providing the captured moving image data to the image processing apparatus 120b and the image processing apparatus 120c, respectively. Is omitted. In the following description, the imaging units 102 a to 102 c may be collectively referred to as the imaging unit 102, and the captured moving image compression units 104 a to 104 c may be collectively referred to as the captured moving image compression unit 104.

  The functions and operations of the image processing device 120b and the image processing device 120c are the same as the functions and operations of the image processing device 120a, except that the captured moving image data is acquired from the imaging device 100b and the imaging device 100c, respectively. Good. Therefore, the description is omitted. Further, the image processing device 170 generates one moving image from the plurality of associated feature region moving image data and background region moving image data received from the imaging device 100b and the imaging device 100c, respectively, and displays each of the display devices 180b. And supplied to the display device 180c. In addition, the display device 180b and the display device 180c display each moving image supplied from the image processing device 170.

  When the image processing system 10 of the present embodiment is actually applied as a monitoring system, for example, there may be a case where a subject that is characteristic as a monitoring target such as a person or a moving object can be left with high image quality. In addition, the amount of moving image data may be reduced.

  FIG. 2 shows an example of a block configuration of the image processing apparatus 120. The image processing apparatus 120 includes a compressed moving image acquisition unit 201, a compressed moving image decompression unit 202, a feature region detection unit 203, an image division unit 204, an image generation unit 205, a fixed value unit 210, a reduction unit 220, an encoding unit 230, An association processing unit 206 and an output unit 207 are included.

  Fixed value unit 210 includes a plurality of fixed value units 211a to 211c (hereinafter collectively referred to as fixed value unit 211). The reduction unit 220 includes a plurality of image quality reduction units 221a-d (hereinafter collectively referred to as image quality reduction units 221).

  The encoding unit 230 includes a background region moving image encoding unit 231a and a plurality of feature region moving image encoding units 231b-d (hereinafter collectively referred to as a feature region moving image encoding unit 231). The background area moving image encoding unit 231a and the feature area moving image encoding unit 231b-d may be collectively referred to as an encoding unit 231.

  The image quality reduction unit 221a and the background area moving image encoding unit 231a function as the compression unit 240a. Further, the image quality reduction unit 221b and the background area moving image encoding unit 231b function as the compression unit 240b. Similarly, the image quality reduction unit 221c and the background area moving image encoding unit 231c function as the compression unit 240c. The image quality reduction unit 221d and the background area moving image encoding unit 231d function as a compression unit 240d. The plurality of compression units 240a-d are collectively referred to as the compression unit 240.

  The compressed moving image acquisition unit 201 acquires a compressed moving image. Specifically, the compressed moving image acquisition unit 201 acquires encoded captured moving image data generated by the imaging device 100. The compressed moving image decompression unit 202 restores the moving image acquired by the compressed moving image acquisition unit 201 and generates a plurality of moving image constituent images included in the moving image. Specifically, the compressed moving image decompression unit 202 decodes the captured moving image data acquired by the compressed moving image acquisition unit 201, and generates a plurality of moving image constituent images included in the moving image. The moving image composition image includes a frame image and a field image.

  The feature region detection unit 203 detects a feature region from a plurality of moving image constituent images included in the moving image. Then, the image dividing unit 204 divides each of the plurality of moving image constituent images into a feature area and a background area.

  The image generation unit 205 generates a plurality of feature area compression moving images each including a plurality of feature area images by extracting feature area images from the plurality of moving image constituent images. Specifically, the image generation unit 205 reproduces a moving image to compress a plurality of feature region moving images and compress a plurality of feature region moving images and a background region moving image to compress a background region moving image. Generate a video.

  Then, the fixed value converting unit 211 converts the pixel values of regions other than the feature region image in the plurality of moving image constituent images included in the feature region compression moving image to a fixed value. For example, the fixed value unit 211 sets a pixel value of an area other than the feature area image to a predetermined value (for example, a luminance value of 0). Then, the compression unit 240 compresses a plurality of feature region compression images each including a plurality of moving image constituent images in which pixel values of regions other than the feature region image are fixed values with an intensity corresponding to each feature amount. . As described above, the compression unit 240 compresses each of the plurality of feature region compression moving images and the background region compression moving image with an intensity corresponding to each feature amount.

  As described above, the feature region detection unit 203 detects a feature region from the image. Then, the image dividing unit 204 divides the image into a feature region and a background region other than the feature region. Then, the compression unit 240 compresses the feature area image that is the image of the feature area and the background area image that is the image of the background area with different strengths. Then, the compression unit 240 compresses the feature area moving image including a plurality of characteristic area images and the background area moving image including a plurality of background area images with different strengths.

  Note that in the compression unit 240b, the compression unit 240c, and the compression unit 240d, it is determined in advance which type of feature area moving image should be compressed, and the compression unit 240b, the compression unit 240c, and the compression unit 240d are determined in advance. The feature region moving image of the specified feature type is compressed. Note that the compression strength in the case of compressing the feature region moving image is determined in advance for each of a plurality of feature types, and the compression unit 240b, the compression unit 240c, and the compression unit 240d are features of the predetermined feature types. The area moving image is compressed with a compression strength predetermined for the type of the feature. As described above, the compression unit 240 compresses a plurality of regions in parallel using the compressor provided for each image region divided by the image dividing unit 204.

  Note that the compression unit 240 may be implemented by a single compressor, and may sequentially compress a plurality of feature area videos and background area videos. In addition, the compression unit 240 is predetermined for each feature type and background of each region for each region obtained by dividing the captured moving image decoded by the compressed moving image decompression unit 202 by the image dividing unit 204. One video data may be generated by compressing at a compression rate.

  Note that the feature region detection unit 203 detects a plurality of feature regions having different types of features from a plurality of moving image constituent images included in a moving image that is an image. Then, the image dividing unit 204 divides the plurality of moving image constituent images into each of a plurality of feature areas and a background area other than the plurality of feature areas. Then, the compression unit 240 compresses the plurality of feature area moving images and the background area moving image with an intensity corresponding to each feature amount. The feature amount includes the type of subject, the size of the subject, the moving speed at which the moving object moves, and the size of the feature region.

  Specifically, the image quality reduction unit 221 compresses a plurality of feature area moving images and background area moving images by reducing the image quality according to each feature amount. More specifically, the image quality reduction unit 221 compresses a plurality of feature region moving images and background region moving images by reducing the resolution or frame rate according to each feature amount. Then, the encoding unit 231 compresses the plurality of feature area moving images and the background area moving image by encoding using setting values corresponding to the respective feature amounts. For example, the encoding unit 231 compresses a plurality of feature area moving images and background area moving images by encoding them using an assigned code amount corresponding to each feature amount.

  The association processing unit 206 includes a plurality of feature region moving image data and background region moving image data generated by the compression units 240 compressing a plurality of feature region moving images and background region moving images, for example, with tag information and the like. And correspond to each other. The output unit 207 sends the plurality of feature area moving image data and background area moving image data associated by the association processing unit 206 to the communication network 110.

  In the configuration of this figure, each of the plurality of compression units 240 included in the image processing device 120 compresses each of the plurality of feature region images and the background region image. , And one compression unit 240 may compress the images of the plurality of characteristic regions and the images of the background region with different strengths. For example, a plurality of feature region images and a background region image are sequentially supplied to one compression unit 240 in a time-sharing manner, and the one compression unit 240 converts the plurality of feature region images and the background region image with different intensities. You may compress sequentially.

  In addition, the one compression unit 240 quantizes the image information of the plurality of feature regions and the image information of the background region with different quantization coefficients, respectively, thereby respectively converting the images of the plurality of feature regions and the images of the background region. It may be compressed with different strengths. In addition, an image obtained by converting a plurality of feature region images and a background region image into images of different image quality is supplied to one compression unit 240, and the supplied plurality of feature region images and background region images are combined into one image. The compression units 240 may compress each.

  In the form in which one compression unit 240 quantizes with a different quantization coefficient for each region as described above, or one compression unit 240 compresses an image converted into a different image quality for each region, one compression unit 240 240 may compress one image, or may compress each of the images divided by the image dividing unit 204 as described with reference to FIG. Note that when one compression unit 240 compresses one image, the division processing by the image division unit 204 and the fixed value processing by the fixed value conversion unit 211 do not have to be performed. The dividing unit 204 and the fixed value unit 210 may not be provided.

  FIG. 3 shows an example of a block configuration of the image processing apparatus 170. The image processing apparatus 170 includes a compressed moving image acquisition unit 301, an association analysis unit 302, a compressed moving image decompression unit 310, a composition unit 303, and an output unit 304. The compressed moving image expansion unit 310 includes a plurality of compressed moving image expansion units 311a-d (hereinafter collectively referred to as a compressed moving image expansion unit 311).

  The compressed moving image acquisition unit 301 acquires a plurality of associated feature region moving image data and background region moving image data output from the output unit 207. The association analysis unit 302 analyzes, for example, attached tag information, and extracts a plurality of associated feature region movie data and background region movie data acquired by the compressed movie acquisition unit 301.

  The compressed moving image decompression unit 311 decodes the plurality of characteristic area moving image data and background area moving image data. Specifically, the compressed moving image decompression unit 311a decodes the background area moving image data. In addition, the compressed moving image decompression unit 311b-d decodes one feature region moving image out of the plurality of feature region moving image data, and acquires a plurality of feature region moving images and a background region moving image. Note that the compressed moving image decompression unit 311b-d is provided for each type of feature, and decodes any type of feature region moving image data.

  The synthesizing unit 303 synthesizes the moving image constituent images obtained by the compressed moving image decompressing unit 311 decoding. Specifically, the synthesizing unit 303 superimposes the feature region images of the moving image constituent images included in each of the plurality of feature region moving images decoded by the compressed moving image decompression unit 311b-d on the moving image constituent images included in the background region moving image. A combined moving image composition image is generated. The output unit 304 supplies a moving image including a plurality of moving image constituent images generated by the combining unit 303 to the display device 180. Note that the output unit 304 may record a moving image including a plurality of moving image constituent images generated by the combining unit 303 in the image DB 175. Note that the image processing apparatus 170 may record a plurality of feature area moving image data and background area moving image data output in association with each other from the output unit 207 in the image DB 175. In this case, the compressed moving image acquisition unit 301 may acquire a plurality of associated feature region moving image data and background region moving image data recorded in the image DB 175.

  Note that the compressed video decompression unit 310 of the present embodiment includes a plurality of compressed video decompression units 311 corresponding to the number of feature types, but in another form, one compressed video decompression unit included in the compressed video decompression unit 310. 311 may sequentially decode the background area moving image data and the plurality of characteristic area moving image data. In addition, when provided as one moving image data from the image processing device 120, one compressed moving image decompression unit 311 decodes the one moving image data, and the output unit 304 outputs the decoded moving image. May be.

  FIG. 4 shows an example of the processing flow of the image processing apparatus 120. The compressed moving image acquisition unit 201 acquires captured moving image data (401). The compressed moving image decompression unit 202 generates a plurality of frame images 410 by decoding the captured moving image data. The feature region detection unit 203 detects a region of interest (ROI) that is an example of a feature region based on the image content of the frame image 410 or the image content of the plurality of frame images 410 (402).

  The feature region detection unit 203 detects regions including human faces, human bodies, and moving objects as different types of ROIs. For example, the feature area detection unit 203 detects, by pattern matching or the like, an area including an object having a degree of coincidence with a pattern related to a predetermined person's face that is greater than a predetermined degree of coincidence, and detects the detected face area as an ROI. And Further, the feature region detection unit 203 detects a human body region including an object having a matching degree with respect to a pattern related to a human body greater than a predetermined matching degree by pattern matching or the like, and sets the detected region as an ROI. Note that the feature region detection unit 203 may detect a human body region from a region existing in the vicinity of the face region.

  In addition, the feature region detection unit 203 identifies a moving region that is a region including a moving object based on the image contents of a plurality of frame images. For example, an area in which the amount of change in pixel value with respect to another frame image is larger than a predetermined amount of change is specified as a moving area. In addition, the feature region detection unit 203 extracts objects included in each of the plurality of frame images by edge extraction or the like. Then, the feature region detection unit 203 identifies objects that are included in different positions of other frame images and that match with a degree of coincidence that is greater than a predetermined degree of coincidence, and includes an area that includes the identified object. It may be specified as a moving area.

  As described above, the feature region detection unit 203 detects a region that meets a predetermined condition relating to image content as an ROI. More specifically, the feature area detection unit 203 detects an area including an object that meets a predetermined condition as an ROI. For example, the feature area detection unit 203 detects, as an ROI, an area that includes an object having a degree of coincidence with a predetermined shape that is greater than a predetermined degree of coincidence. Further, the feature area detection unit 203 detects an area where the change amount of the image is larger than the predetermined change amount as the ROI. For example, an area where the change amount of the pixel value between other frame images is larger than a predetermined change amount is detected as the ROI.

  In addition to the above-described human face and human body, the feature region detection unit 203 includes at least one part of the human head such as a part of the head of the person, a part of the human body such as the hand of the person, or a living body other than the human body. An area where a part of the part is imaged can be detected as an ROI. The living body includes a specific tissue existing inside the living body such as a tumor tissue or a blood vessel inside the living body. In addition to the living body, the feature region detection unit 203 may detect a region where a card such as money, a cash card, a vehicle, or a license plate of the vehicle is captured as an ROI.

  In addition to pattern matching by template matching or the like, the feature region detection unit 203 detects ROI based on a learning result by machine learning (for example, Adaboost) described in, for example, Japanese Patent Application Laid-Open No. 2007-188419. You can also For example, an image feature amount extracted from a predetermined subject image and an image feature amount extracted from a subject image other than the predetermined subject are extracted from the predetermined subject image. Learn the features of the image features. Then, the feature region detection unit 203 may detect, as an ROI, a region from which an image feature amount having a feature that matches the learned feature is extracted. Thereby, the feature region detection unit 203 can detect a region where a predetermined subject is imaged as an ROI. Note that the feature region detection unit 203 may detect an ROI having an arbitrary shape including a rectangle.

  Note that the feature region detection unit 203 may detect a feature region by the method described in Japanese Patent Application No. 2008-078641. For example, the feature region detection unit 203 includes an image group including a captured image and one or more thinned images by thinning out the captured image of the object detection target at a predetermined ratio, or by thinning out in stages at the predetermined ratio. Is generated. Then, the feature region detection unit 203 calculates the evaluation value by applying the first filter to the relatively small first image in the generated image group. Here, the first filter acts on a two-dimensionally expanding region on the image, thereby generating an evaluation value representing the probability that a specific type of object exists in the region. In addition, the first filter includes a plurality of filters each acting on a plurality of areas having different numbers of pixels corresponding to the area of the area on the image or different in stages at the predetermined ratio. It may be a filter that acts on a relatively narrow region of the filter group that includes the filter group. The feature area detection unit 203 extracts a primary candidate area from which an evaluation value exceeding a predetermined first threshold is obtained from the first image.

  Then, the feature region detection unit 203 is a region that is one step wider than the first filter in the filter group in a region corresponding to the primary candidate region in the second image that has one more pixel number than the first image. The evaluation value is calculated by applying the second filter that operates on the. Then, the feature region detection unit 203 extracts secondary candidate regions from which an evaluation value exceeding a predetermined second threshold value can be obtained.

  Here, the feature region detection unit 203 repeats the extraction process of extracting the candidate region by applying the plurality of filters that operate on the regions having different widths to the corresponding region of the image group. Go. At this time, the feature region detection unit 203 sequentially repeats an extraction process that operates a filter that operates on a relatively narrow area, and an extraction process that operates a filter that operates on a relatively wide area. Specifically, the feature region detection unit 203 operates a filter that operates on a relatively large image on a relatively large image from an extraction process that operates on a relatively small image on a relatively small image. Repeat sequentially for the extraction process. Then, the feature region detection unit 203 detects a specific type of object by repeating two or more extraction processes and finally extracting candidate regions. Then, the feature region detection unit 203 detects a region where the specific type of object exists as a feature region. As described above, the feature region detection unit 203 causes the filter to be limited to the region extracted in the immediately preceding process in the subsequent extraction process. Therefore, the presence / absence of the object is sequentially selected in each of the plurality of extraction processes, and the feature region can be detected with higher accuracy. In addition, since the feature area is roughly screened with a small-sized image, the feature area can be detected at a higher speed.

  The feature region detection unit 203 may detect the feature region by a method described in Japanese Patent Application No. 2008-078636. For example, the feature region detection unit 203 operates on a region of a predetermined area that spreads two-dimensionally with the captured image, and calculates a plurality of feature amounts that are different from each other among the contour and the inside of a specific type of object. A feature region is detected using a filter. Specifically, the feature region detection unit 203 calculates a plurality of feature amounts by causing the plurality of filters to act on regions of a predetermined area on the captured image that is the object detection target. Here, a correspondence relationship between each feature amount calculated by each of the plurality of filters and a primary evaluation value representing the probability of being a specific type of object is associated with the plurality of filters. The feature region detection unit 203 obtains each primary evaluation value corresponding to each calculated feature amount based on the correspondence relationship. Then, the feature area detection unit 203 obtains a secondary evaluation value representing the probability that a specific type of object exists in the area by integrating a plurality of primary evaluation values corresponding to the plurality of filters. Then, the feature region detection unit 203 compares the secondary evaluation value with a threshold value, and extracts a region with a high probability that a specific type of object exists beyond the threshold value. Thereby, the feature region detection unit 203 detects the region as a feature region where a specific type of object exists. As described above, the feature region detection unit 203 combines a plurality of filters that extract feature amounts representing various features inside and inside the object outline, so that, for example, the feature area detection unit 203 has higher accuracy than the extraction based only on the outline shape. Regions can be extracted.

  The feature region detection unit 203 may detect the feature region by a method in which the method described in Japanese Patent Application No. 2008-078636 and the method described in Japanese Patent Application No. 2008-078641 are combined. Specifically, the plurality of filters described in relation to the method described in Japanese Patent Application No. 2008-078636 have a plurality of areas in which the number of pixels is different at a predetermined ratio or stepwise at a predetermined ratio. A plurality of filters may be included for each area, each acting on each of the regions. Each of the filters may be associated with the correspondence relationship corresponding to each of the filters. Then, the feature area detection unit 203 thins out the captured image of the object detection target at a predetermined ratio, or thins out the image at a predetermined ratio step by step, thereby obtaining an image group including the captured image and one or more thinned images. Generate. The feature region detection unit 203 calculates a plurality of feature amounts by applying a plurality of first filters that act on a relatively narrow region to a relatively small first image in the image group. Then, the feature area detection unit 203 obtains each primary evaluation value corresponding to each calculated feature amount based on the correspondence relationship corresponding to each of the plurality of first filters. Then, the feature area detection unit 203 obtains a secondary evaluation value representing the probability that a specific type of object exists in the area by integrating a plurality of primary evaluation values. Then, the feature region detection unit 203 compares the obtained secondary evaluation value with the first threshold value, and extracts a primary candidate region that has a high probability that a specific type of object exists beyond the first threshold value. .

  In addition, the feature region detection unit 203 has one stage higher than the plurality of first filters in the region corresponding to the primary candidate region of the second image having one step higher number of pixels than the first image in the image group. A plurality of feature amounts are calculated by applying a plurality of second filters that operate over a wide area. Then, the feature region detection unit 203 obtains each primary evaluation value corresponding to each calculated feature amount based on the correspondence relationship corresponding to each of the plurality of second filters. Then, the feature region detection unit 203 integrates a plurality of primary evaluation values corresponding to the plurality of second filters, and thereby represents a probability that a specific type of object exists in the region corresponding to the primary candidate region. Obtain an evaluation value. Then, the feature region detection unit 203 compares the obtained secondary evaluation value with the second threshold value, and extracts a secondary candidate region that has a high probability that a specific type of object exists beyond the second threshold value. .

  Here, the feature region detection unit 203 repeats the extraction process of extracting the candidate region by applying the plurality of filters that operate on the regions having different widths to the corresponding region of the image group. Go. At this time, the feature region detection unit 203 sequentially repeats an extraction process that operates a filter that operates on a relatively narrow area, and an extraction process that operates a filter that operates on a relatively wide area. Specifically, the feature region detection unit 203 operates a filter that operates on a relatively large image on a relatively large image from an extraction process that operates on a relatively small image on a relatively small image. Repeat sequentially for the extraction process. Then, the feature region detection unit 203 detects a specific type of object by repeating two or more extraction processes and finally extracting candidate regions. Then, the feature region detection unit 203 detects a region where the specific type of object exists as a feature region.

  The feature region detection unit 203 may detect the feature region by the method described in Japanese Patent Application No. 2008-098600. For example, the feature region detection unit 203 detects a feature region from a plurality of captured images included in moving images captured by the plurality of imaging devices 100. As an example, it is assumed that the imaging device 100a and the imaging device 100b capture the same scene. For example, the imaging device 100a and the imaging device 100b can function as a stereo camera. In the following description, the first captured image captured by the image capturing apparatus 100a and the second captured image captured by the image capturing apparatus 100b are referred to as a pair image. The feature region detection unit 203 detects a specific type of object that is projected on the pair image from the pair image, and detects the detected region of the specific type of object as a feature region.

  For each of the first and second captured images regarded as a pair image, the feature region detection unit 203 extracts a region in which a specific type of object is imprinted on each captured image. Here, the feature area detection unit 203 may detect an area in which a specific type of object is imprinted with rough detection accuracy. The feature region detection unit 203 detects a specific type of object by detecting a pair of regions corresponding to each other from the extracted region on the first captured image and the region on the second captured image. To do. For example, the feature area detection unit 203 calculates the distance from the image of the paired area to the subject imaged in the area. The feature area detection unit 203 can detect a specific type of object based on the three-dimensional shape of the subject based on the distance to the subject.

  Here, when detecting a pair of regions corresponding to each other, the feature region detection unit 203 detects a region in which a specific type of object detected from the first and second captured images considered as a pair image is imprinted. Divide into multiple sub-regions. Then, the feature region detection unit 203 calculates a vector over a plurality of subregions of the feature amount characterizing the partial image projected in each subregion. Here, as the feature amount, a pixel value can be exemplified. Moreover, as a vector over a plurality of sub-regions, a gradient vector (for example, a pixel value gradient vector) can be exemplified. Then, the feature area detection unit 203 calculates a logical distance between the calculated area vector on the first image and the vector on the second image. The feature region detection unit 203 detects a pair of regions whose logical distance is smaller than a predetermined value as a pair of regions corresponding to each other. As the logical distance, the square root of the sum of squares with respect to the difference between the components forming the vector can be exemplified. In this way, the feature region detection unit 203 can extract a pair of corresponding regions from the pair image with high accuracy, and thus can calculate the distance to the subject with high accuracy. Therefore, the feature region detection unit 203 can recognize the three-dimensional shape of the subject with high accuracy, and as a result, can detect a specific type of object with higher accuracy.

  The feature region detection unit 203 may detect the feature region by the method described in Japanese Patent Application No. 2008-091562. For example, the feature region detection unit 203 obtains a similar subject shape similar to a specific type of subject from each of the plurality of captured images included in the moving image, the size of the similar subject shape, and the position on the angle of view of the imaging device 100. Extract with information. The position information on the angle of view can be exemplified by the position on the image area in the captured image. Then, the feature region detection unit 203 determines whether or not the extracted subject of the similar subject shape is a specific type of subject, and extracts the specific type of subject. For example, the feature region detection unit 203 counts the number of detected similar subject shapes classified into the same size from a predetermined search region around the similar subject shape, and the count value is If the threshold is equal to or greater than the threshold, the subject having the similar subject shape may be extracted as a specific type of subject. Then, the feature region detection unit 203 may detect a region including a specific type of subject as a feature region. For this reason, the feature region detection unit 203 can detect a subject having a similar subject shape in an image region in which subjects having a size close to a predetermined size are concentrated and detected as a specific type of subject. Then, it is not necessary to detect a subject having a similar subject shape in a region other than the image region as a specific type of subject. For this reason, it is possible to reduce the probability of erroneously detecting a subject having a similar subject shape in a region other than the image region as a specific type of subject.

  Note that when the imaging apparatus 100 can capture an image with a variable angle of view, the position information on the angle of view can be exemplified by the imaging direction of the imaging apparatus 100 and the position on the captured image. In addition, when a plurality of imaging devices 100 can capture a continuous scene that is wider than the subject captured by one imaging device 100, the positional information on the angle of view includes a plurality of imaging devices. The respective imaging directions of 100 and the positions on the captured image captured by each of the plurality of imaging devices 100 can be exemplified.

  Then, the image processing apparatus 120 generates a moving image for compression based on the detected ROI (403). Specifically, the image dividing unit 204 divides the frame image into ROI and other areas. Then, the image generation unit 205 generates a feature region moving image 430, a feature region moving image 440, a feature region moving image 450, and a background region moving image 420 by duplicating the plurality of frame images 410. Specifically, the image generation unit 205 reproduces a plurality of frame images 410, thereby generating a facial region feature region moving image 450, a human region feature region moving image 440, a moving region feature region moving image 430, and A background area moving image 420 for the background area is generated.

  Then, the image processing apparatus 120 reduces the image quality of the feature region moving image 430, the feature region moving image 440, the feature region moving image 450, and the background region moving image 420 by the fixed value converting unit 211 and the image quality reducing unit 221. (404a, 404b, 404c, 404d). Specifically, the fixed value unit 211 changes the pixel values in the ROI divided by the image dividing unit 204 in the frame images included in each of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450. First, the pixel value for the region other than the ROI is set to a predetermined value (for example, luminance value 0). Note that the fixed value converting unit 211 may set the pixel values in the region other than the ROI as an average pixel value of the pixel values in the region near the ROI.

  As described above, the image generation unit 205 and the fixed value conversion unit 211 generate the feature region moving image 430, the feature region moving image 440, the feature region moving image 450, and the background region moving image 420 each including a plurality of frame images having the same viewing angle. The As will be described in detail later, the image processing apparatus 170 converts a moving image in which a region other than the ROI, such as the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450, into a fixed value is used as the background region moving image. A moving image superimposed on 420 is generated. Therefore, the background area animation 420, the feature area animation 430, the feature area animation 440, and the feature area animation 450 can be regarded as a background layer, a motion area layer, a human area layer, and a face area layer, respectively.

  In addition, the image quality reduction unit 221 reduces the image quality of the image in the ROI in the frame images included in each of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 according to the type of feature. Specifically, at least one of resolution, the number of gradations, and the number of colors is determined in advance as a parameter that determines the image quality of each of the face area, the human area, and the moving area. For example, a high resolution is determined in advance in the order of the face area, the human area, and the moving area.

  Then, the image quality reduction unit 221 sets the image quality of the image in the ROI in the frame image included in each of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 to a resolution predetermined for the type of feature, Convert to an image with the number of gradations and the number of colors. In addition, the image quality reduction unit 221 makes the image quality of the frame image of the background area moving image lower than the image quality of the image in the ROI. For example, the image quality reduction unit 221 makes the resolution of the frame image of the background area moving image smaller than that of the image in the ROI.

  In addition, the image quality reduction unit 221 reduces the frame rates of the background area moving image 420, the feature area moving image 430, the feature area moving image 440, and the feature area moving image 450. For example, a frame rate is determined in advance for each type of feature, for example, for each face area, human area, and moving area. Then, the image quality reduction unit 221 thins out the frame images included in the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 at a predetermined interval according to a frame rate that is predetermined according to the type of the feature. The frame rates of the feature area animation 430, the feature area animation 440, and the feature area animation 450 are reduced. In addition, the image quality reduction unit 221 reduces the frame rate of the background area moving image 420 by thinning out the frame images included in the background area moving image 420 according to a frame rate predetermined as the frame rate of the background area moving image.

  The image quality reduction unit 221a reduces the image quality of the background area moving image 420. In addition, the image quality reduction unit 221b, the image quality reduction unit 221c, and the image quality reduction unit 221d reduce the image quality of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450, respectively.

  Then, the background area moving image encoding unit 231a and the area moving image encoding unit 231b-d encode the moving images whose image quality has been reduced by the image quality reducing unit 221 (405a, 405b, 405c, and 405d). For example, the background area moving image encoding unit 231a and the area moving image encoding unit 231b-d each encode the moving image whose image quality has been reduced by the image quality reducing unit 221.

  For example, the background area moving image encoding unit 231a performs MPEG encoding on the background area moving image with the encoding setting for the background area moving image. The feature region moving image encoding unit 231b, the feature region moving image encoding unit 231c, and the feature region moving image encoding unit 231d are encoding settings for the feature region moving image of the moving region, the human region, and the face region, respectively. Each area moving image is MPEG-encoded. Note that the coding setting includes setting a quantization table in MPEG coding, for example. The encoding setting will be described with reference to FIG.

  Then, the association processing unit 206 adds tag information to the background region moving image data and the plurality of feature region moving image data obtained by encoding by the background region moving image encoding unit 231a and the feature region moving image encoding unit 231b-d. The output unit 207 outputs the information to the display device 180 by associating it with an accessory (406). At this time, the association processing unit 206 adds timing information (for example, time stamp) including information indicating the display timing of the frame image included in each of the background area moving image and the plurality of characteristic area moving images to the tag information or the like. Good. In addition, the association processing unit 206 identifies the imaging device 100 that captured the feature region information including information indicating the ranges of the plurality of feature regions, the background region movie, and the captured movie data that is the basis of the plurality of feature region movies. Identification information or the like to be added may be added to tag information or the like.

  As described above, the feature region detection unit 203 detects a plurality of feature regions having different types of subjects from a plurality of moving image constituent images included in a moving image that is an image. Then, the compression unit 240 compresses the plurality of feature area moving images with an intensity corresponding to the type of each subject. In the present embodiment, a human face and a human body region have been described as examples of the types of subjects. However, in other examples, vehicle license plates and vehicle bodies other than license plates may be used as the types of subjects.

  Further, the front face and the side face of the person may be the subject type. Further, a stationary subject and a moving subject may be used as the subject type. Further, the feature region detection unit 203 may detect a region including a plurality of subjects having different distances from the imaging device 100 to the subject as a plurality of feature regions having different types of features.

  Note that the compression unit 240 may compress the motion region, the human body, the side profile of the person, and the face in front of the person in order of large compression strengths, which are examples of feature types. In the present embodiment, in consideration of the purpose of using the image processing system 10 as a monitoring system, an area including a human face is detected as an ROI, and the detected ROI has a higher image quality than an area other than the ROI. . However, the image processing system 10 can also be used as, for example, a system that captures landscapes of street corners. When the image processing system 10 is used in such a form, for the purpose of protecting personal information, an area including a human face is detected as an ROI, and the detected ROI has a lower image quality than an area other than the ROI. It may be. For example, the compressing unit 240 may compress the feature region moving image and the background region moving image with a large compression strength in the order of the face in front of the person, the profile of the person, the human body, the motion region, and the background region.

  In addition, the feature region detection unit 203 may detect a plurality of feature regions having different subject velocities from a plurality of frame images. In this case, the image quality reduction unit 221 may convert the feature area moving image into a moving image having a higher frame rate as the subject speed increases. As described above, the compression unit 240 may compress a plurality of feature region moving images with an intensity corresponding to the speed of each subject.

  As described above, the image processing apparatus 120 generates a plurality of feature region moving images and background region moving images having the same viewing angle by fixing a region other than the ROI in the frame image included in the feature region moving image. For this reason, according to the image processing apparatus 120, the feature region moving image may be compressed at a high compression rate using a general-purpose encoder without using a specially designed encoder. For example, when a feature area moving image is encoded using a motion vector as in MPEG encoding, the difference value between pixel values is often 0 for a macroblock in a fixed value area. Therefore, such fixed values may reduce the manufacturing cost of the image processing apparatus 120 while maintaining a high compression rate.

  In the above description, the compression unit 240 compresses the feature region moving image including the frame image in which the region other than the ROI region is fixed. Note that the compression unit 240 may cut out and compress an image in the ROI in the frame image included in the feature area moving image and output it as a feature area moving image.

  When the feature area detection unit 203 has not detected the ROI, the output unit 207 outputs the background area moving image data output from the compression unit 240a to the image processing apparatus 170. In this case, the image generation unit 205 does not have to generate the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450. Then, the image generation unit 205 generates the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 on condition that the feature region detection unit 203 detects the ROI, and the output unit 207 performs the above processing. The plurality of characteristic area moving image data and background area moving image data generated by the above is output to the image processing apparatus 170. During this time, the compression unit 240a may continue to compress the background area moving image 420 at a predetermined compression ratio for the background area in order to compress the background area moving image 420.

  In addition, while the feature region detection unit 203 does not detect the ROI, the compression unit 240 is set in advance to be lower than the compression rate for the background region and higher than the compression rate for compressing the feature region moving image. The background area moving image may be compressed at the ROI non-detection period compression rate. Then, the compression unit 240 may compress the background region moving image with the background region compression rate on condition that the feature region detection unit 203 detects the ROI. At this time, the compression unit 240 may compress the feature region moving image at a compression rate lower than the ROI non-detection period compression rate.

  The compression unit 240 compresses the background region moving image of a predetermined period after the ROI is detected by the feature region detection unit 203 at the ROI non-detection period compression rate, and a period longer than the predetermined period elapses. As a condition, the background area moving image may be compressed at the compression ratio for the background area. In this case, even when the region that should originally be detected as ROI is not detected as ROI, a background region moving image having a certain level of image quality may be provided. The compression unit 240 also sets the image quality of the region including the ROI region in the other frame image estimated from the position of the ROI region in the plurality of frame images detected by the feature region detection unit 203 to be higher than the image quality of the other region. Therefore, the compression rate may be different for each region.

  FIG. 5 shows an example of the image quality of a plurality of feature area videos and background area videos. Here, for the purpose of simplifying the description, it is assumed that the frame rate of the captured moving image data acquired by the compressed moving image acquisition unit 201 is 16 fps, and the resolution of the frame image included in the captured moving image data is 72 dpi.

  The resolution ratio indicating the ratio of the resolution of the frame image included in the background area moving image 420 after image quality reduction to the resolution of the frame image 410 included in the captured moving image is predetermined as 1/8. The image quality reduction unit 221 generates a 9 dpi frame image having a resolution of 1/8 with respect to the frame image included in the background area video 420 before image quality reduction generated by copying the captured video by the image generation unit 205. This is generated by thinning out the pixels of the frame image included in the background area moving image 420 before image quality reduction. In addition, the frame rate ratio indicating the ratio of the frame rate of the background area moving image 420 after image quality reduction to the frame rate of the captured moving image is set to 1/8 in advance. The image quality reduction unit 221 converts the background area video 420 having a frame rate of 2 fps, which is a frame rate of 1/8 of the frame rate of the background area video 420 before the image quality reduction, into a frame image included in the background area video 420 before the image quality reduction. Generate by thinning out.

  Similarly, a resolution ratio and a frame rate ratio are determined for each of the plurality of characteristic area moving images. For example, the resolution ratio and the frame rate ratio are set to 1/4 for the feature area moving image 430, and the resolution ratio and the frame rate ratio are set to 1/2 for the feature area moving image 440. Is defined as 1/1 as the resolution ratio and the frame rate ratio. As a result, the image quality reduction unit 221b generates a feature region moving image 430 having a frame rate of 4 fps and a frame image resolution of 18 dpi. In addition, the image quality reduction unit 221c generates a feature region moving image 440 having a frame rate of 8 fps and a frame image resolution of 36 dpi. In addition, the image quality reduction unit 221d generates a feature region moving image 450 having a frame rate of 16 fps and a frame image resolution of 72 dpi.

  In the above example, the case where the image quality reduction unit 221 reduces the image quality of the frame image by thinning out the pixels of the frame image included in the plurality of feature region moving images and the background region moving image has been described as an example. In addition, the image quality reduction unit 221 may reduce the image quality of the frame image using a filter that passes through a predetermined frequency region, such as a low-pass filter. In this case, the frequency band through which the filter passes and the intensity indicating the degree of passage may be determined in advance for each type of feature, for example, for each of the background region, the moving region, the human region, and the face region.

  In addition to the image quality reduction by the image quality reduction unit 221, or in place of the image quality reduction by the image quality reduction unit 221, the encoding unit 231 may reduce the image quality of the frame image. For example, the encoding unit 231 can reduce the image quality by increasing the value of the quantization table in MPEG encoding. The magnitude of the value of the quantization table may be determined in advance for each type of feature. For example, each of the background region moving image encoding unit 231a and the plurality of feature region moving image encoding units 231b-d may perform encoding using a quantization table having a preset value. In addition, a value for each frequency component in the quantization table may be determined in advance for each type of feature.

  Further, the image quality reduction unit 221 may further average a plurality of frame images included in the background area moving image. According to this, when an object indicating a moving object is included in the frame image, a frame image in which the objects of the moving object are averaged is obtained. When such a plurality of frame images are continuously displayed, the movement of the moving object may appear smoothly in the viewer's eyes.

  In the above description, after the image generation unit 205 replicates the captured moving image, frame images included in a plurality of feature region moving images and background region moving images obtained by copying are thinned out or pixels are thinned out. In the above, the mode of compressing a plurality of feature region moving images and background region moving images has been described. In addition, the image generation unit 205 generates a plurality of feature area videos and background area videos with a reduced frame rate by thinning out and selecting a plurality of frame images included in the captured video according to the frame rate ratio. Also good. Then, the image quality of the plurality of feature area moving images and the background area moving images may be reduced by the fixed value converting unit 211a fixing the value and the image quality reducing unit 221 reducing the resolution.

  FIG. 6 shows an example of the processing flow of the image processing apparatus 170. The compressed moving image acquisition unit 301 acquires a plurality of associated feature area moving image data and background area moving image data from the image processing device 120, and identifies timing information, information for identifying the imaging device 100, and the like using the attached tag information. Is acquired (601). Then, the compressed moving image decompression unit 311 generates a background region moving image 610 indicating a background layer by decoding the plurality of feature region moving image data and background region moving image data (602a). At the same time, the compressed moving image decompression unit 311 generates a feature region moving image 620 indicating a moving region layer, a feature region moving image 630 indicating a human region layer, and a feature region moving image 640 indicating a face region layer (602b, 602c, and 602d). .

  The synthesizing unit 303 synthesizes frame images included in the background area moving image 610, the characteristic area moving image 620, the characteristic area moving image 630, and the characteristic area moving image 640 (603). At this time, the synthesis unit 303 enlarges the frame image so that the subject at the same position as the frame image having the maximum resolution overlaps according to the resolution of the frame image included in each of the background area moving image and the plurality of characteristic area moving images. Then, the enlarged frame images are superimposed to generate a composite frame image.

  At this time, the synthesizing unit 303 cuts out the feature region image in the frame image included in the feature region moving image 620, the feature region moving image 630, and the feature region moving image 640, and overwrites the frame image included in the background region moving image 610. To generate a composite frame image. In addition, when the frame rates of the background area moving image 610, the feature area moving image 620, the feature area moving image 630, and the feature area moving image 640 are different, the synthesizing unit 303 performs the background area moving image 610, the feature area moving image 620, and the feature area moving image 630. , And the latest frame image of the feature region moving image 640 are synthesized.

  Thereby, the composition unit 303 generates a composite frame image. The combining unit 303 generates a combined moving image 650 including a plurality of combined frame images. Then, the output unit 304 selects the display device 180 that displays the composite video based on the tag information acquired by the compressed video acquisition unit 301 and supplies the composite video to the selected display device 180 (604).

  FIG. 7 shows an example of the configuration of a compression control unit 700 further included in the image processing apparatus 120. The compression control unit 700 includes a boundary region specifying unit 710, an information amount calculating unit 720, an identical subject region specifying unit 730, a compression strength determining unit 740, a condition storage unit 750, a condition acquisition unit 752, and a compression control unit 760.

  The condition storage unit 750 stores different assignment conditions according to the features of the feature region, which assign different compression strengths according to the features of the feature region. Specifically, the condition storage unit 750 stores different conditions depending on the type of feature area.

  Then, the compression control unit 760 controls the compression strength at which the compression unit 240 compresses each of the plurality of feature region images based on the conditions stored in the condition storage unit 750 in accordance with the feature region features. Specifically, the compression control unit 760 determines, based on the conditions stored in the condition storage unit 750, the compression strength at which the compression unit 240 compresses each of the plurality of feature region images according to the features of the feature region. Control. More specifically, the compression control unit 760 determines, based on the conditions stored in the condition storage unit 750, the compression strength at which the compression unit 240 compresses each of the plurality of feature region images according to the type of feature region. Control.

  The compressing unit 240 compresses each of a plurality of feature region moving images including a plurality of feature region images having the same type of feature in a plurality of moving image constituent images. At this time, the compression control unit 760 includes the compression strength at which the compression unit 240 compresses each of the plurality of feature region moving images based on the conditions stored in the condition storage unit 750. Control is performed according to the feature type of the feature region image.

  Note that the condition storage unit 750 may store different image quality reduction amounts depending on the types of feature regions. Then, the image quality reduction unit 221 may reduce the image quality of each of the plurality of feature area moving images including a plurality of feature area images having the same type of feature in the plurality of moving image constituent images. At this time, based on the image quality reduction amount stored in the condition storage unit 750, the compression control unit 760 includes a feature region that each of the plurality of feature region videos includes an image quality reduction amount for which the image quality reduction unit 221 reduces the image quality. You may control according to the kind of the feature of an image, respectively.

  Note that the compression control unit 760 includes a feature that each of the plurality of feature region moving images includes a compression strength at which the compression unit 240 compresses each of the plurality of feature region moving images based on the conditions stored in the condition storage unit 750. You may control according to the kind of feature of a field image, respectively. Note that the compressed moving image acquisition unit 201 acquires a plurality of moving images obtained by imaging from different positions. For example, the compressed moving image acquisition unit 201 acquires a plurality of moving images captured by the imaging devices 100a-c.

  Then, the same subject region specifying unit 730 includes the same subject as the subject included in the feature region extracted from the moving image constituent image included in one of the plurality of moving images acquired by the moving image acquiring unit that acquires the moving image. The feature region extracted from the moving image composition image included in the other moving image is specified. Specifically, the same subject area specifying unit 730 refers to a plurality of moving images obtained by the compressed moving image data acquired by the compressed moving image acquiring unit 201 being expanded by the compressed moving image expanding unit 202, and A feature region extracted from a moving image constituent image included in another moving image including the same subject as the subject included in the characteristic region extracted from the moving image constituent image included in one of the moving images is specified. The compression control unit 760 then obtains a feature region image of at least one feature region of feature regions including the same subject specified by the same subject region specifying unit 730 based on the conditions stored in the condition storage unit 750. The compression strength that the compression unit 240 compresses is controlled according to the features of the feature region, and the compression strength that the compression unit 240 compresses the feature region image of another feature region compresses the at least one feature region image. Control higher than compression rate.

  The boundary region specifying unit 710 specifies a boundary region that is a region near the boundary between the feature region and a region other than the feature region. Then, the compression strength determination unit 740 compresses the compression strength between the compression strength for compressing the image included in the feature region and the compression strength for compressing the image included in the background region, and the compression strength for compressing the image in the boundary region. Determine as. Then, the compression unit 240 compresses the image included in the boundary region with the compression strength determined by the compression strength determination unit 740. Thus, the compression unit 240 compresses the image included in the feature region, the image included in the background region, and the image in the boundary region with different strengths.

  The compression strength determination unit 740 determines a compression strength closer to the compression strength for compressing the image included in the feature region as the compression strength for compressing the image in the region closer to the feature region. Then, the compression unit 240 compresses the image of the boundary region with the compression strength corresponding to the region of the boundary region determined by the compression strength determination unit 740.

  The compression strength determination unit 740 determines the post-compression image quality of the image included in the feature area according to the feature type of the feature area, and includes a lower image quality than the post-compression image quality of the image included in the feature area in the background area. The image quality between the image quality after compression of the image included in the feature area and the image quality after compression of the image included in the background area is determined as the image quality after compression of the image in the boundary area. decide. Then, the image quality reduction unit 221 reduces the image quality included in the feature region, the image quality included in the background region, and the image quality in the boundary region to the image quality determined by the compression strength determination unit 740.

  More specifically, the compression strength determining unit 740 determines the post-compression resolution of the image included in the feature area according to the type of feature in the feature area, and is lower than the post-compression resolution of the image included in the feature area. The resolution is determined as the compressed resolution of the image included in the background area, and the resolution between the compressed resolution of the image included in the feature area and the compressed resolution of the image included in the background area is determined as the boundary area image. Determined as the resolution after compression. Then, the image quality reduction unit 221 reduces the resolution of the image included in the feature region, the resolution of the image included in the background region, and the image resolution of the boundary region to the resolution determined by the compression strength determination unit 740.

  The information amount calculation unit 720 calculates the information amount that the subject included in the feature area has. For example, taking the ROI of the face area as an example, the information amount may be the information amount as the face of a person that the object included in the ROI has. For example, the information amount may be a value indicating the ease of recognition when recognizing a human face from an image. As an example, the information amount may be the number of pixels included in an area of a person's face or an image area indicating the person's face.

  Then, the compression unit 240 compresses the feature area image that is an image of the feature area. At this time, the compression strength determination unit 740 determines a greater compression strength at which the compression unit 240 compresses the feature region image as the amount of information increases. In many cases, an ROI in which a face is captured with a sufficiently large size maintains a quality capable of sufficiently distinguishing the face even if the image quality is somewhat degraded by compressing the ROI with a high compression strength. Therefore, when the image processing apparatus 120 of the present embodiment is applied to an actual product, the ROI is compressed at a high compression rate when, for example, the face is captured with a sufficiently large size, the amount of image data is wasted. In some cases, it is possible to prevent the increase.

  Note that the feature region detection unit 203 may detect, as a feature region, a region that includes an object having a degree of fitness that matches a predetermined condition that is greater than a predetermined value. Then, the feature amount detection unit converts a region including an object having an adaptability matching a predetermined condition relating to image content indicating an object indicating the head of the person 130 to a value larger than a predetermined value. May be detected as

  In this case, the information amount calculation unit 720 may calculate a larger amount of information as the degree of suitability that the object included in the feature region matches the condition is larger. Then, the compression strength determination unit 740 may determine a greater compression strength at which the compression unit 240 compresses the feature region as the fitness level is higher.

  Note that the feature amount detection unit may detect an area including an object having a pattern with a degree of coincidence with a predetermined pattern larger than a predetermined degree of coincidence as a characteristic area. In this case, the information amount calculation unit 720 may calculate a larger information amount as the degree of coincidence increases. Then, the compression strength determination unit 740 may determine a greater compression strength at which the compression unit 240 compresses the feature region as the degree of coincidence increases.

  Note that the information amount calculation unit 720 may specify the distance between the imaging device 100 that has captured the image and the subject included in the feature region, and may calculate a larger amount of information as the specified distance is shorter. Further, the information amount calculation unit 720 may calculate a larger information amount as the size of the feature region is larger.

  In addition, when the feature region detection unit 203 detects a plurality of feature regions having different types of features from the image, the information amount calculation unit 720 calculates the amount of information held by the subject included in each of the plurality of feature regions. It's okay. Then, the compression strength determination unit 740 may determine the degree of compression of the feature region having a larger amount of information for each of the plurality of feature region images. Then, the compression unit 240 may compress the plurality of feature region images at the degree of compression determined by the compression strength determination unit 740.

  In addition, when the feature region detection unit 203 detects a feature region from each of a plurality of moving image configuration images included in the moving image, the information amount calculation unit 720 includes the plurality of feature regions in each of the plurality of moving image configuration images. The amount of information held by the subject to be captured may be calculated. Then, the compression strength determination unit 740 may determine a greater compression strength at which the compression unit 240 compresses the feature region image as the information amount is larger for each feature region image of the plurality of moving image constituent images. Then, the compression unit 240 may compress each feature region image of the plurality of moving image constituent images with the compression strength determined by the compression strength determination unit 740.

  The same subject area specifying unit 730 specifies a plurality of feature areas including the same subject among the feature areas in each of the plurality of moving image constituent images. Specifically, the same subject area specifying unit 730 specifies an ROI that includes the same person in each of the plurality of frame images.

  Then, the compression strength determination unit 740 selects a feature region having an information amount within a predetermined information amount range including the maximum information amount among the information amounts of the plurality of feature regions specified by the same subject region specifying unit 730. A compression strength smaller than the compression strength for compressing the feature region image in at least one of the other moving image constituent images is determined as the compression strength for compressing the feature region image in the one or more moving image constituent images. Thus, for example, when there is a frame image containing a lot of information as a subject in a moving image, the frame image is recorded with high image quality, and the compression strength is set for other frame images in which the same subject is captured. There are cases where it can be increased. For this reason, the subject imaged at a more desirable timing can be recorded with high image quality while effectively reducing the amount of moving image data.

  The condition acquisition unit 752 acquires information stored in the condition storage unit 750 from the outside of the image processing apparatus 120. Specifically, the condition acquisition unit 752 may acquire different assignment conditions according to the features of the feature region, which assign different compression strengths according to the features of the feature region. Specifically, the condition acquisition unit 752 may acquire different allocation conditions depending on the type of feature region. For example, the condition acquisition unit 752 reduces image quality that varies depending on the type of feature area, such as a reduction amount of spatial resolution that varies depending on the type of feature area, and a reduction amount of temporal resolution that varies depending on the type of feature area. You may get the amount. The condition storage unit 750 may store image quality reduction amounts that are acquired by the condition acquisition unit 752 depending on the type of feature region. As described above, the condition storage unit 750 may store the allocation condition acquired by the condition acquisition unit 752.

  In addition, the condition acquisition unit 752 may acquire different allocation conditions according to the feature of the region indicated by the feature region. Specifically, the condition acquisition unit 752 may acquire different allocation conditions according to the number of feature areas, the size of the feature areas, or the position of the feature areas. In addition, the condition acquisition unit 752 may acquire different assignment conditions according to the features of the objects included in the feature region image. Specifically, the condition acquisition unit 752 may acquire different allocation conditions according to the shape of the object, the object orientation, the object movement direction, or the object movement amount included in the feature region image.

  The condition acquisition unit 752 may acquire the above-described allocation conditions from the image processing apparatus 170 or the display apparatus 180 through the communication network 110. For example, the image processing apparatus 170 may transmit information indicating the image quality level corresponding to the remaining storage capacity that can be recorded on the recording medium included in the image DB 175 to the image processing apparatus 120. For example, the image processing apparatus 170 may transmit information indicating a lower image quality level to the image processing apparatus 120 when the remaining storage capacity is smaller.

  FIG. 8 shows an example of data stored in the condition storage unit 750 in a table format. The condition storage unit 750 includes the number of ROIs indicating the number of detected ROIs, the ROI area indicating the area of the detected ROI, the ROI position indicating the position of the detected ROI, and the face indicating the orientation of the face of the person in the ROI A spatial resolution score is stored in association with each of the orientation and the number of face elements indicating the number of objects in the ROI recognized as face objects. Note that the number of face objects may be the number of objects such as eyes, mouths, noses, and the like that coincide with an object that should be included in the face with a degree of coincidence higher than a predetermined degree of coincidence.

  The spatial resolution score may be an index value indicating the image quality of the compressed frame image. Specifically, the spatial resolution score is an index indicating at least one of the resolution of the frame image after compression, the number of gradations, the number of colors, or a quantization coefficient indicating the degree of quantization in the encoding unit 230. It may be. Thus, the spatial resolution score includes real space resolution and color space resolution. Note that a larger spatial resolution score may indicate higher image quality.

  As shown in the figure, the condition storage unit 750 may store a larger spatial resolution score in association with a smaller number of ROIs. The condition storage unit 750 may store a larger spatial resolution score in association with a smaller ROI area. When the distance between a predetermined position on the frame image and the center position of the ROI is used as an index of the ROI position, the condition storage unit 750 stores a larger spatial resolution score as the distance is shorter. You can do it. As a result, the spatial resolution score increases as the ROI position is closer to a predetermined position where the face should be present.

  Further, the condition storage unit 750 stores a larger spatial resolution score as the face orientation indicated by the object included in the ROI is closer to the front oblique direction, and stores a lower spatial resolution score as the face orientation is closer to the rear. Good. The condition storage unit 750 may store a larger spatial resolution score in association with a larger number of face elements.

  When the feature region detection unit 203 detects the ROI of the face region, the compression strength determination unit 740 calculates parameters such as the number of ROIs, the ROI area, the ROI position, the face orientation, and the number of face elements for the ROI of the face region. . Then, the compression strength determination unit 740 calculates a spatial resolution score indicating the height of the spatial resolution based on the data stored in the condition storage unit 750. For example, the compression strength determination unit 740 extracts and extracts the spatial resolution score stored in the condition storage unit 750 in association with each of the number of ROIs, the ROI area, the ROI position, the face orientation, and the number of face elements. The total value of the spatial resolution score is calculated.

  Then, the compression strength determination unit 740 determines a higher spatial resolution as the calculated total value increases. Note that the compression strength determination unit 740 may determine the spatial resolution according to a table in which the total value of the spatial resolution score and the spatial resolution are associated in advance.

  In addition, the condition storage unit 750 may store, instead of the spatial resolution score, the resolution, the number of gradations, the number of colors, or the quantization coefficient itself indicating the degree of quantization in the encoding unit 230. . The spatial resolution score may be an index indicating a spatial frequency band in which the compressed frame image should have a significant frequency component.

  FIG. 9 shows an example of other data stored in the condition storage unit 750 in a table format. The condition storage unit 750 stores the time resolution score in association with the moving speed of the ROI. Note that the time resolution score may be an index value indicating the time resolution of the compressed moving image. Specifically, the temporal resolution score may be an index indicating the frame rate of the compressed moving image or the bit rate of the compressed moving image. Note that a larger temporal resolution score may indicate a higher temporal resolution. As shown in the figure, the condition storage unit 750 may store a larger spatial resolution score in association with a higher moving speed.

  When the feature region detection unit 203 detects the ROI of the face region from each frame image, the compression strength determination unit 740 calculates the moving speed of the ROI based on the position of the ROI in each frame image. Then, the compression strength determination unit 740 extracts the time resolution score stored in the condition storage unit 750 in association with the calculated moving speed. Then, the compression strength determination unit 740 determines a time resolution according to the extracted time resolution score. The condition storage unit 750 may store the compressed frame rate or the compressed bit rate itself instead of the temporal resolution score.

  8 and 9, an example of the image quality index value assigned to the ROI of the face area stored in the condition storage unit 750 is shown. However, the condition storage unit 750 may include other than the face area, For each of the ROI of the human area and the moving area, a spatial resolution score and a temporal resolution score associated with the various parameters as described above may be stored. Therefore, the spatial resolution score and the temporal resolution score are determined according to each of the plurality of objects to be included in the ROI, and thereby the image quality level of each of the plurality of ROIs in the frame image is determined.

  As described above, the condition storage unit 750 can store the image quality after compression in each of the plurality of ROIs or the code amount after compression in each of the plurality of ROIs for each type of the plurality of objects. For this reason, the condition storage unit 750 stores, for each type of a plurality of objects, a priority indicating that each of the plurality of ROIs indicates high image quality, a ratio of a code amount after compression in each of the plurality of ROIs, and the like. be able to.

  As described above, the compression strength determination unit 740 determines the compression strength for compressing the image included in the feature region according to the feature of the feature region. Specifically, the compression strength determination unit 740 determines the compression strength for compressing the image included in the feature region according to the feature type of the feature region. Further, the compression strength determination unit 740 determines a compression strength larger than the feature region as a compression strength for compressing the image included in the background region.

  The condition storage unit 750 stores a reduction amount of a spatial resolution that differs depending on the type of feature region or a reduction amount of a temporal resolution that differs depending on the type of feature region. Then, the compression control unit 760 uses the spatial resolution reduction amount that the image quality reduction unit 221 reduces the image quality based on at least one of the spatial resolution reduction amount and the temporal resolution reduction amount stored in the condition storage unit 750, or The amount of time resolution reduction is controlled according to the feature type of the feature region image included in each of the plurality of feature region moving images.

  In addition, the condition storage unit 750 stores different conditions according to the feature of the region indicated by the feature region. Then, the compression control unit 760 determines the compression strength at which the compression unit 240 compresses each of the plurality of feature region images based on the conditions stored in the condition storage unit 750 according to the feature of the region indicated by the feature region. Control.

  More specifically, the condition storage unit 750 stores different conditions according to the number of feature areas, the size of the feature areas, or the position of the feature areas. Then, the compression control unit 760 determines the compression strength at which the compression unit 240 compresses each of the plurality of feature region images based on the conditions stored in the condition storage unit 750, and the number and size of the regions indicated by the feature regions. Or control according to the position.

  In addition, the condition storage unit 750 stores different conditions according to the features of the object included in the feature region image. The compression control unit 760 converts the compression strength at which the compression unit 240 compresses each of the plurality of feature region images into the feature of the object included in the feature region image based on the conditions stored in the condition storage unit 750. Control accordingly.

  The condition storage unit 750 may store different conditions according to the shape of the object, the direction of the object, the direction of movement of the object, or the amount of movement of the object included in the feature region image. Then, the compression control unit 760 determines the compression strength at which the compression unit 240 compresses each of the plurality of feature region images based on the conditions stored in the condition storage unit 750, the shape of the object included in the feature region image, Control may be performed according to the direction of the object, the direction of movement of the object, or the amount of movement of the object.

  FIG. 10 shows an example of data stored in the information amount calculation unit 720 in a table format. The information amount calculation unit 720 associates the information amount with the face direction indicating the face direction of the person and the face element matching degree indicating the matching degree that the object included in the ROI matches the object included in the person's face. Is stored.

  For example, the information amount calculation unit 720 specifies the face direction based on the image content of the human face object included in the ROI of the face region. For example, the information amount calculation unit 720 specifies the face direction based on the positions of the objects indicating the eyes, the nose, and the mouth in the object indicating the head. Then, the information amount calculation unit 720 calculates a larger amount of information as the specified direction is closer to the front oblique direction, and calculates a smaller amount of information as the specified direction is closer to the back.

  Further, the information amount calculation unit 720 calculates the degree of coincidence between the contour of the human face object and a predetermined human face pattern, for example, by pattern matching. Then, the information amount calculation unit 720 calculates a larger information amount as the calculated degree of coincidence increases.

  As described above, the information amount calculation unit 720 is larger as the degree of suitability of the object included in the feature region that matches the predetermined condition regarding the image content indicating that the object is included in the face of the person 130 is larger. The amount of information is calculated. Further, the information amount calculation unit 720 specifies the orientation of the head of the person 130 indicated by the matching object with a degree of fitness greater than a predetermined value, and the specified orientation of the head matches the predetermined orientation. When the degree of matching is higher, a larger amount of information is calculated.

  FIG. 11 shows an example in which the compression strength determination unit 740 determines the compression strength. Here, the region in which the compression strength determination unit 740 controls the compression strength will be described using a space having the face element coincidence degree and the body element coincidence degree as coordinate axes. The body element coincidence here may be, for example, the coincidence between the outline of an object included in the frame image and a predetermined human body pattern.

  The feature region detection unit 203 calculates the body element matching degree and the face element matching degree for the objects included in the frame image. Then, the feature region detection unit 203 has a point where the value of the body element coincidence is α on the coordinate axis of the body element coincidence, and a point where the value of the face element coincidence is β on the coordinates of the face element coincidence. When the line is in the area opposite to the origin with respect to the connecting line, the area including the object is detected as the ROI. Note that the method described here is an example of a detection method for detecting ROI, and it goes without saying that the feature region detection unit 203 may detect the ROI using another method.

  Then, the compression strength determination unit 740 determines a lower compression strength that is better as the face element matching degree is smaller, on condition that the face element matching degree is smaller than a predetermined value γ. Then, when the face element matching degree is greater than or equal to γ, the compression strength determining unit 740 determines a higher compressive strength as the face element matching degree is higher.

  As described above, the compression strength determination unit 740 reduces the compression strength of an area including an object having a high probability of being a person and having a low possibility of being a person's face. For this reason, the compression strength determination unit 740 determines a low compression strength as the compression strength for compressing the ROI in which the face is not clearly captured, so that the ROI is compressed with a high compression strength to further distinguish the face. In some cases, it can be prevented from becoming difficult. In this figure, for the purpose of simplifying the explanation, the operation in which the compression strength determination unit 740 determines the compression strength using the face element matching degree which is one of the information amount indexes has been described. It goes without saying that the compression strength determination unit 740 may determine the compression strength based on the amount of information using an index other than the face element matching degree as an index.

  FIG. 12 shows an example of the time evolution of the information amount calculated by the information amount calculation unit 720. Reference numerals P1 to P15 each denote a plurality of frame images. The information amount indicated by the points indicated by reference signs P1 to P15 indicates the information amount of the subject included in the ROI including the same subject detected from each of the frame images indicated by reference signs P1-15. Referring to this figure, the amount of information increases as time passes, and the amount of information decreases after the amount of information reaches the maximum value in the ROI in the frame image indicated by P7.

  In such a case, the compression strength determination unit 740 includes the compressed ROI detected from each of the frame images P6 and P8 and the frame image P7 captured in the vicinity of the timing at which the frame image P7 is captured. Compression rate for compressing the ROI detected from each frame image so that the amount of information to be generated is greater than the information amount included in the data after compressing the ROI detected from the other frame images P1-P5, P9-P15 To decide. Further, the compression strength determination unit 740 determines the compression strength of each ROI so that the amount of information included in the data after compressing the ROI detected from the frame images P1-P5 and P9-P15 is substantially constant.

  Specifically, the compression strength determination unit 740 is a predetermined compression rate that compresses the ROI detected from the frame images P6 to P8, so that the amount of reduction in the information amount due to the compression of the ROI can be further reduced. Determine a low compression ratio. In addition, the compression strength determination unit 740 detects ROIs (frame images P1-P3 and P12-P15 having an information amount equal to or less than a predetermined value) among ROIs detected from frame images other than the frame images P6-P8. As a compression strength for compressing the generated ROI), a low compression ratio that can further reduce the amount of reduction in the information amount due to the compression is determined. Then, the compression strength determining unit 740 has an ROI (from frame images P4, P5, P9, and P10) having an information amount larger than a predetermined value among ROIs detected from frame images other than the frame images P6-P8. As the compression strength for compressing the detected ROI), the amount of information included in the compressed data is substantially the same as the amount of information included in the compressed data of the ROI detected from the frame images P1-P3 and P12-P15. Therefore, a medium compression ratio larger than the low compression ratio is determined.

  The time evolution of the information amount as shown in this figure may be obtained, for example, when a person approaches the imaging device 100 from a distance and then moves away from the imaging device 100. In such a case, the image processing device 120, for example, compresses the ROI of the frame images indicated by P6, P7, and P8 at a low compression rate to reliably record a person with high image quality, while also recording other frame images. May be recorded with some degradation in image quality. For this reason, according to the image processing apparatus 120, there are cases where the amount of moving image data can be reduced while recording a person with high image quality.

  FIG. 13 shows an example of the boundary region in the ROI. As described above, the compression strength determination unit 740 determines the compression strength for compressing the ROI according to the type of ROI, the number of ROIs, and the like. For example, the compression strength determination unit 740 determines the resolution of the ROI. The compression strength determination unit 740 further controls the resolution of the boundary area between the ROI and the background area.

  Here, it is assumed that the feature region detection unit 203 detects a region including the region 1310, the region 1311, and the region 1312 as the ROI of the face region. Then, it is assumed that the feature region detection unit 203 detects a region including the region 1320, the region 1321, and the region 1322 as a human region ROI in the vicinity of the face region. Then, as described with reference to FIG. 1-12, the compression strength determination unit 740 performs resolution A as the resolution of the ROI of the face region, resolution B as the resolution of the ROI of the human region, and resolution C as the resolution of the background region. Is determined.

  In this case, the boundary region specifying unit 710 specifies a region having a predetermined width in the peripheral portion of the ROI of the face region and a region having a predetermined width in the peripheral portion of the ROI of the human region as the boundary region. Note that the boundary region specifying unit 710 may specify the boundary region either inside the ROI or outside the ROI. In addition, the boundary area specifying unit 710 may specify a boundary area that extends over both the area inside the ROI and the area outside the ROI. Then, the compression strength determination unit 740 determines the resolution in the boundary area according to the resolution of the ROI including the boundary area and the resolution of the other ROI or the background area.

  Specifically, the compression strength determination unit 740 determines an average resolution of resolution A and resolution B as the resolution of the boundary regions 1312 and 1322 between the ROI of the face region and the ROI of the human region. Further, the compression strength determination unit 740 determines an average resolution of resolution A and resolution C as the resolution of the boundary region 1311 between the ROI of the face region and the background region. Further, the compression strength determination unit 740 determines an average resolution of resolution B and resolution C as the resolution of the boundary region 1321 between the ROI of the human region and the background region. Note that the compression strength determination unit 740 may determine the resolution that gradually changes with a predetermined gradient as the resolution in the boundary region.

  As described above, the boundary region specifying unit 710 specifies a region in the feature region in the vicinity of the boundary between the region other than the feature region and the feature region as the boundary region. Note that the boundary region specifying unit 710 may specify a larger boundary region when the feature region is larger. The boundary region specifying unit 710 specifies a larger boundary region when the difference between the compression strength for compressing the image included in the feature region and the compression strength for compressing the image included in the background feature is larger. Also good.

  In addition, the boundary area specifying unit 710 specifies an area in the vicinity of the boundary between one feature area and the other feature area as a boundary area for adjacent feature areas of the plurality of feature areas. Then, the compression strength determination unit 740 determines a compression strength between a compression strength for compressing an image included in one feature region and a compression strength for compressing an image included in the other feature region, and the one feature region and the other. This is determined as the compression strength for compressing the image in the boundary region in the vicinity of the boundary with the feature region. Then, the compression unit 240 compresses the image of the boundary region near the boundary between one feature region and the other feature region with the compression strength determined by the compression strength determination unit 740.

  In this way, the compression strength determination unit 740 determines an intermediate resolution between the resolution of the ROI and the resolution of the background area as the resolution of the boundary area. For this reason, in the frame image included in the synthesized moving image generated by the image processing apparatus 170, the difference between the resolution of the ROI and the resolution of the background area may be made inconspicuous.

  FIG. 14 shows another example of a block configuration in the image processing apparatus 120. The image processing apparatus 120 includes a compressed moving image acquisition unit 201, a compressed moving image expansion unit 202, a feature region detection unit 203, a compression unit 240, a compression control unit 700, an association processing unit 206, and an output unit 207. The image dividing unit 204 includes an input moving image quality control unit 280, an image quality reduction unit 281, and a plurality of inter-layer difference compression units 282a-d (hereinafter collectively referred to as inter-layer difference compression units 282).

  Note that the functions and operations of the compressed video acquisition unit 201, the compressed video expansion unit 202, the feature region detection unit 203, the compression control unit 700, the association processing unit 206, and the output unit 207 are related to FIGS. 2 to 13, respectively. Are substantially the same as the functions and operations of the compressed moving image acquisition unit 201, the compressed moving image expansion unit 202, the feature region detection unit 203, the compression control unit 700, the association processing unit 206, and the output unit 207 described above. The description is omitted except for.

  The input moving image quality control unit 280 controls the image quality of the feature region and the image quality of the region other than the feature region in accordance with the feature amount of the feature region in each of the plurality of moving image constituent images generated by the compressed moving image decompression unit 202.

  The image quality reduction unit 281 generates a plurality of moving images having different predetermined image quality by reducing the image quality of the moving images. Then, the image quality reduction unit 281 provides the generated moving images with different image quality to the inter-layer difference compression unit 282. Specifically, the image quality reduction unit 281 generates moving images having different image quality by reducing the frame rate of moving images or by reducing the resolution of moving image constituent images included in the moving images. Then, the inter-layer difference compression unit 282 acquires a moving image having a predetermined image quality from the image quality reduction unit 281 and compresses the acquired moving image. Note that the inter-layer difference compression unit 282 compresses moving images having different image quality. Note that the moving image constituent image included in the moving image supplied to the inter-layer difference compression unit 282a has lower image quality than the characteristic area image that is the moving image constituent image included in the moving image supplied to the inter-layer difference compression unit 282b-d. That is, the image quality reduction unit 281 generates a low-quality image having a lower image quality than the feature region image supplied to the inter-layer difference compression unit 282b-d, and supplies the low-quality image to the inter-layer difference compression unit 282a.

  As described above, the inter-layer difference compression unit 282a acquires the moving image constituent image having a lower resolution than the moving image constituent image received by any of the inter-layer difference compression units 282b-d from the image quality reduction unit 281 and compresses it. Note that the inter-layer difference compression unit 282b-d acquires, from the image quality reduction unit 281, a moving image constituent image having a lower resolution in the order of the inter-layer difference compression unit 282b, the inter-layer difference compression unit 282c, and the inter-layer difference compression unit 282d. Compress.

  The inter-layer difference compression unit 282b expands the moving image constituent image compressed by the inter-layer difference compression unit 282a, and the moving image constituent image obtained by the expansion is the same as the resolution of the moving image constituent image acquired from the image quality reduction unit 281. Enlarge to the resolution. Then, the inter-layer difference compression unit 282b compresses the difference image between the moving image constituent image obtained by the enlargement and the moving image constituent image acquired from the image quality reduction unit 281. Note that the inter-layer difference compression unit 282b generates and compresses a difference image that has a difference value in the feature region but does not have a difference value in a region other than the feature region.

  Further, the inter-layer difference compression unit 282c expands the moving image constituent image compressed by the inter-layer difference compressing unit 282b, and the resolution of the moving image constituent image obtained from the image quality reduction unit 281 is obtained by expanding the moving image constituent image. To the same resolution. Then, the inter-layer difference compression unit 282c compresses the difference image between the moving image constituent image obtained by the enlargement and the moving image constituent image acquired from the image quality reduction unit 281. Note that the inter-layer difference compression unit 282c has a difference value in at least a part of the plurality of feature areas according to the feature amount of the feature area, but has a difference value in an area other than the at least some of the feature areas. A difference image having no value is generated and compressed.

  In addition, the inter-layer difference compression unit 282d expands the moving image constituent image compressed by the inter-layer difference compression unit 282c. Then, the hierarchical difference compression unit 282d expands the moving image constituent image obtained by the decompression to the same resolution as the resolution of the moving image constituent image acquired from the input moving image quality control unit 280. Then, the inter-layer difference compression unit 282d compresses a difference image between the enlarged moving image constituent image and the moving image constituent image acquired from the input moving image quality control unit 280. Note that the inter-layer difference compression unit 282d has a difference value in at least a part of the plurality of feature areas according to the feature amount of the feature area, but has a difference value in an area other than the at least some of the feature areas. A difference image having no value is generated and compressed.

  In this way, the inter-layer difference compression unit 282b-d is between the moving image constituent image received from the input moving image quality control unit 280 or the image quality reducing unit 281 and the moving image constituent image obtained by enlarging the lower moving image constituent image. The difference image obtained by taking the difference is compressed. Then, the association processing unit 206 associates the compressed moving image data including a plurality of moving image constituent images obtained by compression by the inter-layer difference compression units 282a-d with information for specifying the feature region. Then, the output unit 207 transmits the compressed moving image data associated with the information for specifying the feature area by the association processing unit 206 to the image processing apparatus 170. In this way, the image processing apparatus 120 can provide a moving image that is scalable and encoded according to the feature amount of the feature region.

  FIG. 15 shows an example of the block configuration of the inter-layer difference compression units 282a and b. The inter-layer difference compression unit 282a includes a motion analysis unit 285a, a motion encoding unit 286a, a difference processing unit 287a, and an encoding unit 288a. The motion analysis unit 285a includes a difference target area determination unit 294a and a position difference information generation unit 295a. The difference processing unit 287a includes a difference pixel image generation unit 296a, a spatial frequency domain conversion unit 297a, and a quantization unit 298a.

  The inter-layer difference compression unit 282b includes a motion analysis unit 285b, a motion encoding unit 286b, a difference processing unit 287b, an image enlargement unit 293b, an image decoding unit 292b, a pixel value changing unit 291b, and an encoding unit 288b. The motion analysis unit 285b includes a difference target area determination unit 294b and a position difference information generation unit 295b. The difference processing unit 287b includes a difference pixel image generation unit 296b, a spatial frequency domain conversion unit 297b, a quantization unit 298b, and a frequency domain image quality conversion unit 299b. Note that the inter-layer difference compression unit 282c and the inter-layer difference compression unit 282d have substantially the same constituent elements as the inter-layer difference compression unit 282b, and a description thereof will be omitted.

  The function and operation of each component of the inter-layer difference compression unit 282a will be described below. The motion analysis unit 285a analyzes the motion over the plurality of moving image constituent images based on the image contents of the plurality of moving image constituent images received from the image quality reduction unit 281 and compresses the moving image constituent images based on the motion To decide.

  Specifically, the difference target area determination unit 294a becomes a difference target in the case where a moving picture constituent image is compressed by a difference from another moving picture constituent image based on the pixel values of partial areas that span a plurality of moving picture constituent images. A partial area in another moving image constituent image is determined. The difference target area determination unit 294a supplies pixel information of the compression target partial area and pixel information of the difference target partial area to the difference processing unit 287a.

  In addition, the position difference information generation unit 295a generates position difference information indicating a position difference between the partial area to be compressed by the difference and the partial area to be the difference target. Specifically, the position difference information generation unit 295a generates a motion vector used for motion compensation. Then, the position difference information generation unit 295a supplies the generated position difference information to the motion encoding unit 286a.

  The motion encoding unit 286a encodes the position difference information supplied from the position difference information generation unit 295a and supplies the encoded information to the association processing unit 206. For example, the motion encoding unit 286 encodes the difference between the position difference information in the adjacent partial areas and supplies the encoded difference to the association processing unit 206.

  The difference processing unit 287a compresses the image of the partial area to be compressed based on the difference between the pixel information of the partial area to be compressed and the pixel information of the partial area to be difference received from the motion analysis unit 285a. Specifically, the difference pixel image generation unit 296a generates a difference pixel image based on the difference between the pixel information of the compression target partial area and the pixel information of the difference target partial area.

  Then, the spatial frequency domain conversion unit 297a converts the difference pixel image into a spatial frequency domain for each partial region. Specifically, the spatial frequency domain conversion unit 297a converts each partial area in the difference pixel image into a spatial frequency domain by discrete cosine transform (DCT). The spatial frequency domain transform unit 297a may transform the difference pixel image into a spatial frequency domain for each partial region by various frequency transforms such as Hadamard transform or wavelet transform.

  When the motion analysis unit 285a determines that the compression is not performed due to a difference from another partial region of the moving image constituent image, the difference processing unit 287a converts the pixel information of the partial region to be compressed into the spatial frequency domain conversion unit. 297a. The spatial frequency domain conversion unit 297a converts the pixel information into a spatial frequency domain for each partial area as described above.

  The quantization unit 298a quantizes the transform coefficient obtained by the spatial frequency domain transform unit 297a transforming into the spatial frequency domain. Then, the encoding unit 288a compresses the transform coefficient quantized by the quantization unit 298a by encoding. For example, the encoding unit 288 encodes the transform coefficient quantized by the quantization unit 298a by entropy coding such as Huffman coding or arithmetic coding. Then, the encoding unit 288a supplies the encoded moving image to the association processing unit 206.

  Hereinafter, functions and operations of each component included in the inter-layer difference compression unit 282b will be described. Among the components included in the inter-layer difference compression unit 282b, the components denoted by the same reference numerals as those included in the inter-layer difference compression unit 282a are similar in function to the components included in the inter-layer difference compression unit 282a and Since it operates, description is abbreviate | omitted except the difference.

  Similar to the difference target region determination unit 294a, the difference target region determination unit 294b should take a difference from the compression target partial region included in the video component image for each of the plurality of video component images received from the image quality reduction unit 281. Then, a partial area in another moving image constituent image is specified. In this manner, the difference target area determination unit 294b determines a feature area partial area that is a partial area in a feature area image generated from another moving image constituent image, which should take a difference from the feature area image. Then, the difference target area determination unit 294b supplies pixel information of the compression target partial area and pixel information of the difference target partial area to the pixel value changing unit 291b.

  In addition, the image decoding unit 292b acquires a moving image constituent image from the encoding unit 288a, and acquires positional difference information from the motion encoding unit 286a. And the image decoding part 292b decodes the moving image structure image acquired from the encoding part 288a using the positional difference information acquired from the motion encoding part 286a. The image decoding unit 292b may acquire and decode the moving image constituent image quantized by the quantization unit 298a, or may acquire and decode the moving image constituent image encoded by the encoding unit 288a.

  The image enlarging unit 293b generates an enlarged image by enlarging the moving image constituent image decoded by the image decoding unit 292b. Then, the pixel value changing unit 291b does not change the pixel value of the partial region including the characteristic region among the partial regions determined by the difference target region determining unit 294b, and changes the pixel value of the partial region not included in the characteristic region. The pixel value of the partial area in the enlarged image is replaced. As described above, the pixel value changing unit 291b generates a feature area image in which the pixel values of the area other than the feature area are replaced with the pixel values of the enlarged image from the input moving image constituent image.

  The difference processing unit 287b receives, from the pixel value changing unit 291b, the feature region image to be compressed, the image information of the partial region that is the difference target of the partial region included in the feature region image, and the enlarged image. Then, the difference processing unit 287b encodes each of a plurality of partial regions included in the compression target feature region image using pixel information of the same feature region image (hereinafter referred to as intra coding). Whether encoding is performed using a difference from a partial area to be included in another moving image constituent image (hereinafter referred to as inter encoding), or encoding is performed using a difference from an enlarged image (hereinafter referred to as a hierarchy). Called inter-coding). At this time, the difference processing unit 287b preferentially selects an encoding method with a smaller code amount after encoding. Note that in regions other than the feature region, the pixel values are replaced so as not to have a difference, so inter-layer coding is selected. Therefore, the case where inter-layer coding is selected will be described first, but in addition, the case where inter coding and intra coding are selected will also be described.

  When the inter-layer coding is selected, the difference pixel image generation unit 296b generates a difference pixel image indicating a difference in pixel values between the feature region image and the enlarged image. Specifically, the difference pixel image generation unit 296b generates a difference pixel image based on the difference between the feature region image in which the pixel values of the regions other than the feature region are replaced and the enlarged image. In the feature region image, since the pixel values of the regions other than the feature region are replaced with the pixel values of the enlarged image, the difference pixel image generation unit 296b performs a difference between the feature region image and the enlarged image in the feature region. A difference pixel image having a difference value of pixel values and not having a difference value of pixel values in an area other than the feature area can be generated.

  When inter coding is selected, the difference pixel image generation unit 296b generates a feature region image generated by the pixel value change unit 291b and a feature region image generated by the pixel value change unit 291b from another moving image configuration image. The difference between is taken. Specifically, the difference pixel image generation unit 296b obtains a difference between the partial region image included in the feature region and the difference target partial region image determined by the difference target region determination unit 294b for the partial region. . In the feature region image, since the pixel values of the region other than the feature region are replaced with the pixel values of the enlarged image, the difference pixel image generation unit 296b determines the difference target region determination unit 294b in the partial region included in the feature region. A difference pixel image having a difference value of the pixel value between the partial area and the partial area determined by the difference target area determination unit 294b in the area other than the feature area is generated. .

  When intra coding is selected, the difference pixel image generation unit 296b uses the partial region image included in each feature region image as the pixel value of another region of the feature region image or the pixel of the same partial region. A difference pixel image is generated by taking a difference from the value.

  The spatial frequency domain conversion unit 297b converts the difference pixel image into a spatial frequency domain for each partial region. Specifically, the spatial frequency domain transform unit 297b converts the difference value indicated by the difference pixel image into a partial region by discrete cosine transform (DCT), Hadamard transform, wavelet transform, or the like, similar to the spatial frequency domain transform unit 297a. Each is converted to the spatial frequency domain. Similar to the quantization unit 298a, the quantization unit 298b quantizes the transform coefficient obtained by the spatial frequency domain transform unit 297b transforming into the spatial frequency domain.

  Then, the frequency domain image quality conversion unit 299b includes at least some partial regions including regions other than the feature region among the spatial frequency components for each partial region obtained by being converted into the spatial frequency region by the spatial frequency region conversion unit 297b. A feature region difference image or a feature region difference image is generated by reducing the data amount of the spatial frequency component. Specifically, the frequency domain image quality conversion unit 299b reduces the size of a conversion coefficient indicating a frequency component higher than a predetermined frequency. The frequency domain image quality conversion unit 299b may set the conversion coefficient indicating a frequency component higher than a predetermined frequency to zero.

  As described above, the difference processing unit 287b has a spatial frequency component in which the difference between the feature region image and the enlarged image in the feature region is converted into the spatial frequency region, and the spatial frequency component data in the region other than the feature region. A feature region difference image with a reduced amount is generated. Then, the encoding unit 288b encodes the feature region difference image generated by the difference processing unit 287b.

  In addition, as described above, the difference processing unit 287b generates a feature region difference image that indicates a difference image between the feature region image in the feature region image and the feature region image in the low-quality image. More specifically, the difference processing unit 287 generates a feature region difference image between the feature region image in the feature region image and the enlarged image of the feature region image in the low-quality image.

  In the above description, the pixel value changing unit 291b has at least a region other than the feature region in the difference pixel image (a region other than the feature region having a predetermined feature type and having a higher resolution than the feature region. In order to set the difference value to 0 in the region other than the feature region having the type of feature to have, the pixel value other than the feature region is replaced with the pixel value of the enlarged image. However, the difference value in the difference pixel image can be set to 0 by other methods.

  For example, the pixel value changing unit 291b converts a pixel value of a region other than the feature region of the moving image constituent image acquired from the image quality reduction unit 281 into a predetermined pixel value, and also has the same image region as the region other than the feature region in the enlarged image May be converted into the predetermined pixel value. Even in this case, the difference value of the region other than the feature region in the difference pixel image can be set to 0, and the information amount of the region other than the feature region can be substantially reduced.

  In this way, the pixel value changing unit 291b generates a feature region image in which the pixel values of the regions other than the feature region and the pixel values of the region other than the feature region in the enlarged image are replaced with predetermined values from the moving image constituent image. . Then, the difference pixel image generation unit 296 generates a difference pixel image based on the difference between the feature region image and the enlarged image in which the pixel values of the regions other than the feature region are replaced.

  Note that the pixel value changing unit 291b sends the pixel values of the region other than the feature region of the moving image constituent image acquired from the image quality reducing unit 281 to the lower layer inter-layer difference compression unit 282 (for example, the inter-layer difference compression unit 282a). The provided moving image constituent image may be replaced with the pixel value of the same region in the enlarged image. Even in this case, the difference value in the difference pixel image can be made substantially zero, and the information amount of the area other than the feature area can be substantially reduced.

  Note that the position difference information generation unit 295b generates position difference information indicating the position difference of the partial area that is the difference target and is included in the area other than the feature area. Specifically, the positional difference information generation unit 295b indicates the positional difference between the partial region to be compressed by the difference and the differential target partial region that is the partial region that is the differential target, as in the positional difference information generation unit 295a. Generate position difference information. The position difference information includes a motion vector in motion compensation.

  The position difference information changing unit 290b changes the position difference information so that the position difference information indicates that a partial area included in an area other than the feature area is different from the partial area at the same position. Specifically, the position difference information changing unit 290b converts the position difference information in the partial area included in the area other than the feature area into information indicating that there is no position difference. Also, the position difference information changing unit 290b acquires the position difference information from the motion encoding unit 286a, and converts the position difference information in the partial area included in the area other than the feature area into information indicating that there is no position difference. . Specifically, the position difference information changing unit 290b sets the magnitude of the motion vector in an area other than the feature area to zero. For example, the position difference information changing unit 290b sets the size of the motion vector received from the position difference information generating unit 295b to 0 for the region other than the feature region and the size of the motion vector received from the motion encoding unit 286a. Set to 0.

  Then, the motion encoding unit 286b encodes the position difference information. Specifically, similarly to the motion encoding unit 286a, the motion encoding unit 286b encodes a difference between position difference information in adjacent partial regions. The position difference information encoded by the motion encoding unit 286b is supplied to the association processing unit 206.

  In the present embodiment, the position difference information changing unit 290 converts the position difference information of the region other than the feature region, but the position difference information changing unit 290b is encoded by the motion encoding unit 286b. Above, position difference information of regions other than the feature region may be converted. That is, the position difference information changing unit 290b changes the position difference information encoded by the motion encoding unit 286 to indicate that a difference is taken between the partial areas other than the feature areas and the partial areas at the same position. May be.

  Note that the encoding unit 288b may generate encoded data having no difference information in a region other than the feature region. Specifically, encoded data that does not have difference information of partial areas included in areas other than the characteristic areas may be generated. In addition, the motion encoding unit 286b may generate encoded data that does not have position difference information in a partial region included in a region other than the feature region. As described above, the encoding unit 288b and the motion encoding unit 286b indicate that the image content of the region other than the feature region is the same as the image content of the same region in the other moving image constituent images, and the difference information and the position difference information. The encoded data shown by not having is generated. For example, the encoding unit 288b and the motion encoding unit 286b indicate the type of the partial area indicating that the image content of the partial area included in the area other than the feature area is the same as the image content of the same area in the other moving image constituent images. May be generated.

  For example, the encoding unit 288b and the motion encoding unit 286b are encoded in the encoding mode indicating that the partial region included in the region other than the feature region is simple inter-frame prediction and has no transform coefficient. Encoded data including the type of the partial area indicating the partial area may be generated. For example, the type of the partial area may be a type corresponding to NonMC NotCoded in MPEG encoding. As described above, the encoding unit 288b and the motion encoding unit 286b generate encoded data having no information indicating that the magnitude of the motion vector and the difference information are 0, so that a moving image constituent image after encoding is generated. Can be further reduced. The inter-layer difference compression unit 282b selects a prediction mode capable of minimizing the rate / distortion cost based on the Lagrange's undetermined multiplier method when determining a prediction mode including the encoding mode. It's okay.

  Note that the inter-layer difference compression unit 282c and the inter-layer difference compression unit 282d have components having the same functions as the inter-layer difference compression unit 282b. In the following description, components having the same names as the inter-layer difference compression unit 282 included in the inter-layer difference compression unit 282c and the inter-layer difference compression unit 282d are denoted by the same reference numerals. Then, it is distinguished from which of the inter-layer difference compression units 282b-d the component included by the end (b, c, d) of the code.

  For example, the motion analysis unit 285c is one of the components included in the inter-layer difference compression unit 282c, and the motion analysis unit 285d is one of the components included in the inter-layer difference compression unit 282d. Note that in the following description, the constituent elements having no alphabetic character at the end of the reference numerals indicate the entire constituent elements having the reference numerals included in the inter-layer difference compression unit 282b-d. For example, the pixel value changing unit 291 indicates pixel value changing units 291b-d.

  Note that the functions and operations of the inter-layer difference compression unit 282c and the inter-layer difference compression unit 282d and the functions and operations of the inter-layer difference compression unit 282b are acquired from the image quality reduction unit 281 and processed. The position difference information changing units 290c and 290c and the image decoding units 292c and d receive the position difference information and moving image constituent images for difference processing from other inter-layer difference compressing units 282 that process moving images with lower image quality. The place to get is different.

  More specifically, the position difference information changing unit 290c acquires the position difference information from the motion encoding unit 286b, and converts the acquired position difference information. In addition, the image decoding unit 292c acquires the position difference information from the motion encoding unit 286b, acquires the moving image configuration image from the encoding unit 288b, and decodes the acquired moving image configuration image using the position difference information. . Also, the position difference information changing unit 290d acquires the position difference information from the motion encoding unit 286c, and converts the acquired position difference information. Further, the image decoding unit 292d acquires the position difference information from the motion encoding unit 286c, acquires the moving image constituent image from the encoding unit 288c, and decodes the acquired moving image constituent image using the position difference information. .

  Note that the feature region detection unit 203 detects a plurality of feature regions having different types of features from the input moving image composition image. In this case, the image quality reduction unit 281 generates one feature region image from the moving image constituent image by reducing the resolution in the feature region having one feature type, and in the feature region having another feature type. Another feature region image having a higher resolution than the feature region image is generated from the moving image constituent image. The inter-layer difference compression unit 282b-c is prepared for each type of feature region, and each of the inter-layer difference compression units 282b-c is connected to a region other than the feature region in at least a predetermined type of feature region. Compress feature area images with resolution differences between them.

  Specifically, the inter-layer difference compression unit 282b processes a low-resolution feature region image having the lowest resolution in a region including all feature regions. The inter-layer difference compression unit 282c processes a medium-resolution feature region image having a higher resolution than the low-resolution feature region image in a predetermined type of feature region. Then, the inter-layer difference compression unit 282d processes a high-resolution feature area image having a high resolution in another predetermined type of feature area.

  In this way, the difference processing unit 287 converts the difference between one feature region image and an enlarged image into a spatial frequency region in a feature region having one feature type and a feature region having another feature type. A feature region difference image is generated in which the amount of spatial frequency component data is reduced in regions other than the feature region having the spatial frequency component and having one feature type and the other feature type.

  As described above, the difference processing unit 287 has a spatial frequency component in which a difference between one feature region image and an enlarged image in a feature region having one feature type is converted into a spatial frequency region, A feature region difference image in which the amount of spatial frequency component data is reduced in a region other than a feature region having one feature type is generated, and another feature region image and another feature region in a feature region having another feature type are generated. The difference between the feature area image and the image obtained by enlarging the feature area has a spatial frequency component converted into the spatial frequency domain, and the data amount of the spatial frequency component is reduced in areas other than the feature areas having other feature types. The difference image between the feature regions thus generated is generated.

  Also, the encoding unit 288 encodes the feature region difference image, the feature region difference image, and the low-quality image. Then, the association processing unit 206 uses the positional difference information encoded by the motion encoding units 286a-d and the moving image constituent image encoded by the encoding units 288a-d (for example, a low-quality image, a feature region difference image). , And the difference image between feature areas) are associated with information specifying the feature area.

  As described above, the inter-layer difference compression unit 282a generates a moving image constituent image having a low image quality of the entire image region including the feature region, that is, a moving image constituent image having a low spatial frequency component of the input moving image constituent image. . Then, the inter-layer difference compression unit 282b generates a moving image composition image having a higher frequency component than the inter-layer difference compression unit 282a and having a lower frequency component than the inter-layer difference compression unit 282c. Then, the inter-layer difference compression unit 282b generates a moving image constituent image in which a difference value with respect to the moving image constituent image generated by the inter-layer difference compression unit 282a is reduced in a region other than the feature region.

  Similarly, the inter-layer difference compression unit 282c generates a moving image composition image having a higher frequency component than the inter-layer difference compression unit 282b and having a lower frequency component than the inter-layer difference compression unit 282d. . The inter-layer difference compression unit 282d generates a moving image constituent image having a higher frequency component than the inter-layer difference compression unit 282c. Then, the inter-layer difference compressing unit 282c and the inter-layer difference compressing unit 282d are the moving image constituent images in which the difference values with respect to the moving image constituent images generated by the inter-layer difference compressing units 282b and c are reduced in the regions other than the feature regions, respectively. Generate.

  In this way, each of the inter-layer difference compression units 282b-d processes a moving image having a higher image quality than the other regions for a feature region having a predetermined feature type, thereby improving the image quality according to the feature type. Can provide different videos to the outside. At this time, since the inter-layer difference compression unit 282b-d compresses by the difference with the low-quality moving image constituent image processed by the other inter-layer difference compression unit 282, it can be compressed efficiently.

  Note that the feature region detection unit 203 may calculate a certainty factor indicating the degree of reliability detected as the feature region for each of the plurality of feature regions when detecting the feature amounts of the plurality of feature regions. Then, the inter-layer difference compression unit 282b-d compresses the image of the feature region whose resolution is adjusted according to the feature amount and the certainty factor of each of the plurality of feature regions. For example, the image quality reduction unit 281 may adjust the resolution of each image of the plurality of feature regions in accordance with the feature amount and the certainty factor and supply the images to the inter-layer difference compression unit 282. For example, the image quality reduction unit 281 may set each image of the plurality of feature regions to a higher resolution than a predetermined resolution according to the feature amount as the certainty factor is lower.

  Note that the image processing device 120 encodes hierarchically by encoding image differences between a plurality of layers having different resolutions as described above. As is clear from this, a part of the compression method by the image processing apparatus 120 is H.264. It is clear that a compression scheme according to H.264 / SVC is included. When the image processing apparatus 170 decompresses such a hierarchical compressed moving image, a plurality of moving image constituent images can be obtained for each layer by decoding the moving image data of each layer for each layer. it can. Then, the image processing apparatus 170 adds the region encoded by the difference between layers in the obtained moving image composition image to the region on the moving image composition image of the other layer that is the difference target. Thereby, the image processing apparatus 170 can generate a plurality of moving image constituent images included in the moving image for each of the plurality of layers.

  FIG. 16 shows an example of an image processing system 20 according to another embodiment. The configuration of the image processing system 20 in this embodiment is different from the image processing device 120 except that the imaging devices 100a-d have image processing units 1604a-d (hereinafter collectively referred to as image processing units 1604). Thus, the configuration is the same as that of the image processing system 10 described in FIG.

  The image processing unit 1604 has components other than the compressed moving image acquisition unit 201 and the compressed moving image decompression unit 202 among the components included in the image processing apparatus 120. The functions and operations of the constituent elements included in the image processing unit 1604 are changed to that the constituent elements included in the image processing device 120 process the captured moving image obtained by the decompression processing by the compressed moving image decompression unit 202. The functions and operations of the components included in the image processing apparatus 120 may be substantially the same except that the captured moving image captured by the imaging unit 102 is processed. Also in the image processing system 20 having such a configuration, the same effects as those described in relation to the image processing system 10 in FIGS. 1 to 15 can be obtained.

  In addition to the image processing apparatus 170, the condition acquisition unit 752 included in the imaging apparatus 100 may acquire the above-described allocation conditions from a user who uses the imaging apparatus 100. For example, the imaging apparatus 100 may include a display device that displays a setting screen, and the display device may display a setting screen that sets assignment conditions for assigning different image quality for each object. Then, the condition acquisition unit 752 may acquire an allocation condition from the user through the setting screen. For example, the condition acquisition unit 752 may acquire the allocation condition by acquiring operation information by the user during a period in which the setting screen is displayed on the display device.

  Note that the image processing unit 1604 may acquire a captured moving image including a plurality of moving image constituent images expressed in the RAW format from the imaging unit 102. Then, the image processing unit 1604 may detect one or more feature regions from each of the plurality of moving image constituent images represented in the RAW format. Further, the image processing unit 1604 may compress a plurality of moving image constituent images expressed in the RAW format included in the acquired captured moving image in the RAW format. At this time, the image processing unit 1604 can compress the captured moving image by the compression method described in relation to the operation of the image processing apparatus 120 with reference to FIGS.

  The image processing apparatus 170 can acquire a plurality of moving image constituent images expressed in the RAW format by expanding the compressed moving image acquired from the image processing unit 1604. Then, the image processing apparatus 170 performs a synchronization process on a plurality of moving image constituent images expressed in the RAW format acquired by decompression, for example, for each of a plurality of regions including a region other than a feature region and a plurality of feature regions. May be applied. At this time, the image processing apparatus 170 may perform synchronization processing with higher accuracy in the feature region than in the region other than the feature region.

  Note that the image processing apparatus 170 may perform super-resolution processing on the image of the feature region in the moving image constituent image obtained by the synchronization processing. Super-resolution processing in the image processing apparatus 170 includes super-resolution processing based on principal component analysis as described in JP-A-2006-350498, or subject as described in JP-A-2004-88615. The super-resolution processing based on the movement of the can be illustrated.

  Note that the image processing apparatus 170 may perform super-resolution processing for each object included in the feature region. For example, when the feature region includes a human face image, the image processing apparatus 170 may perform super-resolution processing for each face part (for example, eyes, nose, mouth, etc.) as an example of the object. In this case, the image processing apparatus 170 acquires learning data such as a model based on principal component analysis (PCA) as described in JP-A-2006-350498 for each facial part (for example, eyes, nose, mouth, etc.). Remember it. Then, the image processing apparatus 170 may perform super-resolution processing on the image of each face part using the learning data selected for each face part included in the feature region.

  In this way, the image processing apparatus 170 can reconstruct the image of the feature region using principal component analysis (PCA). As an image reconstruction method by the image processing apparatus 170 and a learning method for the image reconstruction, in addition to learning and image reconstruction by principal component analysis (PCA), local preserving projection (LPP) , Linear discriminant analysis (LDA), independent component analysis (ICA), multidimensional scaling (MDS), support vector machine (support vector regression), neural network, hidden Markov model, Bay Inference, maximum posterior probability estimation, iterative backprojection, Wavelet transform, locally linear embedding (locally linea) Techniques such as reembedding (LLE) and Markov random field (MRF) can be used.

  In addition to the model as described in Japanese Patent Application Laid-Open No. 2006-350498, the learning data includes low frequency components and high frequency components of the object image respectively extracted from a large number of sample images of the object. Good. Here, by clustering the low frequency components of the object image for each of the plurality of object types by the K-means method or the like, the low frequency components of the object image in each of the plurality of object types are converted into a plurality of clusters. It may be clustered. In addition, a typical low frequency component (for example, centroid value) may be determined for each cluster.

  Then, the image processing device 170 extracts a low frequency component from the image of the object included in the feature region in the moving image constituent image. Then, the image processing apparatus 170 determines, as a representative low-frequency component, a value that matches the extracted low-frequency component among the low-frequency component clusters extracted from the sample image of the extracted object type object. Identify the cluster. Then, the image processing apparatus 170 identifies a cluster of high frequency components associated with the low frequency component included in the identified cluster. In this way, the image processing apparatus 170 can specify a cluster of high-frequency components that are correlated with the low-frequency components extracted from the objects included in the moving image composition image.

  Then, the image processing apparatus 170 may convert the image of the object into a high-quality image with higher image quality using high-frequency components that represent the specified cluster of high-frequency components. For example, the image processing apparatus 170 may add the high-frequency component selected for each object with a weight according to the distance from the center of each object to the processing target position on the face to the object image. Note that the representative high-frequency component may be generated by closed-loop learning. As described above, the image processing apparatus 170 selects and uses desired learning data for each object from learning data generated by learning for each object. There are cases where image quality can be improved. Note that, also in the configuration of the image processing system 10 described with reference to FIGS. 1 to 15, the image processing apparatus 170 can perform the above-described super-resolution processing on the image of the feature region.

  Note that in super-resolution processing based on principal component analysis as described in JP-A-2006-350498, an image of an object is represented by a principal component vector and a weighting coefficient. The data amounts of these weighting coefficients and principal component vectors are significantly smaller than the data amount of pixel data included in the object image itself. Therefore, the image processing unit 1604 may calculate the weighting coefficient from the image of the object included in the feature region in the compression processing for compressing the image of the feature region in the plurality of moving image constituent images acquired from the imaging unit 102. In other words, the image processing unit 1604 can compress the image of the object included in the feature region by representing the principal component vector and the weighting coefficient. Then, the image processing unit 1604 may transmit the principal component vector and the weighting coefficient to the image processing device 170.

  In this case, the image processing apparatus 170 can reconstruct an image of an object included in the feature region using the principal component vector and the weighting coefficient acquired from the image processing unit 1604. Note that the image processing unit 1604 is included in the feature region using a model that represents an object with various feature parameters in addition to a model based on principal component analysis as described in JP-A-2006-350498. Needless to say, an image of an object can be compressed.

  FIG. 17 shows an exemplary hardware configuration of the image processing apparatus 120 and the image processing apparatus 170. The image processing device 120 and the image processing device 170 include a CPU peripheral unit, an input / output unit, and a legacy input / output unit. The CPU peripheral section includes a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 that are connected to each other by a host controller 1582. The input / output unit includes a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 that are connected to the host controller 1582 by the input / output controller 1584. The legacy input / output unit includes a ROM 1510, a flexible disk drive 1550, and an input / output chip 1570 connected to the input / output controller 1584.

  The host controller 1582 connects the RAM 1520, the CPU 1505 that accesses the RAM 1520 at a high transfer rate, and the graphic controller 1575. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 to control each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505. The communication interface 1530 is connected to the network communication device 1598 to transmit / receive programs or data. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program that is executed when the radiation imaging system is started up, a program that depends on the hardware of the radiation imaging system, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via the flexible disk drive 1550 or a parallel port, serial port, keyboard port, mouse port, and the like.

  A program executed by the CPU 1505 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program stored in the recording medium may be compressed or uncompressed. The program is installed in the hard disk drive 1540 from the recording medium, read into the RAM 1520, and executed by the CPU 1505. The program executed by the CPU 1505 causes the image processing apparatus 120 to execute the compressed moving image acquisition unit 201, the compressed moving image decompression unit 202, the feature region detection unit 203, the image division unit 204, and the image generation described with reference to FIGS. Unit 205, fixed value converting unit 211, image quality reducing unit 221, encoding unit 231, association processing unit 206, output unit 207, boundary region specifying unit 710, information amount calculating unit 720, same subject region specifying unit 730, compression strength It functions as a determination unit 740, a condition storage unit 750, a condition acquisition unit 752, and a compression control unit 760. Further, the program executed by the CPU 1505 causes the image processing apparatus 170 to be connected to the compressed moving image acquisition unit 301, the association analysis unit 302, the compressed moving image decompression unit 311, the combining unit 303, and the like described with reference to FIGS. It functions as the output unit 304.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet is used as a recording medium, and is provided to the image processing device 120 and the image processing device 170 as a program via the network. Also good.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of an image processing system 10 according to an embodiment. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 120. FIG. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 170. FIG. 3 is a diagram illustrating an example of a processing flow of the image processing apparatus 120. FIG. It is a figure which shows an example of the image quality of several characteristic area | region animation and background area | region animation. It is a figure which shows an example of the processing flow of the image processing apparatus. 3 is a diagram illustrating an example of a configuration of a compression control unit 700 included in the image processing apparatus 120. FIG. It is a figure which shows an example of the data which the condition storage part 750 has stored. It is a figure which shows an example of the other data which the condition storage part 750 has stored. It is a figure which shows an example of the data which the information amount calculation part 720 has stored. It is a figure which shows an example in which the compression strength determination part 740 determines compression strength. It is a figure which shows an example of the time development of the information amount which the information amount calculation part 720 calculated. It is a figure which shows an example of the boundary area | region in ROI. It is a figure which shows another example of the block configuration in the image processing apparatus. It is a figure which shows an example of the block configuration of the hierarchy difference compression parts 282a and b. It is a figure which shows another example of the image processing system. 2 is a diagram illustrating an example of a hardware configuration of an image processing apparatus 120 and an image processing apparatus 170. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing system 20 Image processing system 100 Imaging device 102 Imaging part 104 Imaging moving image compression part 110 Communication network 120 Image processing apparatus 130 Person 140 Moving object 150 Monitoring object space 160 Space 170 Image processing apparatus 175 Image DB
180 Display device 201 Compressed video acquisition unit 202 Compressed video expansion unit 203 Characteristic region detection unit 204 Image segmentation unit 205 Image generation unit 206 Association processing unit 207 Output unit 210 Fixed value unit 211 Fixed value unit 220 Reduction unit 221 Image quality Reduction unit 230 Encoding unit 231 Encoding unit 231a Background region moving image encoding unit 231b-d Feature region moving image encoding unit 240 Compression unit 301 Compressed moving image acquisition unit 302 Association analysis unit 310 Compressed moving image expansion unit 311 Compressed moving image expansion unit 303 Compositing unit 304 Output unit 700 Compression control unit 710 Boundary region specifying unit 720 Information amount calculating unit 730 Same subject region specifying unit 740 Compression strength determining unit 750 Condition storage unit 752 Condition acquisition unit 760 Compression control unit 1604 Image processing unit

Claims (15)

A feature region detector for detecting a plurality of feature regions from the image;
A condition storage unit for storing different assignment conditions according to the features of the feature region, wherein different compression strengths are assigned according to the features of the feature region;
A compression unit that compresses each of a plurality of feature region images that are images of each of the plurality of feature regions;
An image processing comprising: a compression control unit that controls, based on the conditions stored in the condition storage unit, the compression strength at which the compression unit compresses each of the plurality of feature region images according to the feature region feature. apparatus. The feature region detection unit detects a plurality of feature regions from a plurality of moving image constituent images included in the moving image that is the image,
The compression control unit controls, based on the conditions stored in the condition storage unit, compression strength at which the compression unit compresses each of the plurality of feature region images according to features of the feature region. Item 8. The image processing apparatus according to Item 1. The condition storage unit stores the different conditions according to the type of feature region,
The compression control unit controls the compression strength at which the compression unit compresses each of the plurality of feature region images according to the type of feature region, based on the condition stored in the condition storage unit. Item 3. The image processing apparatus according to Item 2. The feature region detection unit detects a plurality of feature regions having different types of features from the plurality of video composition images,
The compression unit compresses each of a plurality of feature region moving images including a plurality of the feature region images having the same type of feature in the plurality of moving image constituent images,
The compression control unit includes a compression strength at which the compression unit compresses each of the plurality of feature region moving images based on the condition stored in the condition storage unit. The image processing apparatus according to claim 3, wherein the image processing apparatus performs control according to a feature type of the feature region image. The condition storage unit stores different image quality reduction amounts according to the types of feature areas,
The compression unit is
An image quality reduction unit that reduces the image quality of each of a plurality of feature region videos including a plurality of feature region images having the same type of feature in the plurality of movie composition images;
The compression control unit includes the feature region image in which each of the plurality of feature region moving images includes an image quality reduction amount that the image quality reduction unit reduces the image quality based on the image quality reduction amount stored in the condition storage unit. The image processing apparatus according to claim 4, wherein each of the image processing apparatuses is controlled according to a type of the feature. The condition storage unit stores a reduction amount of a different spatial resolution depending on a type of the feature region or a reduction amount of a temporal resolution different depending on the type of the feature region,
The compression control unit includes a spatial resolution reduction amount or a temporal resolution that the image quality reduction unit reduces the image quality based on at least one of a spatial resolution reduction amount and a temporal resolution reduction amount stored in the condition storage unit. The image processing apparatus according to claim 5, wherein the amount of reduction is controlled according to a feature type of the feature region image included in each of the plurality of feature region moving images. It further comprises a condition acquisition unit that acquires different image quality reduction amounts according to the types of feature areas,
The image processing apparatus according to claim 5, wherein the condition storage unit stores different image quality reduction amounts according to the types of feature regions acquired by the condition acquisition unit. The condition storage unit stores the different conditions according to the feature of the region indicated by the feature region,
The compression control unit is configured to determine a compression strength at which the compression unit compresses each of the plurality of feature region images based on the condition stored in the condition storage unit, according to the feature of the region indicated by the feature region. The image processing apparatus according to claim 2 to be controlled. The condition storage unit stores the different conditions according to the number of the feature areas, the size of the feature areas, or the position of the feature areas,
The compression control unit, based on the condition stored in the condition storage unit, the compression strength at which the compression unit compresses each of the plurality of feature region images, the number and size of regions indicated by the feature region The image processing apparatus according to claim 8, wherein the image processing apparatus is controlled according to a position. The condition storage unit stores the different conditions according to the characteristics of the object included in the feature area image,
The compression control unit, based on the condition stored in the condition storage unit, the compression strength at which the compression unit compresses each of the plurality of feature region images, and the feature of the object included in the feature region image The image processing apparatus according to claim 2, wherein the image processing apparatus is controlled according to the control. The condition storage unit stores different conditions according to the shape of the object, the direction of the object, the moving direction of the object, or the moving amount of the object included in the feature region image,
The compression control unit is configured to determine a compression strength at which the compression unit compresses each of the plurality of feature region images based on the condition stored in the condition storage unit, and a shape of an object included in the feature region image. The image processing apparatus according to claim 10, wherein the image processing apparatus performs control according to an object orientation, an object movement direction, or an object movement amount. An image generation unit that generates the plurality of feature region videos for each type of the feature by duplicating the movie,
The compression control unit includes a compression strength at which the compression unit compresses each of the plurality of feature region moving images based on the condition stored in the condition storage unit. The image processing apparatus according to claim 4, wherein control is performed according to a feature type of the feature region image. A video acquisition unit that acquires a plurality of videos obtained by imaging from different positions;
Extracted from a moving image constituent image included in another moving image including the same subject as the subject included in the feature area extracted from the moving image constituent image included in one of the plurality of moving images acquired by the moving image acquisition unit A same subject area specifying unit for specifying the feature area
The compression control unit includes the feature region of at least one of the feature regions including the same subject specified by the same subject region specifying unit based on the condition stored in the condition storage unit. The compression strength at which the compression unit compresses the image is controlled according to the feature of the feature region, and the compression strength at which the compression unit compresses the feature region image of the other feature region is set to the at least one feature region. The image processing apparatus according to claim 2, wherein the image processing apparatus is controlled to be higher than a compression rate for compressing the image. A feature region detection stage for detecting a plurality of feature regions from the image;
A condition storage stage for storing different allocation conditions according to the features of the feature region, wherein different compression strengths are assigned according to the features of the feature region;
A compression step of compressing each of a plurality of feature region images that are images of each of the plurality of feature regions;
An image processing comprising: a compression control step for controlling the compression strength at which each of the plurality of feature region images is compressed in the compression step according to the feature of the feature region based on the condition stored in the condition storage step Method. A program for an image processing device, wherein the image processing device is
A feature region detection unit for detecting a plurality of feature regions from an image;
A condition storage unit that stores different assignment conditions according to the features of the feature region, wherein different compression strengths are assigned according to the features of the feature region;
A compression unit that compresses each of a plurality of feature region images that are images of each of the plurality of feature regions;
A program that causes the compression unit to function as a compression control unit that controls the compression strength for compressing each of the plurality of feature region images according to the feature of the feature region based on the condition stored in the condition storage unit.

Images