JP4817260B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program. The present invention particularly relates to an image processing apparatus and an image processing method for processing an image, and a program for the image processing apparatus.

2. Description of the Related Art An image encoding device that encodes a filtered image signal by selectively filtering an image signal in a region other than a specific region in an input image signal in at least one of a spatial direction and a time axis direction is known. (For example, refer to Patent Document 1). In addition, by comparing the encoded screen with the screen to be encoded, a change area in the screen is extracted, the screen to be encoded is divided into a plurality of image blocks, and an image block including the change area is obtained. A moving image encoding apparatus that performs encoding is known (see, for example, Patent Document 2). In this moving image encoding apparatus, more information bits than the number of information bits assigned to an image block including another change area are compared with an image block including a more limited area among image blocks including a change area. Assign a number and encode.
Japanese Patent No. 3046379 Japanese Patent No. 2828977

  However, with the technique of the above-mentioned patent document, a certain amount of image signal remains even if filtering is performed on areas other than a specific area or the number of information bits is reduced. Therefore, according to the technique of the above-mentioned patent document, when an image area other than such a specific area is unnecessary, a band is wasted.

  In order to solve the above-described problem, according to the first aspect of the present invention, there is provided an image processing device, a feature region detecting unit for detecting a feature region from a plurality of moving image constituent images included in a moving image, and a plurality of moving images. An inclusion area specifying unit that specifies an inclusion area that includes a plurality of feature areas in each of the constituent images; a fixed value converting unit that fixes pixel values of areas other than the inclusion areas in each of the plurality of moving image constituent images; and An encoding unit that encodes a plurality of fixed-value moving image constituent images.

  An image comparison unit that compares at least one moving image constituent image among the plurality of fixed moving image constituent images with another moving image constituent image is further provided, and the encoding unit at least based on a comparison result by the image comparing unit A plurality of fixed-value moving image constituent images may be encoded by encoding one moving image constituent image.

  A moving image dividing unit that divides the moving image into a plurality of partial moving images each including a plurality of moving image constituting images is further provided, and the inclusion area specifying unit includes a plurality of moving image constituting images included in the partial moving images in each of the plurality of partial moving images. The inclusion area that includes a plurality of feature areas in each is specified, and the fixed value unit fixes the pixel values of the areas other than the inclusion areas in the plurality of moving image constituent images included in the partial moving image in each of the partial moving images. The image comparison unit compares at least one moving image constituent image with another moving image constituent image included in the same partial moving image in each of the plurality of partial moving images, and the encoding unit applies to each of the plurality of partial moving images. By encoding at least one moving image constituent image based on the comparison result by the image comparison unit, a plurality of fixed moving image constituent images It may be encoded for each minute video.

  The inclusion area specifying unit specifies a set area of predetermined partial areas (blocks, macroblocks, slices) including a plurality of feature areas in each of a plurality of moving image constituent images as an inclusion area, and an image comparison unit In each of the plurality of partial videos, for each of a plurality of partial areas included in at least one video component image, the encoding unit compares the plurality of partial video images with other video component images included in the same partial video. In each of the above, a plurality of fixed moving image constituent images may be encoded for each of the plurality of partial moving images by compressing at least one moving image constituent image for each partial region based on the comparison result by the image comparison unit. .

  The image comparison unit includes, for each of the plurality of partial videos, for each of a plurality of partial areas included in at least one video constituent image, an image between the images included in other video constituent images included in the same partial video The encoding unit calculates a difference amount, and encodes at least one moving image constituent image for each partial region based on the difference amount calculated by the image comparison unit in each of the plurality of partial moving images. The plurality of moving image constituent images may be encoded for each of the plurality of partial moving images.

  In each of the plurality of partial moving images, the image comparison unit compares each of the plurality of partial regions included in the at least one moving image constituent image with another partial moving image included in the same partial moving image. The difference amount of the image is a region smaller than a predetermined value, the difference target region in another moving image constituent image is specified, the difference amount of the image between the specified difference target region is calculated, A plurality of fixed values obtained by encoding at least one moving image constituent image for each partial area based on the difference amount calculated by the image comparison unit and the difference between the position of the difference target area and the position of the partial area. May be encoded for each of a plurality of partial videos.

  The inclusion area specifying unit specifies a block aggregation area in MPEG encoding that includes a plurality of feature areas in each of a plurality of moving image constituent images as an inclusion area, and the image comparison unit includes a plurality of partial moving pictures in each of the partial moving images. For each of a plurality of blocks or macroblocks included in at least one moving image constituent image, the difference between the motion vector and the image is calculated in comparison with other moving image constituent images included in the same partial moving image. , Encoding at least one moving image constituent image for each block and macroblock based on the motion vector calculated by the image comparison unit and the difference amount of the image, thereby converting a plurality of fixed moving image constituent images into a plurality of portions MPEG encoding may be performed for each moving image.

  According to the second aspect of the present invention, there is provided an image processing method, a feature region detecting step for detecting a feature region from a plurality of moving image constituent images included in a moving image, and a plurality of features in each of the plurality of moving image constituent images. An inclusion area specifying stage for specifying an inclusion area including the area, a fixed value stage for fixing pixel values of areas other than the inclusion area in each of the plurality of moving image constituent images, and a plurality of fixed-value moving pictures An encoding stage for encoding the constituent images.

  According to a third aspect of the present invention, there is provided a program for an image processing device, wherein the image processing device detects a feature region from a plurality of moving image constituent images included in a moving image, and a plurality of moving image configurations An inclusion area specifying unit that specifies an inclusion area that includes a plurality of feature areas in each of the images, a fixed value conversion unit that fixes pixel values of areas other than the inclusion areas in each of the plurality of moving image constituent images, and a fixed value It is made to function as an encoding part which encodes the plurality of moving image constituent images.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of an image processing system 10 according to an embodiment. An object of the image processing system 10 is to reduce the amount of image data while maintaining a high quality image of a characteristic subject.

  The image processing system 10 includes a plurality of imaging devices 100a-c (hereinafter collectively referred to as imaging devices 100) that image the monitoring target space 150, and a plurality of image processing devices 120a-c (hereinafter referred to as image processing devices) that process images. 120), an image processing device 170, a communication network 110, and a plurality of display devices 180a-c (hereinafter collectively referred to as display device 180).

  The image processing device 120a is connected to the imaging device 100a. Further, the image processing device 120b is connected to the imaging device 100b. Further, the image processing device 120c is connected to the imaging device 100c. Note that the image processing device 170 and the display device 180 are provided in a space 160 different from the monitoring target space 150.

  Hereinafter, operations of the imaging device 100a, the image processing device 120a, the image processing device 170, and the display device 180a will be described. The imaging apparatus 100a MPEG-encodes a captured moving image obtained by imaging the monitoring target space 150, generates captured moving image data, and outputs the captured moving image data to the image processing apparatus 120a to which the imaging apparatus 100a is connected.

  The image processing device 120a acquires the captured moving image data generated by the imaging device 100a. The image processing device 120 decodes the captured moving image data acquired from the imaging device 100 to generate a captured moving image, and a plurality of features having different types of features such as a person 130 and a moving object 140 such as a vehicle from the generated captured moving image. Detect areas. Then, the image processing device 120a generates a plurality of feature area moving images by generating a moving image in which the characteristic region has a higher image quality than the other regions for each type of feature from the captured moving image. In addition, the image processing apparatus 120a generates a background area moving image that is a moving image of a background area other than the characteristic area and has a lower image quality than the characteristic area moving image.

  Then, the image processing device 120a generates a plurality of feature region moving image data and background region moving image data by encoding the generated plurality of feature region moving images and background region moving images, respectively. At the same time, the image processing device 120a associates a plurality of characteristic area moving image data and background region moving image data obtained by encoding with each other and transmits them to the image processing device 170 through the communication network 110.

  The image processing device 170 acquires a plurality of feature region moving images and a background region moving image by decoding the associated plurality of feature region moving image data and background region moving image data received from the image processing device 120a, respectively. Then, the image processing apparatus 170 generates a single combined moving image by combining the plurality of feature region moving images and the background region moving image, and supplies the generated combined moving image to the display device 180a. The display device 180a displays the moving image supplied from the image processing device 170.

  The functions and operations of the imaging device 100b and the imaging device 100c are the same as the functions and operations of the imaging device 100a, except that the imaging moving image data is provided to the image processing device 120b and the image processing device 120c, respectively. The description is omitted. The functions and operations of the image processing device 120b and the image processing device 120c are the same as the functions and operations of the image processing device 120a, except that the captured moving image data is acquired from the imaging device 100b and the imaging device 100c, respectively. Good. Therefore, the description is omitted. Further, the image processing device 170 generates one moving image from the plurality of associated feature region moving image data and background region moving image data received from the imaging device 100b and the imaging device 100c, respectively, and displays each of the display devices 180b. And supplied to the display device 180c. In addition, the display device 180b and the display device 180c display each moving image supplied from the image processing device 170.

  When the image processing system 10 of the present embodiment is actually applied as a monitoring system, for example, there may be a case where a subject that is characteristic as a monitoring target such as a person or a moving object can be left with high image quality. In addition, the amount of moving image data may be reduced.

  FIG. 2 shows an example of a block configuration of the image processing apparatus 120. The image processing apparatus 120 includes a compressed moving image acquisition unit 201, a compressed moving image decompression unit 202, a feature region detection unit 203, an image division unit 204, an image generation unit 205, a fixed value unit 210, a reduction unit 220, an encoding unit 230, An association processing unit 206 and an output unit 207 are included.

  Fixed value unit 210 includes a plurality of fixed value units 211a to 211c (hereinafter collectively referred to as fixed value unit 211). The reduction unit 220 includes a plurality of image quality reduction units 221a-d (hereinafter collectively referred to as image quality reduction units 221).

  The encoding unit 230 includes a background region moving image encoding unit 231a and a plurality of feature region moving image encoding units 231b-d (hereinafter collectively referred to as a feature region moving image encoding unit 231). The background area moving image encoding unit 231a and the feature area moving image encoding unit 231b-d may be collectively referred to as an encoding unit 231.

  The image quality reduction unit 221a and the background area moving image encoding unit 231a function as the compression unit 240a. Further, the image quality reduction unit 221b and the background area moving image encoding unit 231b function as the compression unit 240b. Similarly, the image quality reduction unit 221c and the background area moving image encoding unit 231c function as the compression unit 240c. The image quality reduction unit 221d and the background area moving image encoding unit 231d function as a compression unit 240d. The plurality of compression units 240a-d are collectively referred to as the compression unit 240.

  The compressed moving image acquisition unit 201 acquires a compressed moving image. Specifically, the compressed moving image acquisition unit 201 acquires encoded captured moving image data generated by the imaging device 100. The compressed moving image decompression unit 202 restores the moving image acquired by the compressed moving image acquisition unit 201 and generates a plurality of moving image constituent images included in the moving image. Specifically, the compressed moving image decompression unit 202 decodes the captured moving image data acquired by the compressed moving image acquisition unit 201, and generates a plurality of moving image constituent images included in the moving image. The moving image composition image includes a frame image and a field image.

  The feature region detection unit 203 detects a feature region from a plurality of moving image constituent images included in the moving image. Then, the image dividing unit 204 divides each of the plurality of moving image constituent images into a feature area and a background area.

  The image generation unit 205 generates a plurality of feature area compression moving images each including a plurality of feature area images by extracting feature area images from the plurality of moving image constituent images. Specifically, the image generation unit 205 reproduces a moving image to compress a plurality of feature region moving images and compress a plurality of feature region moving images and a background region moving image to compress a background region moving image. Generate a video.

  Then, the fixed value converting unit 211 converts the pixel values of regions other than the feature region image in the plurality of moving image constituent images included in the feature region compression moving image to a fixed value. For example, the fixed value unit 211 sets a pixel value of an area other than the feature area image to a predetermined value (for example, a luminance value of 0). Then, the compression unit 240 compresses a plurality of feature region compression images each including a plurality of moving image constituent images in which pixel values of regions other than the feature region image are fixed values with an intensity corresponding to each feature amount. . As described above, the compression unit 240 compresses each of the plurality of feature region compression moving images and the background region compression moving image with an intensity corresponding to each feature amount.

  As described above, the feature region detection unit 203 detects a feature region from the image. Then, the image dividing unit 204 divides the image into a feature region and a background region other than the feature region. Then, the compression unit 240 compresses the feature area image that is the image of the feature area and the background area image that is the image of the background area with different strengths. Then, the compression unit 240 compresses the feature area moving image including a plurality of characteristic area images and the background area moving image including a plurality of background area images with different strengths.

  Note that in the compression unit 240b, the compression unit 240c, and the compression unit 240d, it is determined in advance which type of feature area moving image should be compressed, and the compression unit 240b, the compression unit 240c, and the compression unit 240d are determined in advance. The feature region moving image of the specified feature type is compressed. Note that the compression strength in the case of compressing the feature region moving image is determined in advance for each of a plurality of feature types, and the compression unit 240b, the compression unit 240c, and the compression unit 240d are features of the predetermined feature types. The area moving image is compressed with a compression strength predetermined for the type of the feature. As described above, the compression unit 240 compresses a plurality of regions in parallel using the compressor provided for each image region divided by the image dividing unit 204.

  Note that the compression unit 240 may be implemented by a single compressor, and may sequentially compress a plurality of feature area videos and background area videos. In addition, the compression unit 240 is predetermined for each feature type and background of each region for each region obtained by dividing the captured moving image decoded by the compressed moving image decompression unit 202 by the image dividing unit 204. One video data may be generated by compressing at a compression rate.

  Note that the feature region detection unit 203 detects a plurality of feature regions having different types of features from a plurality of moving image constituent images included in a moving image that is an image. Then, the image dividing unit 204 divides the plurality of moving image constituent images into each of a plurality of feature areas and a background area other than the plurality of feature areas. Then, the compression unit 240 compresses the plurality of feature area moving images and the background area moving image with an intensity corresponding to each feature amount. The feature amount includes the type of subject, the size of the subject, the moving speed at which the moving object moves, and the size of the feature region.

  Specifically, the image quality reduction unit 221 compresses a plurality of feature area moving images and background area moving images by reducing the image quality according to each feature amount. More specifically, the image quality reduction unit 221 compresses a plurality of feature region moving images and background region moving images by reducing the resolution or frame rate according to each feature amount. Then, the encoding unit 231 compresses the plurality of feature area moving images and the background area moving image by encoding using setting values corresponding to the respective feature amounts. For example, the encoding unit 231 compresses a plurality of feature area moving images and background area moving images by encoding them using an assigned code amount corresponding to each feature amount.

  The association processing unit 206 includes a plurality of feature region moving image data and background region moving image data generated by the compression units 240 compressing a plurality of feature region moving images and background region moving images, for example, with tag information and the like. And correspond to each other. The output unit 207 sends the plurality of feature area moving image data and background area moving image data associated by the association processing unit 206 to the communication network 110.

  FIG. 3 shows an example of a block configuration of the image processing apparatus 170. The image processing apparatus 170 includes a compressed moving image acquisition unit 301, an association analysis unit 302, a compressed moving image decompression unit 310, a composition unit 303, and an output unit 304. The compressed moving image expansion unit 310 includes a plurality of compressed moving image expansion units 311a-d (hereinafter collectively referred to as a compressed moving image expansion unit 311).

  The compressed moving image acquisition unit 301 acquires a plurality of associated feature region moving image data and background region moving image data output from the output unit 207. The association analysis unit 302 analyzes, for example, attached tag information, and extracts a plurality of associated feature region movie data and background region movie data acquired by the compressed movie acquisition unit 301.

  The compressed moving image decompression unit 311 decodes the plurality of characteristic area moving image data and background area moving image data. Specifically, the compressed moving image decompression unit 311a decodes the background area moving image data. In addition, the compressed moving image decompression unit 311b-d decodes one feature region moving image out of the plurality of feature region moving image data, and acquires a plurality of feature region moving images and a background region moving image. Note that the compressed moving image decompression unit 311b-d is provided for each type of feature, and decodes any type of feature region moving image data.

  The synthesizing unit 303 synthesizes the moving image constituent images obtained by the compressed moving image decompressing unit 311 decoding. Specifically, the synthesizing unit 303 superimposes the feature region images of the moving image constituent images included in each of the plurality of feature region moving images decoded by the compressed moving image decompression unit 311b-d on the moving image constituent images included in the background region moving image. A combined moving image composition image is generated. The output unit 304 supplies a moving image including a plurality of moving image constituent images generated by the combining unit 303 to the display device 180.

  Note that the compressed video decompression unit 310 of the present embodiment includes a plurality of compressed video decompression units 311 corresponding to the number of feature types, but in another form, one compressed video decompression unit included in the compressed video decompression unit 310. 311 may sequentially decode the background area moving image data and the plurality of characteristic area moving image data. In addition, when provided as one moving image data from the image processing device 120, one compressed moving image decompression unit 311 decodes the one moving image data, and the output unit 304 outputs the decoded moving image. May be.

  FIG. 4 shows an example of the processing flow of the image processing apparatus 120. The compressed moving image acquisition unit 201 acquires captured moving image data (401). The compressed moving image decompression unit 202 generates a plurality of frame images 410 by decoding the captured moving image data. The feature region detection unit 203 detects a region of interest (ROI) that is an example of a feature region based on the image content of the frame image 410 or the image content of the plurality of frame images 410 (402).

  The feature region detection unit 203 detects regions including human faces, human bodies, and moving objects as different types of ROIs. For example, the feature area detection unit 203 detects, by pattern matching or the like, an area including an object having a degree of coincidence with a pattern related to a predetermined person's face that is greater than a predetermined degree of coincidence. And Further, the feature region detection unit 203 detects a human body region including an object whose degree of matching with a pattern related to a human body is greater than a predetermined matching level by pattern matching or the like, and sets the detected region as an ROI. Note that the feature region detection unit 203 may detect a human body region from a region existing in the vicinity of the face region.

  In addition, the feature region detection unit 203 identifies a moving region that is a region including a moving object based on the image contents of a plurality of frame images. For example, an area in which the amount of change in pixel value with respect to another frame image is larger than a predetermined amount of change is specified as a moving area. In addition, the feature region detection unit 203 extracts objects included in each of the plurality of frame images by edge extraction or the like. Then, the feature region detection unit 203 identifies objects that are included in different positions of other frame images and that match with a degree of coincidence that is greater than a predetermined degree of coincidence, and includes an area that includes the identified object. It may be specified as a moving area.

  As described above, the feature region detection unit 203 detects a region that meets a predetermined condition relating to image content as an ROI. More specifically, the feature area detection unit 203 detects an area including an object that meets a predetermined condition as an ROI. For example, the feature area detection unit 203 detects, as an ROI, an area that includes an object having a degree of coincidence with a predetermined shape that is greater than a predetermined degree of coincidence. Further, the feature area detection unit 203 detects an area where the change amount of the image is larger than the predetermined change amount as the ROI. For example, an area where the change amount of the pixel value between other frame images is larger than a predetermined change amount is detected as the ROI.

  Then, the image processing apparatus 120 generates a moving image for compression based on the detected ROI (403). Specifically, the image dividing unit 204 divides the frame image into ROI and other areas. Then, the image generation unit 205 generates a feature region moving image 430, a feature region moving image 440, a feature region moving image 450, and a background region moving image 420 by duplicating the plurality of frame images 410. Specifically, the image generation unit 205 reproduces a plurality of frame images 410, thereby generating a facial region feature region moving image 450, a human region feature region moving image 440, a moving region feature region moving image 430, and A background area moving image 420 for the background area is generated.

  Then, the image processing apparatus 120 reduces the image quality of the feature region moving image 430, the feature region moving image 440, the feature region moving image 450, and the background region moving image 420 by the fixed value converting unit 211 and the image quality reducing unit 221. (404a, 404b, 404c, 404d). Specifically, the fixed value unit 211 changes the pixel values in the ROI divided by the image dividing unit 204 in the frame images included in each of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450. First, the pixel value for the region other than the ROI is set to a predetermined value (for example, luminance value 0). Note that the fixed value converting unit 211 may set the pixel values in the region other than the ROI as an average pixel value of the pixel values in the region near the ROI.

  As described above, the image generation unit 205 and the fixed value conversion unit 211 generate the feature region moving image 430, the feature region moving image 440, the feature region moving image 450, and the background region moving image 420 each including a plurality of frame images having the same viewing angle. The As will be described in detail later, the image processing apparatus 170 converts a moving image in which a region other than the ROI, such as the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450, into a fixed value is used as the background region moving image. A moving image superimposed on 420 is generated. Therefore, the background area animation 420, the feature area animation 430, the feature area animation 440, and the feature area animation 450 can be regarded as a background layer, a motion area layer, a human area layer, and a face area layer, respectively.

  In addition, the image quality reduction unit 221 reduces the image quality of the image in the ROI in the frame images included in each of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 according to the type of feature. Specifically, at least one of resolution, the number of gradations, and the number of colors is determined in advance as a parameter that determines the image quality of each of the face area, the human area, and the moving area. For example, a high resolution is determined in advance in the order of the face area, the human area, and the moving area.

  Then, the image quality reduction unit 221 sets the image quality of the image in the ROI in the frame image included in each of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 to a resolution predetermined for the type of feature, Convert to an image with the number of gradations and the number of colors. In addition, the image quality reduction unit 221 makes the image quality of the frame image of the background area moving image lower than the image quality of the image in the ROI. For example, the image quality reduction unit 221 makes the resolution of the frame image of the background area moving image smaller than that of the image in the ROI.

  In addition, the image quality reduction unit 221 reduces the frame rates of the background area moving image 420, the feature area moving image 430, the feature area moving image 440, and the feature area moving image 450. For example, a frame rate is determined in advance for each type of feature, for example, for each face area, human area, and moving area. Then, the image quality reduction unit 221 thins out the frame images included in the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 at a predetermined interval according to a frame rate that is predetermined according to the type of the feature. The frame rates of the feature area animation 430, the feature area animation 440, and the feature area animation 450 are reduced. In addition, the image quality reduction unit 221 reduces the frame rate of the background area moving image 420 by thinning out the frame images included in the background area moving image 420 according to a frame rate predetermined as the frame rate of the background area moving image.

  The image quality reduction unit 221a reduces the image quality of the background area moving image 420. In addition, the image quality reduction unit 221b, the image quality reduction unit 221c, and the image quality reduction unit 221d reduce the image quality of the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450, respectively.

  Then, the background area moving image encoding unit 231a and the area moving image encoding unit 231b-d encode the moving images whose image quality has been reduced by the image quality reducing unit 221 (405a, 405b, 405c, and 405d). For example, the background area moving image encoding unit 231a and the area moving image encoding unit 231b-d each encode the moving image whose image quality has been reduced by the image quality reducing unit 221.

  For example, the background area moving image encoding unit 231a performs MPEG encoding on the background area moving image with the encoding setting for the background area moving image. The feature region moving image encoding unit 231b, the feature region moving image encoding unit 231c, and the feature region moving image encoding unit 231d are encoding settings for the feature region moving image of the moving region, the human region, and the face region, respectively. Each area moving image is MPEG-encoded. Note that the coding setting includes setting a quantization table in MPEG coding, for example. The encoding setting will be described with reference to FIG.

  Then, the association processing unit 206 adds tag information to the background region moving image data and the plurality of feature region moving image data obtained by encoding by the background region moving image encoding unit 231a and the feature region moving image encoding unit 231b-d. The output unit 207 outputs the information to the display device 180 by associating it with an accessory (406). At this time, the association processing unit 206 adds timing information (for example, time stamp) including information indicating the display timing of the frame image included in each of the background area moving image and the plurality of characteristic area moving images to the tag information or the like. Good. In addition, the association processing unit 206 identifies the imaging device 100 that captured the feature region information including information indicating the ranges of the plurality of feature regions, the background region movie, and the captured movie data that is the basis of the plurality of feature region movies. Identification information or the like to be added may be added to tag information or the like.

  As described above, the feature region detection unit 203 detects a plurality of feature regions having different types of subjects from a plurality of moving image constituent images included in a moving image that is an image. Then, the compression unit 240 compresses the plurality of feature area moving images with an intensity corresponding to the type of each subject. In the present embodiment, a human face and a human body region have been described as examples of the types of subjects. However, in other examples, vehicle license plates and vehicle bodies other than license plates may be used as the types of subjects.

  Further, the front face and the side face of the person may be the subject type. Further, a stationary subject and a moving subject may be used as the subject type. Further, the feature region detection unit 203 may detect a region including a plurality of subjects having different distances from the imaging device 100 to the subject as a plurality of feature regions having different types of features.

  Note that the compression unit 240 may compress the motion region, the human body, the side profile of the person, and the face in front of the person in order of large compression strengths, which are examples of feature types. In the present embodiment, in consideration of the purpose of using the image processing system 10 as a monitoring system, an area including a human face is detected as an ROI, and the detected ROI has a higher image quality than an area other than the ROI. . However, the image processing system 10 can also be used as, for example, a system that captures landscapes of street corners. When the image processing system 10 is used in such a form, for the purpose of protecting personal information, an area including a human face is detected as an ROI, and the detected ROI has a lower image quality than an area other than the ROI. It may be. For example, the compressing unit 240 may compress the feature region moving image and the background region moving image with a large compression strength in the order of the face in front of the person, the profile of the person, the human body, the motion region, and the background region.

  In addition, the feature region detection unit 203 may detect a plurality of feature regions having different subject velocities from a plurality of frame images. In this case, the image quality reduction unit 221 may convert the feature area moving image into a moving image having a higher frame rate as the subject speed increases. As described above, the compression unit 240 may compress a plurality of feature region moving images with an intensity corresponding to the speed of each subject.

  As described above, the image processing apparatus 120 generates a plurality of feature region moving images and background region moving images having the same viewing angle by fixing a region other than the ROI in the frame image included in the feature region moving image. For this reason, according to the image processing apparatus 120, the feature region moving image may be compressed at a high compression rate using a general-purpose encoder without using a specially designed encoder. For example, when a feature area moving image is encoded using a motion vector as in MPEG encoding, the difference value between pixel values is often 0 for a macroblock in a fixed value area. Therefore, such fixed values may reduce the manufacturing cost of the image processing apparatus 120 while maintaining a high compression rate.

  In the above description, the compression unit 240 compresses the feature region moving image including the frame image in which the region other than the ROI region is fixed. Note that the compression unit 240 may cut out and compress an image in the ROI in the frame image included in the feature area moving image and output it as a feature area moving image.

  When the feature area detection unit 203 has not detected the ROI, the output unit 207 outputs the background area moving image data output from the compression unit 240a to the image processing apparatus 170. In this case, the image generation unit 205 does not have to generate the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450. Then, the image generation unit 205 generates the feature region moving image 430, the feature region moving image 440, and the feature region moving image 450 on condition that the feature region detection unit 203 detects the ROI, and the output unit 207 performs the above processing. The plurality of characteristic area moving image data and background area moving image data generated by the above is output to the image processing apparatus 170. During this time, the compression unit 240a may continue to compress the background area moving image 420 at a predetermined compression ratio for the background area in order to compress the background area moving image 420.

  In addition, while the feature region detection unit 203 does not detect the ROI, the compression unit 240 is set in advance to be lower than the compression rate for the background region and higher than the compression rate for compressing the feature region moving image. The background area moving image may be compressed at the ROI non-detection period compression rate. Then, the compression unit 240 may compress the background region moving image with the background region compression rate on condition that the feature region detection unit 203 detects the ROI. At this time, the compression unit 240 may compress the feature region moving image at a compression rate lower than the ROI non-detection period compression rate.

  The compression unit 240 compresses the background region moving image of a predetermined period after the ROI is detected by the feature region detection unit 203 at the ROI non-detection period compression rate, and a period longer than the predetermined period elapses. As a condition, the background area moving image may be compressed at the compression ratio for the background area. In this case, even when the region that should originally be detected as ROI is not detected as ROI, a background region moving image having a certain level of image quality may be provided. The compression unit 240 also sets the image quality of the region including the ROI region in the other frame image estimated from the position of the ROI region in the plurality of frame images detected by the feature region detection unit 203 to be higher than the image quality of the other region. Therefore, the compression rate may be different for each region.

  FIG. 5 shows an example of the image quality of a plurality of feature area videos and background area videos. Here, for the purpose of simplifying the description, it is assumed that the frame rate of the captured moving image data acquired by the compressed moving image acquisition unit 201 is 16 fps, and the resolution of the frame image included in the captured moving image data is 72 dpi.

  The resolution ratio indicating the ratio of the resolution of the frame image included in the background area moving image 420 after image quality reduction to the resolution of the frame image 410 included in the captured moving image is predetermined as 1/8. The image quality reduction unit 221 generates a 9 dpi frame image having a resolution of 1/8 with respect to the frame image included in the background area video 420 before image quality reduction generated by copying the captured video by the image generation unit 205. This is generated by thinning out the pixels of the frame image included in the background area moving image 420 before image quality reduction. In addition, the frame rate ratio indicating the ratio of the frame rate of the background area moving image 420 after image quality reduction to the frame rate of the captured moving image is set to 1/8 in advance. The image quality reduction unit 221 converts the background area video 420 having a frame rate of 2 fps, which is a frame rate of 1/8 of the frame rate of the background area video 420 before the image quality reduction, into a frame image included in the background area video 420 before the image quality reduction. Generate by thinning out.

  Similarly, a resolution ratio and a frame rate ratio are determined for each of the plurality of characteristic area moving images. For example, the resolution ratio and the frame rate ratio are set to 1/4 for the feature area moving image 430, and the resolution ratio and the frame rate ratio are set to 1/2 for the feature area moving image 440. Is defined as 1/1 as the resolution ratio and the frame rate ratio. As a result, the image quality reduction unit 221b generates a feature region moving image 430 having a frame rate of 4 fps and a frame image resolution of 18 dpi. In addition, the image quality reduction unit 221c generates a feature region moving image 440 having a frame rate of 8 fps and a frame image resolution of 36 dpi. In addition, the image quality reduction unit 221d generates a feature region moving image 450 having a frame rate of 16 fps and a frame image resolution of 72 dpi.

  In the above example, the case where the image quality reduction unit 221 reduces the image quality of the frame image by thinning out the pixels of the frame image included in the plurality of feature region moving images and the background region moving image has been described as an example. In addition, the image quality reduction unit 221 may reduce the image quality of the frame image using a filter that passes through a predetermined frequency region, such as a low-pass filter. In this case, the frequency band through which the filter passes and the intensity indicating the degree of passage may be determined in advance for each type of feature, for example, for each of the background region, the moving region, the human region, and the face region.

  In addition to the image quality reduction by the image quality reduction unit 221, or in place of the image quality reduction by the image quality reduction unit 221, the encoding unit 231 may reduce the image quality of the frame image. For example, the encoding unit 231 can reduce the image quality by increasing the value of the quantization table in MPEG encoding. The magnitude of the value of the quantization table may be determined in advance for each type of feature. For example, each of the background region moving image encoding unit 231a and the plurality of feature region moving image encoding units 231b-d may perform encoding using a quantization table having a preset value. In addition, a value for each frequency component in the quantization table may be determined in advance for each type of feature.

  Further, the image quality reduction unit 221 may further average a plurality of frame images included in the background area moving image. According to this, when an object indicating a moving object is included in the frame image, a frame image in which the objects of the moving object are averaged is obtained. When such a plurality of frame images are continuously displayed, the movement of the moving object may appear smoothly in the viewer's eyes.

  In the above description, after the image generation unit 205 replicates the captured moving image, frame images included in a plurality of feature region moving images and background region moving images obtained by copying are thinned out or pixels are thinned out. In the above, the mode of compressing a plurality of feature region moving images and background region moving images has been described. In addition, the image generation unit 205 generates a plurality of feature area videos and background area videos with a reduced frame rate by thinning out and selecting a plurality of frame images included in the captured video according to the frame rate ratio. Also good. Then, the image quality of the plurality of feature area moving images and the background area moving images may be reduced by the fixed value converting unit 211a fixing the value and the image quality reducing unit 221 reducing the resolution. The frame rate of the background area moving image may be fixed, but may be variable. For example, the image quality reduction unit 221 may reduce the image quality of the background area moving image by thinning out frame images at timings other than the timing at which a change exceeding a predetermined amount occurs in the frame image.

  FIG. 6 shows an example of the processing flow of the image processing apparatus 170. The compressed moving image acquisition unit 301 acquires a plurality of associated feature area moving image data and background area moving image data from the image processing device 120, and identifies timing information, information for identifying the imaging device 100, and the like using the attached tag information. Is acquired (601). Then, the compressed moving image decompression unit 311 generates a background region moving image 610 indicating a background layer by decoding the plurality of feature region moving image data and background region moving image data (602a). At the same time, the compressed moving image decompression unit 311 generates a feature region moving image 620 indicating a moving region layer, a feature region moving image 630 indicating a human region layer, and a feature region moving image 640 indicating a face region layer (602b, 602c, and 602d). .

  The synthesizing unit 303 synthesizes frame images included in the background area moving image 610, the characteristic area moving image 620, the characteristic area moving image 630, and the characteristic area moving image 640 (603). At this time, the synthesis unit 303 enlarges the frame image so that the subject at the same position as the frame image having the maximum resolution overlaps according to the resolution of the frame image included in each of the background area moving image and the plurality of characteristic area moving images. Then, the enlarged frame images are superimposed to generate a composite frame image.

  At this time, the synthesizing unit 303 cuts out the feature region image in the frame image included in the feature region moving image 620, the feature region moving image 630, and the feature region moving image 640, and overwrites the frame image included in the background region moving image 610. To generate a composite frame image. In addition, when the frame rates of the background area moving image 610, the feature area moving image 620, the feature area moving image 630, and the feature area moving image 640 are different, the synthesizing unit 303 performs the background area moving image 610, the feature area moving image 620, and the feature area moving image 630. , And the latest frame image of the feature region moving image 640 are synthesized.

  Thereby, the composition unit 303 generates a composite frame image. The combining unit 303 generates a combined moving image 650 including a plurality of combined frame images. Then, the output unit 304 selects the display device 180 that displays the composite video based on the tag information acquired by the compressed video acquisition unit 301 and supplies the composite video to the selected display device 180 (604).

  FIG. 7 shows an exemplary configuration of the compression unit 240. The compression unit 240 includes a moving image dividing unit 730 and an image comparison unit 740. The moving image dividing unit 730 includes a background region moving image dividing unit 731a and a feature region moving image dividing unit 731b-d (hereinafter collectively referred to as a feature region moving image dividing unit 731). The background area moving image dividing unit 731a and the characteristic area moving image dividing unit 731b-d may be collectively referred to as a moving image dividing unit 731.

  The image comparison unit 740 includes a background region image comparison unit 741a and feature region image comparison units 741b-d (hereinafter collectively referred to as a feature region image comparison unit 741). Note that the background region image comparison unit 741a and the feature region image comparison units 741b-d may be collectively referred to as an image comparison unit 741.

  FIG. 8 shows an example of the configuration of the compression control unit 800 further included in the image processing apparatus 120. The compression control unit 800 includes a feature region specifying unit 810, a corresponding region specifying unit 820, an object motion calculating unit 830, a fitness calculating unit 840, a condition storage unit 850, a compression control unit 860, and an inclusion region specifying unit 870. The inclusion region specifying unit 870 includes a boundary region specification unit 872, a peripheral region specification unit 874, and an inclusion region determination unit 876.

  Hereinafter, functions and operations of each component included in the compression unit 240 illustrated in FIG. 7 and the compression control unit 800 illustrated in FIG. 8 will be described together with functions and operations of the image processing apparatus 120 illustrated in FIG.

  The inclusion area specifying unit 870 specifies an inclusion area that includes a plurality of feature areas in each of the plurality of moving image constituent images. Then, the fixed value unit 211 converts the pixel values of the regions other than the inclusion region in each of the plurality of moving image constituent images into fixed values. Then, the encoding unit 231 encodes a plurality of fixed-value moving image constituent images.

  In addition, it is preferable that the fixed value conversion part 211 makes the pixel value of the same position temporally unchanged among several frame images. Moreover, it is preferable that the fixed value unit 211 makes the fixed value by making the pixel values of the regions other than the inclusion region in one frame image the same. Moreover, it is preferable that the fixed value unit 211 at least make the pixel values in the blocks included in the region other than the inclusion region in one frame image the same.

  The image comparison unit 741 compares at least one moving image constituent image among a plurality of fixed moving image constituent images with other moving image constituent images. Then, the encoding unit encodes at least one moving image constituent image based on the comparison result by the image comparing unit 741, thereby encoding a plurality of fixed moving image constituent images.

  The moving image dividing unit 731 divides the moving image into a plurality of partial moving images each including a plurality of moving image constituent images. Specifically, the background area moving image dividing unit 731a divides the background area moving image whose image quality is reduced by the image quality reducing unit 221a. In addition, each of the feature region moving image dividing unit 731b, the feature region moving image dividing unit 731c, and the feature region moving image dividing unit 731d divides the background region moving image in which the image quality reducing units 221b, 221c, and 221d have reduced the image quality.

  Then, the inclusion area specifying unit 870 specifies an inclusion area that includes a plurality of feature areas in each of the plurality of moving image constituent images included in the partial moving image in each of the plurality of partial moving images. Then, the fixed value unit 211 converts the pixel values of the areas other than the inclusion areas in the plurality of moving image constituent images included in the partial moving images into fixed values in each of the plurality of partial moving images.

  Then, the image comparison unit 741 compares at least one moving image constituent image with another moving image constituent image included in the same partial moving image in each of the plurality of partial moving images. Specifically, the background region image comparison unit 741a compares at least one moving image constituent image with another moving image constituent image included in the same partial moving image in each of the plurality of partial moving images divided by the background region moving image dividing unit 731a. To do. In addition, the feature area image comparison unit 741b compares at least one moving image constituent image with another moving image constituent image included in the same partial moving image in each of the plurality of partial moving images divided by the characteristic region moving image dividing unit 731b. Similarly, the feature region image comparison unit 741c compares at least one moving image constituent image with another moving image constituent image included in the same partial moving image in each of the plurality of partial moving images divided by the characteristic region moving image dividing unit 731c. The feature area image comparison unit 741d compares at least one moving image constituent image with another moving image constituent image included in the same partial moving image in each of the plurality of partial moving images divided by the characteristic region moving image dividing unit 731d.

  Then, the encoding unit 231 encodes at least one moving image constituent image based on the comparison result by the image comparison unit 741 in each of the plurality of partial moving images, so that a plurality of fixed moving image constituent images are obtained. Encode for each of a plurality of partial videos.

  Note that the inclusion area specifying unit 870 specifies a predetermined collection area of partial areas including a plurality of feature areas in each of the plurality of moving image constituent images as an inclusion area. Then, in each of the plurality of partial videos, the image comparison unit 741 compares each of the plurality of partial areas included in at least one video composition image with other video composition images included in the same partial video. Then, the encoding unit 231 compresses at least one moving image constituent image for each partial region based on the comparison result by the image comparison unit 741 in each of the plurality of partial moving images, thereby fixing the plurality of fixed moving images. A constituent image is encoded for each of a plurality of partial moving images.

  Specifically, the image comparison unit 741 includes an image included in another moving image constituent image included in the same partial moving image for each of a plurality of partial areas included in at least one moving image forming image in each of the plurality of partial moving images. The difference between the images is calculated. Then, the encoding unit 231 sets the fixed value by encoding at least one moving image constituent image for each partial region based on the difference amount calculated by the image comparison unit 741 in each of the plurality of partial moving images. A plurality of moving image constituent images are encoded for each of a plurality of partial moving images.

In each of the plurality of partial videos, the image comparison unit 741 compares each partial area included in at least one moving picture constituent image with the partial areas compared to other moving picture constituent images included in the same partial moving picture. A difference target area in another moving image constituent image that is an area where the difference amount of the image is smaller than a predetermined value is specified, and the difference amount of the image with the specified difference target area is calculated.
The encoding unit 231 encodes at least one moving image constituent image for each partial region based on the difference amount calculated by the image comparison unit 741 and the difference between the position of the difference target region and the position of the partial region. A plurality of fixed-value moving image constituent images are encoded for each of a plurality of partial moving images. Note that the difference between the position of the difference target area and the position of the partial area may be a movement vector in MPEG coding as an example.

  The inclusion area specifying unit 870 specifies, as an inclusion area, a block aggregation area in MPEG encoding that includes a plurality of feature areas in each of a plurality of moving image constituent images. Then, in each of the plurality of partial videos, the image comparison unit 741 compares each of a plurality of blocks or macroblocks included in at least one video composition image with other video composition images included in the same partial video, The difference amount between the motion vector and the image is calculated.

  Then, the encoding unit 231 encodes at least one moving image constituent image for each block and macroblock based on the motion vector calculated by the image comparison unit 741 and the difference amount between the images, thereby obtaining a plurality of fixed values. Are encoded for each of a plurality of partial moving images. Note that the inclusion area specifying unit 870 may specify a collection area of macroblocks or slices in MPEG encoding that includes a plurality of feature areas in each of a plurality of moving image constituent images.

  The boundary area specifying unit 872 specifies a boundary area that is a partial area including both the characteristic area and a boundary other than the characteristic area, from among the plurality of partial areas including the plurality of characteristic areas in each of the plurality of moving image constituent images. The peripheral region specifying unit 874 specifies a peripheral region that is a partial region that is an aggregate region of partial regions that are adjacent to the boundary region and do not include a feature region. Then, the inclusion region determination unit 876 determines a partial region, a boundary region, and a peripheral region that include the feature region but does not include the region other than the feature region as the inclusion region.

  The peripheral region specifying unit 874 specifies a boundary region including a feature region specified from another moving image constituent image to be compared by the image comparison unit 741 among the boundary regions specified by the boundary region specifying unit 872. You can do it. Then, the peripheral area specifying unit 874 may specify a partial area that is adjacent to the specified boundary area and does not include the specified characteristic area as the peripheral area.

  Note that the compressed moving image acquisition unit 201 acquires a plurality of images captured by each of the plurality of imaging devices 100. The compressed moving image acquisition unit 201 may be an example of an image acquisition unit in the present invention. Then, the feature region detection unit 203 detects the feature region from one of the plurality of images. Then, the corresponding area specifying unit 820 specifies a corresponding area that should include the same subject as the subject included in the feature area in another image of the one image. Then, the feature region specifying unit 810 specifies the feature region in the other image from the corresponding region in the other image specified by the corresponding region specifying unit 820.

  Note that the feature region specifying unit 810 may specify the corresponding region specified by the corresponding region specifying unit 820 as a feature region in another image. In addition, the feature area specifying unit 810 may extract a feature area in another image from the corresponding area specified by the corresponding area specifying unit 820.

  The compressed moving image acquisition unit 201 acquires a plurality of moving image constituent images included in each of a plurality of moving images captured by each of the plurality of imaging devices 100. Then, the feature region detection unit 203 detects a feature region from a plurality of moving image constituent images included in one moving image of the plurality of moving images. Then, the corresponding area specifying unit 820 specifies corresponding areas in a plurality of moving image constituent images included in other moving images that should include the same subject as the subject included in the feature area.

  Then, the feature region specifying unit 810 specifies the feature region in the moving image constituent image included in the other moving image from the corresponding region in the moving image constituent image included in the other moving image specified by the corresponding region specifying unit 820. Note that the corresponding feature region specifying unit 810 is based on the position of the feature region detected by the feature region detection unit 203 in the image and each of the imaging ranges when the plurality of imaging devices capture a plurality of images. May be specified.

  The degree-of-fit calculation unit 840 conforms to a predetermined condition for each of the image content of the feature region image detected by the feature region detection unit 203 and the image content of the feature region image specified by the feature region specification unit 810. Calculate the goodness of fit. Then, the compression unit 240 includes an image of the feature region in which the fitness level calculation unit 840 calculates a higher fitness level among the feature region detected by the feature region detection unit 203 and the feature region specified by the feature region specification unit 810. Compress feature area video with less intensity.

  The condition storage unit 850 stores a condition to which the image content of the area to be specified as the characteristic area should be adapted. Then, the feature region detection unit 203 identifies, as a feature region, a region that meets the conditions stored in the condition storage unit 850 based on the image content of one of the plurality of images acquired by the image acquisition unit. To do. Then, the feature region specifying unit 810 is configured to select a region that matches a condition relaxed from the condition stored in the condition storage unit 850 based on the image content of the image of the corresponding region specified by the corresponding region specifying unit 820. It may be extracted as a region.

  Note that the object motion calculation unit 830 identifies the motion of the object included in the feature area detected by the feature area detection unit 203. Then, the corresponding area specifying unit 820 includes an area included in another moving image including an object indicating a movement that matches the movement of the object calculated by the object movement calculating unit 830, and a plurality of moving image constituent images included in the other moving image. You may specify as a corresponding | compatible area | region.

  FIG. 9 shows an example of the corresponding area specified by the corresponding area specifying unit 820. For example, it is assumed that the feature region detection unit 203 of the image processing device 120a specifies the ROI region 912 from the frame image 910 generated from the captured moving image captured by the imaging device 100a, for example. At this time, the corresponding area specifying unit 820 of the image processing apparatus 120b acquires information indicating the ROI area 912 in the frame image 910 from the image processing apparatus 120a. Then, the corresponding area specifying unit 820 of the image processing device 120b selects the same subject as the subject indicated by the object shown in the ROI region 912 in the frame image 910 from the frame image 920 generated from the captured moving image captured by the imaging device 100b. An area 922 to be included is specified.

  At this time, the corresponding area specifying unit 820 of the image processing apparatus 120b acquires information indicating the imaging range captured by the imaging apparatus 100a and the imaging range captured by the imaging apparatus 100b. Then, the corresponding area specifying unit 820 of the image processing apparatus 120b may specify the area 922 based on the acquired information and the position of the ROI area 912 in the frame image 910. Then, the feature area specifying unit 810 of the image processing apparatus 120b detects the ROI from the area 924 including the area 922 specified by the corresponding area specifying unit 820. Note that the feature region specifying unit 810 of the image processing apparatus 120b may specify the region 922 specified by the corresponding region specifying unit 820 as the ROI in the frame image 920.

  Note that the feature region detection unit 203 of the image processing device 120b overlaps the imaging range of the imaging device 100b that captured the captured moving image including the frame image 920 with the imaging range of the imaging device 100a or the imaging device 100c other than the imaging device 100b. The range is specified, and an overlapping imaging region that is an area on the frame image 920 in which the subject in the specified range is captured is specified. Then, the feature area detection unit 203 of the image processing apparatus 120b may detect the ROI from an area other than the overlapping imaging area in the frame image 920. Then, the image processing device 120b may notify the ROI detected by the feature region detection unit 203 to the corresponding region specification unit 820 of the other image processing device 120. Then, the feature area specifying unit 810 of the image processing apparatus 120b may detect the ROI from the area of the overlapping imaging area.

  As described above, the feature region specifying unit 810 is generated from the captured moving image captured by the other imaging device 100 based on the position of the ROI detected from the frame image generated from the captured moving image captured by the one imaging device 100. The ROI in the frame image thus detected is detected. Thereby, it may be possible to reduce the time required to detect the ROI.

  FIG. 10 shows an example of data stored in the condition storage unit 850 in a table format. The condition storage unit 850 stores the head shape coincidence degree and the face element coincidence degree. The head shape coincidence degree represents a coincidence degree between a head shape pattern predetermined as a contour shape of a person's head and an object outline extracted from the frame image by edge extraction or the like.

  In addition, the degree of matching of the face element is a match between the face pattern including the outline shape of the object such as eyes, mouth, and nose included in the human face and the outline of the object extracted from the frame image by edge extraction or the like. Degrees. The degree of coincidence between the pattern and the contour of the object may be a ratio of the area where the pattern and the object overlap with respect to the area of the pattern or the object when the pattern and the contour of the object are overlapped. .

  Then, the feature region detection unit 203 has a head shape matching degree larger than the head shape matching degree stored in the condition storage unit 850 and a face element matching degree stored in the condition storage unit 850. A larger object is identified as an object representing a person. Then, the feature area detection unit 203 determines an area including the identified object as an ROI.

  Here, the operation in which the feature region detection unit 203 detects the ROI based on the conditions stored in the condition storage unit 850 has been described, but the feature region specification unit 810 also has the same function as the feature region detection unit 203. The ROI may be detected from the region 924 based on the conditions stored in the condition storage unit 850.

  FIG. 11 shows an example of the object motion calculated by the object motion calculation unit 830. The object motion calculation unit 830 calculates the moving speed of the object based on the timing at which each of the plurality of frame images is captured and the position of the object included in the ROI in the plurality of frame images. Note that each object motion calculation unit 830 of each of the plurality of image processing devices 120 calculates the speed at which the object moves. In this way, the object motion calculation unit 830 calculates the temporal changes 1110 and 1120 of the speed. The time change 1110 indicates the time change of the speed calculated by the object motion calculation unit 830 of one image processing apparatus 120, and the time change 1120 indicates the speed calculated by the object motion calculation unit 830 of another image processing apparatus 120. The time change of is shown.

  Then, the compression control unit 860 compares the time change of the speed calculated by the object motion calculation unit 830 with the time change of the speed calculated by the object motion calculation unit 830 of the other image processing device 120 to obtain another image. An object that moves with a temporal change in speed that matches with a temporal change in speed that is calculated by the object motion calculation unit 830 of the processing device 120 and a degree of coincidence greater than a predetermined degree of coincidence is specified. Specifically, the compression control unit 860 may specify the timing at which the speed changes based on the time change of each speed, and may calculate a greater degree of coincidence as the specified timing difference is smaller. In the following description, the object is referred to as a matching object. In addition, an object that is extracted in another image processing apparatus 120 and whose time change in speed matches with a time change in speed of the matching object with a degree of coincidence greater than a predetermined coincidence is referred to as a corresponding object.

  Then, the compression controller 860 determines that the fitness calculated by the fitness calculator 840 is greater than the fitness calculated by the fitness calculator 840 of the other image processing device 120 for the ROI including the corresponding object. A smaller compression strength is determined as the compression strength for compressing the ROI including the matching object. Specifically, the compression control unit 860 compresses the ROI that includes the matching object as a compression strength that is smaller than the compression strength that the compression unit 240 of the other image processing apparatus 120 compresses the ROI that includes the corresponding object. decide. Then, the compression control unit 860 causes the compression unit 240 to compress the determined compression strength.

  Note that the compression control unit 860 determines the fitness calculated by the fitness calculation unit 840 of the other image processing device 120 and the speed calculated by the object motion calculation unit 830 of the other image processing device 120 from the other image processing device 120. The above processing may be performed by acquiring. In addition, the corresponding region specifying unit 820 compares the time change of the speed calculated by the object motion calculation unit 830 with the time change of the speed calculated by the object motion calculation unit 830 of the other image processing device 120, An area including an object moving with a temporal change in velocity that matches with a temporal change in velocity that is calculated by the object motion calculation unit 830 of the image processing apparatus 120 with a degree of coincidence greater than a predetermined coincidence is defined as an ROI that should include the same subject. You may decide.

  In this way, the compression control unit 860 increases the conformity that matches the conditions stored in the condition storage unit 850 with the compression strength of the ROI including the same subject as the ROI detected by the other image processing device 120. The value is lowered, and the degree of conformity that matches the condition stored in the condition storage unit 850 is increased as the degree of adaptation is lower. By such control, when the same subject is imaged by another imaging device 100, the image processing device 120 dynamically changes the compression strength for compressing the ROI including the object indicating the subject according to the degree of fitness. Can be increased or decreased. Accordingly, the total data amount of the moving image data output from the plurality of image processing devices 120 is reduced as compared with the case where the compression strength is not changed even though the same subject is captured by the plurality of imaging devices 100. In some cases, it is possible to provide a higher-quality image of a notable subject.

  FIG. 12 shows an example of the time evolution of the data amount of the moving image data output from each of the image processing devices 120. As already described, each of the plurality of image processing devices 120 can dynamically increase or decrease the compression strength according to the image content of the subject. Here, the total amount of moving image data output from the plurality of image processing apparatuses 120 may be determined in advance. For example, the total code amount of the moving image data output from the plurality of image processing apparatuses 120 may be determined in advance.

  In such a case, the compression control unit 860 determines the compression strength for compressing the feature area moving image and the background area moving image so that the total data amount becomes a predetermined amount. Specifically, the compression control unit 860 determines the degree of fitness already described, the type of ROI, the number of detected ROIs, the area of the detected ROI, the position of the detected ROI, the orientation of the person included in the ROI, When encoding each of a plurality of feature region moving images and background region moving images with an assigned code amount for each type of ROI determined in advance in association with the shape of the object included in the ROI, the type of object included in the ROI, etc. It is determined as the amount of assigned code.

  Then, the compression control unit 860 calculates a value obtained by dividing a predetermined code amount by the total value of the determined allocation code amounts. Then, the compression control unit 860 uses the calculated value as a coefficient, and multiplies the allocation code amount in the case of encoding each of the plurality of characteristic area moving images and the background area moving image. The compression control unit 860 determines a value obtained by multiplying the coefficient as an allocation code amount when each of the plurality of feature region moving images and the background region moving image is encoded. Thereby, the total value of the code amount of the moving image data output from the plurality of image processing devices 120 can be made substantially constant. Note that the allocated code amount may use the VBV buffer amount as an index.

  As described above, the compression unit 240 has the code amount of the feature region moving image including the feature region image detected by the feature region detecting unit 203 and the feature region moving image including the image of the feature region specified by the feature region specifying unit 810. A feature region including a feature region moving image including a feature region image detected by the feature region detecting unit 203 and a feature region image specified by the feature region specifying unit 810 so that the total value of the code amounts falls within a predetermined range. Adjust the compression strength of each video.

  FIG. 13 shows an example of the inclusion area specified by the inclusion area specifying unit 870. Here, a part of the contour of the ROI detected by the feature region detection unit 203 is indicated by reference numeral 1370. Then, the macroblocks inside the ROI and in the vicinity of the contour of the ROI are macroblocks 1301-1304, macroblocks 1311-1315, macroblocks 1321-1326, macroblocks 1331-1337, macroblocks 1341-1347, macroblocks 1351-1357. It shows with.

  The boundary area specifying unit 872 specifies the macroblocks 1311, 1312, 1313, 1323, 1324, 1334, 1335, 1345, and 1355 including the ROI contour 1370 detected by the feature area detecting unit 203. The peripheral area specifying unit 874 specifies macroblocks 1301-1303, 1314, 1325, 1336, 1346, and 1356 that do not include the ROI and are adjacent to the macroblock specified by the boundary area specifying unit 872. Then, the inclusion region determination unit 876 includes the macroblocks 1301-1303, 1314, 1325, 1336, 1346, and 1356 specified by the peripheral region specification unit 874 in addition to the macroblocks including at least the region in the contour 1370. Are determined as inclusion regions. In this way, the inclusion area specifying unit 870 specifies the boundary of the inclusion area indicated by the outline 1380.

  As described above, the inclusion region specifying unit 870 includes a peripheral region including one or more macroblocks in the inclusion region. The fixed value unit 211 converts the area other than the inclusion area to a fixed value. Thereby, there is a case where the possibility that the macroblock in the inclusion area is encoded with a difference between the macroblock and the area fixed by the fixed value conversion unit 211 may be reduced. Therefore, there is a case where it is possible to prevent the data amount of the encoded moving image data from increasing due to the fixed value.

  Note that the fixed value unit 211 may fix the pixel values outside the inclusion area to a fixed value with an average luminance value or average color within the inclusion area. Thus, when a macroblock in the inclusion area is encoded, even if the pixel value of the macroblock in the inclusion area is different from the pixel value of the fixed area, the difference value Can be reduced in size.

  The macroblock may be an example of a partial area in the present invention. In addition, the partial area in the present invention may be a block, a slice, or the like. When the partial region is a slice, the inclusion region determination unit 876 may determine a region including a slice including the ROI among the slices including a plurality of macroblocks continuous in the X direction as the inclusion region. . Also, the inclusion region determination unit 876 may determine a region including a slice including the ROI among the slices including a part of a plurality of macroblocks continuous in the X direction including the ROI as the inclusion region. As described above, the inclusion region determination unit 876 may determine a block region, a macroblock unit, a slice unit, and a combination region of a combination thereof as the inclusion region.

  Note that the inclusion region determination unit 876 may determine, for example, a region including a ROI peripheral region included in a frame image encoded as an I picture or a P picture in MPEG encoding as the inclusion region. In addition, for the frame image encoded into the B picture, the inclusion region determination unit 876 may determine the region including the boundary region specified by the boundary region specification unit 872 in the ROI as the inclusion region.

  FIG. 14 shows an example of the inclusion area specified by the inclusion area specifying unit 870. The inclusion area specifying unit 870 specifies an inclusion area obtained by superimposing a plurality of ROIs detected by the feature area detection unit 203 from each of a plurality of frame images to be encoded as 1 GOP. At this time, the inclusion region specifying unit 870 superimposes the ROIs detected by the feature region detection unit 203 from a plurality of frame images included in one GOP. Thus, the inclusion area specifying unit 870 specifies the inclusion area fixed for each GOP. In addition, as described in relation to FIG. 13, the inclusion area specifying unit 870 may specify an area including the peripheral area as the inclusion area.

  As described above, the inclusion area specifying unit 870 specifies the inclusion area for each GOP, and the fixed value converting unit 211 fixes the areas other than the inclusion area. For this reason, not only the data amount can be reduced by the fixed value, but also the data amount after encoding can be significantly reduced as described with reference to FIG. For this reason, even if the function of the encoding unit 231 is implemented by a general-purpose encoder, the amount of data may be effectively reduced. For this reason, the cost of the image processing apparatus 120 may be significantly reduced as compared with the case where the function of the encoding unit 231 is implemented by an encoder specialized for encoding the ROI region.

  FIG. 15 shows an example of the hardware configuration of the image processing apparatus 120 and the image processing apparatus 170. The image processing device 120 and the image processing device 170 include a CPU peripheral unit, an input / output unit, and a legacy input / output unit. The CPU peripheral section includes a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 that are connected to each other by a host controller 1582. The input / output unit includes a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 that are connected to the host controller 1582 by the input / output controller 1584. The legacy input / output unit includes a ROM 1510, a flexible disk drive 1550, and an input / output chip 1570 connected to the input / output controller 1584.

  The host controller 1582 connects the RAM 1520, the CPU 1505 that accesses the RAM 1520 at a high transfer rate, and the graphic controller 1575. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 to control each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505. The communication interface 1530 is connected to the network communication device 1598 to transmit / receive programs or data. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program that is executed when the radiation imaging system is started up, a program that depends on the hardware of the radiation imaging system, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via the flexible disk drive 1550 or a parallel port, serial port, keyboard port, mouse port, and the like.

  A program executed by the CPU 1505 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program stored in the recording medium may be compressed or uncompressed. The program is installed in the hard disk drive 1540 from the recording medium, read into the RAM 1520, and executed by the CPU 1505. The program executed by the CPU 1505 causes the image processing apparatus 120 to execute the compressed moving image acquisition unit 201, the compressed moving image decompression unit 202, the feature region detection unit 203, the image division unit 204, and the image generation described with reference to FIGS. Unit 205, fixed value conversion unit 211, image quality reduction unit 221, encoding unit 231, association processing unit 206, output unit 207, video segmentation unit 730, video segmentation unit 731, image comparison unit 741, feature region identification unit 810, Corresponding region specification unit 820, object motion calculation unit 830, fitness calculation unit 840, condition storage unit 850, compression control unit 860, inclusion region specification unit 870, boundary region specification unit 872, peripheral region specification unit 874, and inclusion region determination It functions as a unit 876. Further, the program executed by the CPU 1505 causes the image processing apparatus 170 to perform the compressed moving image acquisition unit 301, the association analysis unit 302, the compressed moving image decompression unit 311, the combining unit 303, and the like described with reference to FIGS. It functions as the output unit 304.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet is used as a recording medium, and is provided to the image processing device 120 and the image processing device 170 as a program via the network. Also good.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of an image processing system 10 according to an embodiment. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 120. FIG. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 170. FIG. 3 is a diagram illustrating an example of a processing flow of the image processing apparatus 120. FIG. It is a figure which shows an example of the image quality of several characteristic area | region moving image and background area | region moving image. It is a figure which shows an example of the processing flow of the image processing apparatus. 3 is a diagram illustrating an example of a configuration of a compression unit 240. FIG. It is a figure which shows an example of a structure of the compression control unit 800 which the image processing apparatus 120 further has. It is a figure which shows an example of the corresponding area which the corresponding area specific | specification part 820 specified. It is a figure which shows an example of the data which the condition storage part 850 has stored in the table format. It is a figure which shows an example of the motion of the object which the object motion calculation part 830 calculated. It is a figure which shows an example of the time development of the data amount of the moving image data which each of the image processing apparatus 120 outputs. It is a figure which shows a part of inclusion area which the inclusion area specification part 870 specifies. It is a figure which shows an example of the inclusion area which the inclusion area specification part 870 specified. 2 is a diagram illustrating an example of a hardware configuration of an image processing apparatus 120 and an image processing apparatus 170. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing system 100 Imaging device 110 Communication network 120 Image processing device 130 Person 140 Moving object 150 Monitoring object space 160 Space 170 Image processing device 180 Display device 201 Compressed moving image acquisition unit 202 Compressed moving image decompression unit 203 Feature area detection unit 204 Image division Unit 205 image generation unit 206 association processing unit 207 output unit 210 fixed value unit 211 fixed value unit 220 reduction unit 221 image quality reduction unit 230 encoding unit 231 encoding unit 231a background area moving image encoding unit 231b-d Features Area video encoding unit 240 Compression unit 301 Compressed video acquisition unit 302 Association analysis unit 310 Compressed video expansion unit 311 Compressed video expansion unit 303 Combining unit 304 Output unit 730 Video division unit 731 Video division unit 731a Background region video division unit 731 -D feature region moving image dividing unit 740 image comparison unit 741 image comparison unit 741a background region image comparison unit 741b-d feature region image comparison unit 800 compression control unit 810 feature region specifying unit 820 corresponding region specifying unit 830 object motion calculating unit 840 Degree calculation unit 850 Condition storage unit 860 Compression control unit 870 Inclusion region specifying unit 872 Boundary region specifying unit 874 Surrounding region specifying unit 876 Inclusion region determining unit

Claims (5)

A video dividing unit that divides a video into a plurality of partial videos including a plurality of video composition images;
From each of the plurality of video configuration images, a feature region detecting unit for detecting a feature area,
An inclusion area specifying unit for specifying, as an inclusion area , a collection area of blocks, macroblocks, or slices in MPEG encoding, which includes the plurality of feature areas detected from each of the plurality of moving image constituent images ;
A fixed value generating section for a fixed value of the pixel value of the region other than the inclusion region at each of the plurality of video configuration images,
Other moving picture constituent images included in the same partial moving picture as the at least one moving picture constituent image for each of a plurality of blocks or macroblocks included in at least one moving picture constituent image among the plurality of moving picture constituent images that have been fixed. An image comparison unit that calculates a difference amount and a motion vector of an image between other moving image constituent images included in the same partial moving image, and
The plurality of moving images that have been fixed values by encoding the at least one moving image constituting image for each block and macroblock based on the difference amount of the image calculated by the image comparison unit and the motion vector. An image processing apparatus comprising: an encoding unit that encodes a component image for each of the plurality of partial moving images. The inclusion region specifying part is:
And the boundary region specifying unit for specifying a boundary region which is a partial region including both the boundary other than the pre-Symbol feature region and the feature region that put on each of the plurality of video configuration images,
And a peripheral region specifying unit the adjacent border regions, to identify have peripheral regions including the feature region,
The characteristic part does not include a region other than the region area containing the characteristic region, the boundary region, and the image processing apparatus according to the peripheral region, to claim 1 having the inclusion area determination unit that determines as the inclusion area. The peripheral region specifying unit includes the boundary region including the feature region specified from the other moving image constituent image to be compared by the image comparison unit among the boundary regions specified by the boundary region specifying unit. The image processing apparatus according to claim 2 , wherein a partial region that is adjacent to the identified boundary region and does not include the identified feature region is identified as the peripheral region. A video splitting stage that splits the video into multiple partial videos containing multiple video composition images,
From each of the plurality of video configuration images, a feature region detecting step of detecting a characteristic region,
An inclusion region specifying step for specifying, as an inclusion region , a block, macroblock, or slice aggregation region in MPEG encoding that includes a plurality of the feature regions detected from each of the plurality of moving image constituent images ;
A value fixing step of fixing value of the pixel value of the region other than the inclusion region at each of the plurality of video configuration images,
Other moving picture constituent images included in the same partial moving picture as the at least one moving picture constituent image for each of a plurality of blocks or macroblocks included in at least one moving picture constituent image among the plurality of moving picture constituent images that have been fixed. An image comparison step for calculating an image difference amount and a motion vector between other moving image constituent images included in the same partial moving image, and
By encoding the at least one moving image constituent image for each block and macroblock based on the difference amount of the image calculated in the image comparison step and the motion vector, the plurality of fixed values An image processing method comprising: an encoding step of MPEG encoding a moving image constituent image for each of the plurality of partial moving images. A program for an image processing device, wherein the image processing device is
Video segmentation unit that divides a video into multiple partial videos containing multiple video composition images
From each of the plurality of video configuration images, feature region detecting section that detects a characteristic region,
An inclusion area specifying unit for specifying, as an inclusion area , a block, macroblock, or slice aggregation area in MPEG encoding that includes the plurality of feature areas detected from each of the plurality of moving image constituent images ;
Fixed value generating section for a fixed value of the pixel value of the region other than the inclusion region at each of the plurality of video configuration images,
Other moving picture constituent images included in the same partial moving picture as the at least one moving picture constituent image for each of a plurality of blocks or macroblocks included in at least one moving picture constituent image among the plurality of moving picture constituent images that have been fixed. An image comparison unit that calculates a difference amount and a motion vector of an image between other moving image constituent images included in the same partial moving image
The plurality of moving images that have been fixed values by encoding the at least one moving image constituting image for each block and macroblock based on the difference amount of the image calculated by the image comparison unit and the motion vector. A program that causes a constituent image to function as an encoding unit that performs MPEG encoding for each of the plurality of partial moving images .

Images