JP5117424B2 - Image processing system, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing system, an image processing method, and a program. The present invention particularly relates to an image processing system and an image processing method for processing an image, and a program for the image processing system.

  There is known a vehicle number recognition system that recognizes a vehicle number from images from a plurality of cameras set with overlapping imaging areas (see, for example, Patent Document 1). In this car number recognition system, when a moving body is detected by the head camera, each camera picks up the image simultaneously, processes the image from the corresponding camera, narrows down the license plate area, and recognizes the car number by character recognition. And partial data for each element such as a serial number is extracted. Then, completion data is created by combining the partial data of the same vehicle number and output to the central control device or the like.

  Also, using a single surveillance video camera, a low-resolution whole image and a high-resolution partial image are recorded, and the tracking video is displayed in high definition or enlarged according to the instructions of the surveillance operator. A monitoring device is known (for example, see Patent Document 2).

JP-A-8-5316 JP 2006-33793 A

  With the technique described in the above-mentioned patent document, it is not possible to quickly extract a frame of a portion where the same object is captured from each video obtained by each camera.

  In order to solve the above problems, according to a first aspect of the present invention, there is provided an image processing system, an image acquisition unit that acquires a plurality of moving images captured by each of a plurality of imaging devices, and a plurality of moving images Representing a plurality of feature regions from among a plurality of feature regions having a correlation higher than a predetermined value and a feature region detection unit that detects a plurality of feature regions from a plurality of moving image constituent images included in each A plurality of moving images having a plurality of feature regions having a correlation higher than a predetermined value between a representative feature region determination unit that determines a representative feature region that is a feature region and representative feature region information that is information for specifying the representative feature region And an output unit that outputs the image in association with the component image.

  According to the second aspect of the present invention, there is provided an image processing method, an image acquisition stage for acquiring a plurality of moving images captured by each of a plurality of imaging devices, and a plurality of moving image configurations included in each of the plurality of moving images. A feature region detection stage that detects a plurality of feature regions from an image, and a representative feature region that is a feature region representing the plurality of feature regions from among a plurality of feature regions having a correlation higher than a predetermined value. The representative feature region determination stage to be determined and the representative feature region information, which is information for specifying the representative feature region, are output in association with a plurality of moving image constituent images having a plurality of feature regions having a correlation higher than a predetermined value. Output stage.

  According to the third aspect of the present invention, there is provided a program for an image processing system, wherein a computer is included in each of a plurality of moving images, an image acquisition unit that acquires a plurality of moving images captured by each of a plurality of imaging devices. A feature region detection unit for detecting a plurality of feature regions from a plurality of moving image constituent images, a feature region representing the plurality of feature regions from a plurality of feature regions having a correlation higher than a predetermined value A representative feature region determining unit that determines a representative feature region, and representative feature region information that is information for specifying the representative feature region corresponds to a plurality of moving image constituent images having a plurality of feature regions having a correlation higher than a predetermined value. It is made to function as an output unit that outputs.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 is a diagram illustrating an example of an image processing system 10 according to an embodiment. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 120. FIG. 3 is a diagram illustrating an example of a block configuration of a compression unit 230. FIG. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 170. FIG. It is a figure which shows an example of the other block structure of the compression part 230. FIG. It is a figure which shows an example of the moving image obtained from the different imaging device 100. FIG. It is a figure which shows the state with which the captured image and the characteristic area information were linked | related by the matching process part 206 in a table format. 6 is a diagram showing an example of a search instruction reception screen 800 displayed on the display device 180. FIG. 6 is a diagram showing a moving image of a representative feature area displayed by the display device 180. FIG. It is a figure which shows an example of the calculation parameter which calculates a correlation. 2 is a diagram illustrating an example of a hardware configuration of an image processing device 120 and an image processing device 170. FIG.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of an image processing system 10 according to an embodiment. The image processing system 10 can function as a monitoring system as described below.

  The image processing system 10 includes a plurality of imaging devices 100a to 100d (hereinafter collectively referred to as imaging devices 100) that capture the monitoring target space 150, an image processing device 120 that processes captured images captured by the imaging device 100, and communication. It includes a network 110, an image processing device 170, an image DB 175, and a plurality of display devices 180a-d (hereinafter collectively referred to as display devices 180). The image processing device 170 and the display device 180 are provided in a space 160 different from the monitoring target space 150.

  The imaging device 100a includes an imaging unit 102a and an imaging moving image compression unit 104a. The imaging unit 102a captures a plurality of captured images by continuously capturing the monitoring target space 150. Then, the imaging unit 102a generates a captured moving image including a plurality of captured images. The captured moving image compression unit 104a compresses the captured moving image captured by the imaging unit 102a by MPEG encoding or the like, and generates captured moving image data. Thus, the imaging device 100a encodes the captured moving image obtained by capturing the monitoring target space 150 to generate captured moving image data. The imaging device 100a outputs the captured moving image data to the image processing device 120.

  Note that the imaging unit 102a may continuously capture the subject at an imaging rate higher than a predetermined reference imaging rate. For example, the imaging unit 102a may continuously image the subject at an imaging rate higher than a reference imaging rate that is determined in advance according to the movement speed of the monitoring target that is the monitoring target.

  Note that the imaging device 100b, the imaging device 100c, and the imaging device 100d have the same configuration as that of the imaging device 100a, and thus description of each component of the imaging device 100b, the imaging device 100c, and the imaging device 100d is omitted. In this way, the image processing device 120 acquires the captured moving image data generated by each of the plurality of imaging devices 100 from each of the plurality of imaging devices 100.

  Then, the image processing device 120 decodes the captured moving image data acquired from the imaging device 100 and acquires the captured moving image. The image processing apparatus 120 has different types of features such as an area where a person 130 is imaged, an area where a moving body 140 such as a vehicle is imaged, etc., from each of a plurality of captured images included in the acquired captured video. A plurality of feature regions are detected. Then, the image processing apparatus 120 identifies a set of feature areas that have a high correlation with respect to the image content of the detected feature area from among the captured images included in each captured moving image. Then, the image processing apparatus 120 determines a representative feature area, which is a representative feature area, from the identified set of feature areas, identification information for identifying the representative feature area, and information for identifying each captured image Is generated in correspondence with the feature region information.

  Note that the image processing device 120 generates feature area information including information for specifying the feature area detected from the captured image. Then, the image processing apparatus 120 attaches the feature area information to the compressed moving image data and transmits the feature area information to the image processing apparatus 170 through the communication network 110.

  The image processing device 170 receives the compressed moving image data associated with the feature area information from the image processing device 120. Then, the image processing device 170 expands the received compressed moving image data using the associated feature area information to generate a display moving image, and supplies the generated display moving image to the display device 180. The display device 180 displays the moving image for display supplied from the image processing device 170.

  The image processing apparatus 170 may record the compressed moving image data in the image DB 175 in association with the feature area information associated with the compressed moving image data. Then, in response to a request from the display device 180, the image processing device 170 reads the compressed moving image data and the feature area information from the image DB 175, decompresses the read compressed moving image data using the feature region information, and displays it. A moving image may be generated and supplied to the display device 180.

  The feature region information is text data including identification information for identifying the representative feature region, the position of the feature region, the size of the feature region, the number of the feature regions, the identification information for identifying the captured image in which the feature region is detected, and the like. Alternatively, the text data may be data that has been subjected to processing such as compression and encryption. Then, the image processing apparatus 170 displays an image of the representative feature area and presents it to the supervisor. Then, when the image of the representative feature region in which the subject to be searched is captured is selected by the monitor, the image processing apparatus 170 displays the captured image associated with the identification information for identifying the selected representative feature region. The selected captured images are continuously displayed as moving images on the display device 180.

  As described above, according to the image processing system 10, since the representative feature region is recorded in association with the moving image, it is possible to quickly search and find a picked-up image group that meets a predetermined condition in the moving image. Also, according to the image processing system 10, only a captured image group that meets a predetermined condition can be decoded, so that a partial moving image that meets the predetermined condition can be displayed promptly in response to a reproduction instruction.

  FIG. 2 shows an example of a block configuration of the image processing apparatus 120. The image processing apparatus 120 includes an image acquisition unit 250, a feature region detection unit 203, a compression control unit 210, a feature region information storage unit 220, a compression unit 230, a correlation calculation unit 260, a representative feature region determination unit 270, and an association processing unit 206. , And an output unit 207. The image acquisition unit 250 includes a compressed moving image acquisition unit 201 and a compressed moving image expansion unit 202.

  The compressed moving image acquisition unit 201 acquires a compressed moving image. Specifically, the compressed moving image acquisition unit 201 acquires encoded captured moving image data generated by the imaging device 100. The compressed moving image expansion unit 202 expands the captured moving image data acquired by the compressed moving image acquisition unit 201 and generates a plurality of captured images included in the captured moving image. Specifically, the compressed moving image decompression unit 202 decodes the encoded captured moving image data acquired by the compressed moving image acquisition unit 201, and generates a plurality of captured images included in the captured moving image. The captured image included in the captured moving image may be a frame image and a field image. The captured image in the present embodiment may be an example of a moving image constituent image in the present invention. In this way, the image acquisition unit 250 acquires a plurality of moving images captured by each of the plurality of imaging devices 100.

  The plurality of captured images obtained by the compressed moving image decompression unit 202 are supplied to the feature region detection unit and the compression unit 230. The feature region detection unit 203 detects a feature region from a moving image including a plurality of captured images. Specifically, the feature region detection unit 203 detects a feature region from each of the plurality of captured images. Note that the captured moving image described above may be an example of a moving image in the following description.

  For example, the feature region detection unit 203 may detect a region including an object indicating a moving object as a feature region. Note that the feature region detection unit 203 may detect a plurality of feature regions having different types of features from each of the plurality of captured images. Note that the type of feature may be an index of the type of object such as a person and a moving object. The type of the object may be determined based on the degree of coincidence of the shape of the object or the color of the object. As described above, the feature region detection unit 203 may detect a plurality of feature regions having different types of included objects from a plurality of captured images.

  For example, the feature region detection unit 203 extracts an object that matches a predetermined shape pattern with a matching degree equal to or higher than a predetermined matching degree from each of the plurality of picked-up images, and in the picked-up image including the extracted object. The region may be detected as a feature region having the same feature type. Note that a plurality of shape patterns may be determined for each type of feature. As an example of the shape pattern, a shape pattern of a human face can be exemplified. Different face patterns may be determined for each of a plurality of persons. Thereby, the feature region detection unit 203 can detect different regions each including a different person as different feature regions.

  As described above, the feature region detection unit 203 detects a plurality of feature regions from a plurality of captured images included in each of the plurality of moving images. Then, the feature region detection unit 203 supplies information indicating the detected feature region to the compression control unit 210 and the feature region information storage unit 220. Note that the information indicating the feature region includes coordinate information of the feature region indicating the position of the feature region, type information indicating the type of the feature region, and information for identifying the captured moving image in which the feature region is detected.

  The feature region information storage unit 220 stores information for identifying the images of the plurality of feature regions detected by the feature region detection unit 203 from the plurality of captured images included in the plurality of moving images and the positions and sizes of the feature regions. Note that the feature area information storage unit 220 may temporarily store an image of the feature area. The feature area information storage unit 220 may include a volatile memory that temporarily stores an image of the feature area.

  The correlation calculation unit 260 calculates a correlation between a plurality of feature regions stored in the feature region information storage unit 220. Specifically, the correlation calculation unit 260 may calculate a correlation value related to the image content of the feature region. More specifically, the correlation calculation unit 260 may calculate a correlation value using the degree of coincidence of the image contents of the feature region as an index. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions having a higher degree of coincidence of image contents.

  In addition, the correlation calculation unit 260 may calculate a correlation value related to the type of object included in the feature region image. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions including the same type of object. In addition, the correlation calculation unit 260 may calculate a correlation value related to the shape of the object included in the feature region image. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions having a greater degree of matching of the shapes of the included objects. In addition, the correlation calculation unit 260 may calculate a correlation value related to the color included in the image of the feature region. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions including more similar colors.

  In addition, the correlation calculation unit 260 may calculate a correlation value related to the shape of the feature region. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions having a higher degree of matching of the shape of the feature regions. In addition, the correlation calculation unit 260 may calculate a correlation value related to the size of the feature region. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions having a larger degree of coincidence of feature region sizes. In addition, the correlation calculation unit 260 may calculate a correlation value related to the position of the feature region. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions present at closer positions on the captured image.

  In addition, the correlation calculation unit 260 may calculate a correlation value related to a change in the position of the feature region. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions having a greater degree of coincidence of changes in the positions of the feature regions. Specifically, the correlation calculation unit 260 may calculate a correlation value related to the moving direction of the position of the feature region. The correlation calculation unit 260 may calculate a higher correlation value between feature regions having a greater degree of coincidence in the moving direction of the position of the feature region. In addition, the correlation calculation unit 260 may calculate a correlation value related to the moving speed of the position of the feature region. For example, the correlation calculation unit 260 may calculate a higher correlation value between feature regions having a higher degree of coincidence of movement speeds of the positions of the feature regions.

  Then, the representative feature region determination unit 270 selects a plurality of feature regions having a correlation value higher than a predetermined value based on the correlation value calculated by the correlation calculation unit 260. Then, the representative feature region determining unit 270 determines a representative feature region that is a feature region representing the plurality of feature regions from the plurality of feature regions having a correlation higher than the predetermined value.

  Specifically, the representative feature region determining unit 270 determines a representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the image content of the feature region. For example, the representative feature region determination unit 270 may determine the representative feature region from among a plurality of feature regions in which the degree of coincidence of the image contents of the feature region is greater than a predetermined value.

  In addition, the representative feature region determination unit 270 selects a feature region that satisfies a predetermined condition regarding the image content from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the image content of the feature region. The representative feature area may be determined. For example, the representative feature region determination unit 270 may determine a feature region including a subject image that easily identifies a captured subject as a representative feature region. As an example, the representative feature region determination unit 270 may determine a feature region including a larger face image with higher priority as the representative feature region. In addition, the representative feature region determination unit 270 may determine a feature region including a face image obtained by imaging from an angle closer to a predetermined angle with higher priority as the representative feature region.

  In addition, the representative feature region determination unit 270 may determine the representative feature region based on another correlation value calculated by the correlation calculation unit 260. For example, the representative feature region determination unit 270 may determine a representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the type of object included in the feature region image. The representative feature region determination unit 270 may determine a representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the shape of the object included in the feature region image. Further, the representative feature region determination unit 270 may determine a representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to colors included in the image of the feature region.

  Further, the representative feature region determination unit 270 may determine the representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the shape of the feature region. Further, the representative feature region determination unit 270 may determine a representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the size of the feature region.

  Further, the representative feature region determination unit 270 may determine the representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to the position of the feature region. Specifically, the representative feature region determination unit 270 may determine a representative feature region from among a plurality of feature regions having a correlation higher than a predetermined value with respect to a change in the position of the feature region. More specifically, the representative feature region determination unit 270 may determine a representative feature region from a plurality of feature regions having a correlation higher than a predetermined value with respect to the moving direction of the feature region position. In addition, the representative feature region determination unit 270 may determine the representative feature region from a plurality of feature regions having a correlation higher than a predetermined value with respect to the moving speed of the position of the feature region.

  The compression control unit 210 controls the compression processing of the moving image by the compression unit 230 according to the feature region based on the information indicating the feature region acquired from the feature region detection unit 203. The compression unit 230 compresses a plurality of captured images by reducing the image quality of regions other than the feature region detected by the feature region detection unit 203. Note that the compression unit 230 generates a compressed moving image including a plurality of compressed captured images. A more specific compression operation of the compression unit 230 will be described later.

  The association processing unit 206 associates the information specifying the feature region detected from the captured image and the information specifying the representative feature region determined by the representative feature region determining unit 270 with the captured image. Specifically, the association processing unit 206 uses information for specifying the feature region detected from the captured image and information for specifying the representative feature region determined by the representative feature region determining unit 270, and the captured image as a moving image constituent image. Corresponding to the compressed video included. Then, the output unit 207 outputs the compressed moving image in which the feature area is associated by the association processing unit 206 to the image processing apparatus 170.

  As described above, the output unit 207 outputs representative feature region information, which is information for specifying a representative feature region, in association with a plurality of captured images having a plurality of feature regions having a correlation higher than a predetermined value. . Accordingly, the image processing apparatus 170 can quickly select a captured image associated with information for identifying a representative feature region that meets a predetermined condition.

  FIG. 3 shows an example of a block configuration of the compression unit 230. The compression unit 230 includes an image dividing unit 232, a plurality of fixed value conversion units 234a-c (hereinafter may be collectively referred to as a fixed value conversion unit 234), and a plurality of compression processing units 236a-d (hereinafter referred to as compression processing). Part 236 in some cases).

  The image dividing unit 232 acquires a plurality of captured images from the image acquisition unit 250. Then, the image dividing unit 232 divides the plurality of captured images into a feature region and a background region other than the feature region. Specifically, the image dividing unit 232 divides the plurality of captured images into each of a plurality of feature areas and a background area other than the feature areas. As described above, the image dividing unit 232 divides each of the plurality of captured images into the feature region and the background region. Then, the compression processing unit 236 compresses the feature region image, which is the feature region image, and the background region image, which is the background region image, with different strengths. Specifically, the compression processing unit 236 compresses a feature area moving image including a plurality of feature area images and a background area moving image including a plurality of background area images with different strengths.

  Specifically, the image dividing unit 232 generates a feature area moving image for each of a plurality of feature types by dividing a plurality of captured images. Then, the fixed value unit 234 fixes the pixel values of the regions other than the feature regions of the respective feature types for each of the feature region images included in the plurality of feature region moving images generated for each feature type. To do. Specifically, the fixed value converting unit 234 sets pixel values in regions other than the feature region to predetermined pixel values. Then, the compression processing unit 236 compresses a plurality of feature area moving images for each feature type. For example, the compression processing unit 236 performs MPEG compression on a plurality of feature area moving images for each feature type.

  The fixed value unit 234a, the fixed value unit 234b, and the fixed value unit 234c are a feature region moving image of the first feature type, a feature region moving image of the second feature type, and a third feature type, respectively. The feature area video of is fixed. Then, the compression processing unit 236a, the compression processing unit 236b, and the compression processing unit 236c are the feature region moving image of the first feature type, the feature region moving image of the second feature type, and the feature of the third feature type. Compress area video.

  Note that the compression processing units 236a-c compress the feature area moving image with a predetermined strength according to the type of feature. For example, the compression processing unit 236 may convert the feature area moving image to a different resolution determined in advance according to the type of the feature, and compress the converted feature area moving image. In addition, when compressing the feature region moving image by MPEG encoding, the compression processing unit 236 may compress the feature region moving image with different quantization parameters determined in advance according to the feature type.

  The compression processing unit 236d compresses the background area moving image. Note that the compression processing unit 236d may compress the background region moving image with a strength higher than the strength of any of the compression processing units 236a-c. The feature area moving image and the background area moving image compressed by the compression processing unit 236 are supplied to the association processing unit 206.

  Since regions other than the feature region have been fixed values by the fixed value unit 234, when the compression processing unit 236 performs predictive encoding by MPEG encoding or the like, the region other than the feature region may be connected to the predicted image. The amount of difference between the images can be significantly reduced. Therefore, the compression rate of the feature area moving image can be significantly increased.

  FIG. 4 shows an example of a block configuration of the image processing apparatus 170. The image processing apparatus 170 includes a compressed image acquisition unit 301, an association analysis unit 302, an expansion control unit 310, an expansion unit 320, a synthesis unit 330, an output unit 340, a search instruction acquisition unit 350, a moving image configuration image search unit 360, and a moving image An output unit 370 is provided.

  The compressed image acquisition unit 301 acquires a compressed moving image including the captured image compressed by the compression unit 230. Specifically, the compressed image acquisition unit 301 acquires a compressed moving image including a plurality of feature area moving images and a background area moving image. More specifically, the compressed image acquisition unit 301 acquires a compressed moving image with feature area information attached thereto.

  Then, the association analysis unit 302 separates the plurality of feature area videos and background area videos and feature area information, and supplies the plurality of feature area videos and background area videos to the decompression unit 320. The association analysis unit 302 also analyzes the feature region information and supplies the feature region position and the feature type to the extension control unit 310. The decompression control unit 310 controls the decompression processing by the decompression unit 320 according to the position of the feature region and the feature type acquired from the association analysis unit 302. For example, the decompression control unit 310 decompresses each region of the moving image indicated by the compressed moving image to the decompressing unit 320 according to the compression method in which the compression unit 230 compresses each region of the moving image according to the position of the feature region and the type of feature. Let

  Hereinafter, the operation of each component included in the extension unit 320 will be described. The decompression unit 320 includes a plurality of decoders 322a-d (hereinafter collectively referred to as decoders 322). The decoder 322 decodes one of the plurality of encoded characteristic area moving images and background area moving images. Specifically, the decoder 322a, the decoder 322b, the decoder 322c, and the decoder 322d decode the first feature region moving image, the second feature region moving image, the third feature region moving image, and the background region moving image, respectively.

  The synthesizing unit 330 synthesizes a plurality of feature area moving images and background area moving images expanded by the expanding unit 320 to generate one display moving image. Specifically, the combining unit 330 generates one display moving image by combining the image of the feature region on the captured image included in the plurality of feature region moving images with the captured image included in the background region moving image. The output unit 340 outputs the feature area information and the display moving image acquired from the association analysis unit 302 to the display device 180 or the image DB 175. The image DB 175 associates the position of the feature region indicated by the feature region information, the feature type of the feature region, and the number of feature regions with the information for identifying the captured image included in the display moving image, and the non-volatile memory such as a hard disk. May be recorded on the recording medium.

  The search instruction acquisition unit 350 acquires an instruction to search for a captured image. For example, identification information for identifying a representative feature area that is a feature area to be displayed is transmitted from the monitor terminal to the image processing apparatus 170 by an operation of the monitor terminal by the monitor. The search instruction acquisition unit 350 acquires identification information for identifying the representative feature area transmitted from the supervisor terminal. In this way, the search instruction acquisition unit 350 supplies the moving image constituent image search unit 360 with conditions to which the representative feature region indicated by the search instruction from the supervisor should be matched.

  Then, the moving image constituent image search unit 360 searches for a plurality of captured images associated with the representative feature region indicating the representative feature region that matches the condition supplied from the search instruction acquisition unit 350. As described above, the moving image configuration image search unit 360 includes a plurality of captured images associated with representative feature region information indicating a representative feature region that satisfies a predetermined condition among the plurality of captured images output from the output unit 207. Search for.

  The moving image output unit 370 outputs a moving image including a plurality of captured images searched by the moving image constituent image search unit 360. Specifically, the moving image output unit 370 outputs a moving image including a plurality of captured images included in the plurality of moving images searched by the moving image configuration image search unit 360. Note that the moving image output unit 370 causes the display device 180 to display a moving image including the plurality of captured images.

  FIG. 5 shows an example of another block configuration of the compression unit 230. The compression unit 230 in this configuration compresses a plurality of captured images by a spatial scalable encoding process corresponding to the type of feature.

  The compression unit 230 in this configuration includes an image quality conversion unit 510, a difference processing unit 520, and an encoding unit 530. The difference processing unit 520 includes a plurality of inter-layer difference processing units 522a-d (hereinafter collectively referred to as inter-layer difference processing units 522). Encoding section 530 includes a plurality of encoders 532a-d (hereinafter collectively referred to as encoders 532).

  The image quality conversion unit 510 acquires a plurality of captured images from the image acquisition unit 250. In addition, the image quality conversion unit 510 acquires information specifying the feature region detected by the feature region detection unit 203 and information specifying the type of feature in the feature region. Then, the image quality conversion unit 510 duplicates the captured image, and generates captured images of the number of types of features in the feature area. Then, the image quality conversion unit 510 converts the generated captured image into an image having a resolution corresponding to the type of feature.

  For example, the image quality conversion unit 510 has a captured image converted to a resolution corresponding to the background area (hereinafter referred to as a low resolution image), and a captured image converted to the first resolution corresponding to the type of the first feature ( Called a first resolution image), a captured image converted to a second resolution corresponding to the second feature type (referred to as a second resolution image), and a third resolution corresponding to the third feature type. A captured image (referred to as a third resolution image) converted to is generated. Here, it is assumed that the first resolution image has a higher resolution than the low resolution image, the second resolution image has a higher resolution than the first resolution image, and the third resolution image has a higher resolution than the second resolution image.

  Then, the image quality conversion unit 510 converts the low resolution image, the first resolution image, the second resolution image, and the third resolution image into the inter-layer difference processing unit 522d, the inter-layer difference processing unit 522a, and the inter-layer difference processing unit 522b, respectively. And to the inter-layer difference processing unit 522c. The image quality conversion unit 510 supplies a moving image to each of the inter-layer difference processing units 522 by performing the above-described image quality conversion processing on each of the plurality of captured images.

  Note that the image quality conversion unit 510 may convert the frame rate of the moving image supplied to each of the inter-layer difference processing unit 522 in accordance with the feature type of the feature region. For example, the image quality conversion unit 510 may supply, to the inter-layer difference processing unit 522d, a moving image having a lower frame rate than the moving image supplied to the inter-layer difference processing unit 522a. In addition, the image quality conversion unit 510 may supply a moving image having a lower frame rate than the moving image supplied to the inter-layer difference processing unit 522b to the inter-layer difference processing unit 522a, and a frame lower than the moving image supplied to the inter-layer difference processing unit 522c. The rate movie may be supplied to the inter-tier difference processing unit 522b. Note that the image quality conversion unit 510 may convert the frame rate of the moving image supplied to the inter-layer difference processing unit 522 by thinning out the captured image according to the feature type of the feature region.

  The inter-layer difference processing unit 522d and the encoder 532d predictively encode a background area moving image including a plurality of low-resolution images. Specifically, the inter-layer difference processing unit 522 generates a difference image from a predicted image generated from another low-resolution image. Then, the encoder 532d quantizes the transform coefficient obtained by converting the difference image into a spatial frequency component, and encodes the quantized transform coefficient by entropy coding or the like. Note that such predictive encoding processing may be performed for each partial region of the low-resolution image.

  Further, the inter-layer difference processing unit 522a predictively encodes the first feature region moving image including the plurality of first resolution images supplied from the image quality conversion unit 510. Similarly, the inter-layer difference processing unit 522b and the inter-layer difference processing unit 522c each predictively encode a second feature area moving image including a plurality of second resolution images and a third feature area moving image including a plurality of third resolution images. To do. Hereinafter, specific operations of the inter-layer difference processing unit 522a and the encoder 532a will be described.

  The inter-layer difference processing unit 522a decodes the first resolution image encoded by the encoder 532d, and expands the decoded image to an image having the same resolution as the first resolution. Then, the inter-layer difference processing unit 522a generates a difference image between the enlarged image and the low resolution image. At this time, the inter-layer difference processing unit 522a sets the difference value in the background area to zero. Then, the encoder 532a encodes the difference image in the same manner as the encoder 532d. Note that the encoding processing by the inter-layer difference processing unit 522a and the encoder 532a may be performed for each partial region of the first resolution image.

  Note that, when the first resolution image is encoded, the inter-layer difference processing unit 522a calculates the code amount predicted when the difference image with the low resolution image is encoded, and the other first resolution image. The amount of code predicted when the difference image between the generated prediction image and the prediction image is encoded is compared. In the case where the latter code amount is smaller, the inter-layer difference processing unit 522a generates a difference image from the predicted image generated from the other first resolution image. In addition, the inter-layer difference processing unit 522a, when it is predicted that the code amount is smaller when encoded without taking the difference from the low-resolution image or the predicted image, It is not necessary to take a difference between them.

  Note that the inter-layer difference processing unit 522a does not have to set the difference value in the background area to zero. In this case, the encoder 532a may set the encoded data for difference information in an area other than the feature area to zero. For example, the encoder 532a may set the conversion coefficient after conversion to a frequency component to zero. Note that the motion vector information when the inter-layer difference processing unit 522d performs predictive encoding is supplied to the inter-layer difference processing unit 522a. The inter-layer difference processing unit 522a may calculate a motion vector for a predicted image using the motion vector information supplied from the inter-layer difference processing unit 522d.

  Note that the operations of the inter-layer difference processing unit 522b and the encoder 532b are that the second resolution image is encoded, and the first resolution after encoding by the encoder 532a when the second resolution image is encoded. The operations of the inter-layer difference processing unit 522b and the encoder 532b are substantially the same as the operations of the inter-layer difference processing unit 522a and the encoder 532a, except that the difference from the image may be taken. To do. Similarly, the operations of the inter-layer difference processing unit 522c and the encoder 532c are that the third resolution image is encoded, and when the third resolution image is encoded, after the encoding by the encoder 532b. Except for the fact that a difference from the second resolution image may be obtained, the operation is substantially the same as the operation of the inter-layer difference processing unit 522a and the encoder 532a, and thus the description thereof is omitted.

  As described above, the image quality conversion unit 510 generates, from each of the plurality of captured images, a low image quality image with a low image quality and a feature region image with higher image quality than the low image quality at least in the feature region. Then, the difference processing unit 520 generates a feature region difference image indicating a difference image between the feature region image in the feature region image and the feature region image in the low-quality image. Then, the encoding unit 530 encodes the feature region difference image and the low quality image.

  In addition, the image quality conversion unit 510 generates a low-quality image with reduced resolution from a plurality of captured images, and the difference processing unit 520 includes a feature region image in the feature region image and a feature region image in the low-quality image. A feature region difference image between the image and the image enlarged is generated. Further, the difference processing unit 520 has a spatial frequency component in which the difference between the feature region image and the enlarged image in the feature region is converted into the spatial frequency region, and the data amount of the spatial frequency component is in the region other than the feature region. A reduced feature area difference image is generated.

  As described above, the compression unit 230 performs hierarchical encoding by encoding image differences between a plurality of layers having different resolutions. As is clear from this, a part of the compression method by the compression unit 230 of this configuration is H.264. It is clear that a compression scheme according to H.264 / SVC is included. Note that when the image processing apparatus 170 decompresses such a layered compressed moving image, the moving image data of each layer is decoded, and the difference is taken for the region encoded by the inter-layer difference. The captured image having the original resolution can be generated by the addition process with the captured image decoded in the hierarchy.

  FIG. 6 shows an example of moving images obtained from different imaging devices 100. Note that the moving image A is a moving image obtained by the imaging device 100a, and the moving image B is a moving image obtained by the imaging device 100b.

  The moving image A includes a plurality of captured images 601a-f as moving image constituent images. The feature region detection unit 203 detects feature regions 611a-c from the captured images 601a-c, respectively. Note that the moving image B includes a plurality of captured images 602a-f as moving image constituent images. The feature region detection unit 203 detects feature regions 612a-f from the captured images 602a-f, respectively.

  The feature region information storage unit 220 stores the images of the feature regions 611a-c and the images of the feature regions 612a-f for a predetermined time length. Then, the correlation calculation unit 260 calculates the correlation value of the feature region stored in the feature region information storage unit 220. The correlation calculation unit 260 includes the positions where the imaging device 100a and the imaging device 100b are provided, the imaging directions of the imaging device 100a and the imaging device 100b, the position of the feature region 611 in the captured image 601, and the captured image 601. The correlation value of the feature region is calculated based on the position of the feature region 612 in FIG.

  For example, the correlation calculation unit 260 may calculate a correlation value between feature regions including a position on a captured image where a subject image where the same subject is captured should be higher than a predetermined value. For example, the picked-up image 601a and the picked-up image 602a are picked up at substantially the same timing, and the region in the real space picked up in the feature region 611a and the region in the real space picked up in the feature region 612a are substantially the same. In the case of the same region, a correlation value higher than a predetermined value may be calculated as the correlation value between the feature region 611a and the feature region 612a. The same applies to the feature region 611b and the feature region 612b, and the feature region 611c and the feature region 612c.

  In addition, the correlation calculation unit 260 has an object shape or color matching degree larger than a predetermined value for the feature regions 611a-c. For this reason, the correlation calculation unit 260 calculates a correlation value higher than a predetermined value as the correlation value between the feature regions 611a-c. In addition, for the feature regions 612a-f, the correlation calculation unit 260 selects one of the size matching degree of the feature region 612, the matching degree of the motion direction of the feature region 612, and the matching speed of the motion speed of the feature region 612. Greater than a predetermined value. Therefore, the correlation calculation unit 260 calculates a correlation value higher than a predetermined value as the correlation value of the feature region 612a-f.

  As described above, the correlation calculation unit 260 calculates a correlation value higher than a predetermined value between the feature regions 611a-c and the feature regions 612a-f. Then, the representative feature region determination unit 270 determines, as the representative feature region, the feature region 611b in which the entire face is imaged most greatly from the front among the feature regions 611a-c and 612a-f. Then, the association processing unit 206 identifies information (for example, a frame ID described later) that identifies the captured image 601b in which the feature region 611b is detected, and information that identifies a moving image including the captured image 601b (for example, a moving image ID described later). ), And representative feature region information including information (for example, a feature region ID described later) for identifying the feature region 611b from other feature regions (not shown) detected from the captured image 601b, and feature regions 611a-c and Attached to the feature regions 612a-f.

  The association processing unit 206 satisfies a predetermined condition from among captured images captured at substantially the same timing among captured images in which feature regions having a correlation higher than a predetermined value are detected. One suitable captured image may be selected as a captured image to which the representative feature region information is to be attached. Then, the association processing unit 206 may attach the representative feature region information to the selected captured image. For example, the association processing unit 206 attaches the feature region information to the captured images 601a-c in which the feature regions 611a-c captured from the front are detected, and the feature region information 612a-c includes the feature region information. There is no need to attach it.

  FIG. 7 shows a state in which feature area information is associated with a captured image by the association processing unit 206 in a table format. The association processing unit 206 associates information indicating the position of the feature region detected from the captured image with the moving image ID and the frame ID that are information for identifying the captured image. In addition, the moving image whose moving image ID is A shown in this drawing is the moving image A described with reference to FIG. 6, and the moving image whose moving image ID is B is the moving image B described with reference to FIG. To do.

  In addition, it is assumed that the captured images included in the moving image A and the moving image B are given frame IDs in the order in which the captured images are captured. Assume that # 701, # 702, and # 703 are respectively given as frame IDs to the captured image 601a, the captured image 601b, and the captured image 601c described with reference to FIG. Assume that # 704, # 705, and # 706 are given as frame IDs to the captured image 601d, the captured image 601e, and the captured image 601f described with reference to FIG.

  Note that the information indicating the position of the feature region may be coordinate information of the feature region. For example, when the feature area is a rectangle, the information indicating the position of the feature area includes the coordinates of the vertices located on the diagonal of the rectangle (for example, (x21, y21) and (x22, y22 in this figure)). ). In addition, when a plurality of feature areas are detected from one captured image, the association processing unit 206 identifies information (for example, # 1 in the drawing) that identifies the feature areas from other feature areas. It may be associated with the position of the feature region.

  The association processing unit 206 further associates a representative ID, which is an example of representative feature area information, with the moving image ID and the frame ID. For example, the representative ID includes, as an example, a frame ID (# 702) of a captured image in which the feature region 611b, which is a representative feature region, is detected, a moving image ID (A) including the captured image, and a feature region ID of the feature region 611b ( It may be a character string generated by formatting # 1) in a predetermined format.

  The captured image in which the feature area determined as the representative feature area is detected can be identified from the position where the representative ID and the feature area are detected, and further, the area in the captured image in which the feature area is detected is identified. can do. The representative feature area information is not limited to the representative ID described as an example of the representative feature area information in the present embodiment. The representative feature area information includes a captured image in which the feature area determined as the representative feature area is detected in any moving image, and the feature area determined as the representative feature area exists in any area of the captured image. Any information can be used as long as it can be specified.

  FIG. 8 shows an example of a search instruction reception screen 800 displayed on the display device 180. The image processing device 170 causes the display device 180 to display a search instruction reception screen 800 including representative images 801a-d (hereinafter collectively referred to as representative images 801). The representative images 801a-d may be images of the feature areas in the captured image in which the feature areas determined as the representative feature areas are detected.

  The monitor terminal acquires information for identifying the representative image 801 to be searched among the representative images 801 on the search instruction reception screen 800 by an operation of designating the search target by the monitor. The supervisor terminal transmits representative feature region information for identifying the representative image 801 identified by the information to the image processing apparatus 170. The search instruction acquisition unit 350 acquires the representative feature area information and supplies it to the moving image constituent image search unit 360.

  The moving image constituent image search unit 360 selects a captured image recorded in the image DB 175 with the representative feature region information attached thereto. For example, when the representative image 801a is designated, the moving image constituent image search unit 360 selects the captured images 601a-c and the captured images 602d-f.

  FIG. 9 shows a moving image of the representative feature area displayed by the display device 180. The moving image output unit 370 converts the captured images 601a-c and the captured images 602d-f selected by the moving image constituent image search unit 360 into a one-stream moving image as shown in the drawing and supplies the converted image to the display device 180. The display device 180 displays the moving image supplied from the moving image output unit 370.

  As described with reference to FIGS. 6 to 9, according to the image processing system 10, captured images including objects with high correlation among a plurality of moving images can be continuously displayed on the display device 180. As a result, images of the same person imaged by different imaging devices 100 can be continuously displayed on the display device 180.

  Further, it is also possible to select and display a captured image obtained by capturing the same person from among a plurality of captured images included in one moving image captured by a specific image capturing apparatus 100. In addition, a smaller number of captured images may be selected from a plurality of captured images having a correlation higher than a predetermined value in one moving image and provided to the display device 180. Thus, according to the image processing system 10, it is possible to provide a video in which redundant scenes are grouped.

  In the above description, an example of the operation of each component has been described by taking as an example the case where the representative feature area information is transmitted from the supervisor terminal. However, the supervisor terminal is detected from the captured image to be searched. The condition that the image of the feature area should be matched may be transmitted to the image processing apparatus 170. Then, the moving image constituent image search unit 360 selects a representative feature region that meets the condition, and displays the captured image recorded in the image DB 175 in association with the representative ID that identifies the selected representative feature region. You may search as a captured image.

  FIG. 10 shows an example of calculation parameters for calculating the correlation. The correlation calculation unit 260 calculates the correlation of the shape of the object, the correlation of the appearance of the object, and the correlation of the movement of the object. In the drawing, an example of a shape parameter and a motion parameter calculated by the correlation calculation unit 260 to obtain a correlation value is added to the object 1000 and shown.

  Examples of the shape parameter include the position of the object, the size of the object, the aspect ratio of the object, and the inclination of the object. The correlation between the parameters of each shape and the objects will be described below.

  The correlation calculation unit 260 may calculate barycentric coordinates on a common coordinate axis set on each captured image as an index of the object position. As illustrated in the drawing, the correlation calculation unit 260 calculates the coordinates of the center of gravity 1010 of the object 1000. The correlation calculation unit 260 calculates the distance (Euclidean distance) between the barycentric coordinates of the two objects as the correlation value of the positions between the two objects included in the two feature regions extracted from the two captured images. It's okay. Note that the correlation calculation unit 260 may calculate not the distance between the center-of-gravity coordinates of the object but the distance between the center-of-gravity coordinates of the feature region as the position correlation value.

  The correlation calculation unit 260 may calculate the area of the object as an index of the object size. The area may be expressed using the number of pixels. For example, the correlation calculation unit 260 calculates the number of pixels included in the object 1000. The correlation calculation unit 260 may calculate the correlation value of the size between the two objects according to the difference in the number of pixels in the two objects. Note that the correlation calculation unit 260 may calculate the correlation value of the size according to the difference in the number of pixels in the feature area instead of the difference in the number of pixels in the object.

  The correlation calculation unit 260 may calculate the aspect ratio of the object based on the moment of the image of the object. For example, the correlation calculation unit 260 may calculate a second moment of the image of the object and calculate the tilt angle of the main axis 1011 from the calculated second moment. The correlation calculation unit 260 may calculate the object aspect ratio from the ratio between the calculated object width 1012a in the main axis direction and the object width 1012b in the direction perpendicular to the main axis.

  The correlation calculation unit 260 may calculate the aspect ratio as a correlation value of the aspect ratio between two objects. Note that the correlation calculation unit 260 may calculate not the aspect ratio of the object but the aspect ratio of the feature area as the correlation value of the aspect ratio. The correlation calculation unit 260 may calculate the aspect ratio of the object from the aspect ratio of a rectangle that approximates the object, the aspect ratio of an ellipse that approximates the object, and the like.

  The correlation calculation unit 260 may calculate the inclination of the main axis 1011 of the object as an index of the inclination of the object. The inclination angle of the main axis 1011 can be calculated based on the moment of the image of the object as described above. Then, the correlation calculation unit 260 may calculate the difference between the inclination angles of the main axes as the correlation value of the inclination between the two objects. For example, the correlation calculation unit 260 may calculate cos θ as the correlation value when the difference in tilt angle is θ.

  When calculating the correlation value from the appearance parameters, the object parameters and the object color can be used as indices as the appearance parameters. For example, when the object is a person, the face pattern and clothes color can be used as indices as the appearance parameters.

  For example, the correlation calculation unit 260 may calculate a pattern overlay difference value as a pattern correlation value between two objects. Further, the correlation calculation unit 260 may calculate a normalized correlation value as a correlation value. Further, when color is used as an index, the correlation calculation unit 260 may calculate the degree of coincidence of the color histograms in the object as a correlation value. For example, the correlation calculation unit 260 can calculate a Bacharia coefficient of the color histogram as a correlation value.

  Further, when the correlation value is calculated from the motion parameter, examples of the motion parameter include the moving direction of the object and the direction of the object. For example, the correlation calculation unit 260 can calculate the moving direction of the center of gravity of each object in the moving image as the moving direction of the object. Then, the correlation calculation unit 260 may calculate the correlation value of the pattern between the two objects from the angle formed by the movement directions of the two objects. For example, the correlation calculation unit 260 may calculate cos θ as the correlation value when the angle is θ.

  In addition, when the surface that serves as a reference for the orientation of the object can be defined, such as when the object is a human face, the orientation of the object can be determined based on the orientation of the surface. For example, the correlation calculation unit 260 may determine the human face direction 1030 or the line-of-sight direction 1032 as the object direction. At this time, the correlation value can be calculated from the angle formed by the orientations of the two objects, as in the movement direction.

  As described above, according to the image processing system 10, the representative feature region information that can specify the image of the representative feature region is added to the plurality of captured images in which the feature regions having high correlation are detected. Then, according to the image processing system 10, it is possible to select a representative feature region that meets the specified condition from the image content of the representative feature region indicated by the representative feature region information. For this reason, according to the image processing system 10, it is only necessary to determine whether or not the image of the representative feature region should meet the condition, and whether or not the feature region detected from each captured image meets the condition. You don't have to make a judgment. Therefore, even when the imaging unit 102 is capturing at a high imaging rate and the captured moving image includes a large number of captured images, it is possible to quickly search for a partial moving image including a captured image that meets the specified condition. Can do.

  FIG. 11 shows an example of the hardware configuration of the image processing device 120 and the image processing device 170. The image processing device 120 and the image processing device 170 include a CPU peripheral unit, an input / output unit, and a legacy input / output unit. The CPU peripheral section includes a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 that are connected to each other by a host controller 1582. The input / output unit includes a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 that are connected to the host controller 1582 by the input / output controller 1584. The legacy input / output unit includes a ROM 1510, a flexible disk drive 1550, and an input / output chip 1570 connected to the input / output controller 1584.

  The host controller 1582 connects the RAM 1520, the CPU 1505 that accesses the RAM 1520 at a higher transfer rate, and the graphic controller 1575. The CPU 1505 operates according to the contents of the programs stored in the ROM 1510 and the RAM 1520 and controls each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505. The communication interface 1530 is connected to the network communication device 1598 to transmit / receive programs or data. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program that is executed when the image processing device 120 and the image processing device 170 are started, or a program that depends on the hardware of the image processing device 120 and the image processing device 170. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via the flexible disk drive 1550 or a parallel port, serial port, keyboard port, mouse port, and the like.

  A program executed by the CPU 1505 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program stored in the recording medium may be compressed or uncompressed. The program is installed in the hard disk drive 1540 from the recording medium, read into the RAM 1520, and executed by the CPU 1505. The program executed by the CPU 1505 causes the image processing apparatus 120 to function as each component included in the image processing apparatus 120 described with reference to FIGS. 1 to 10, and causes the image processing apparatus 170 to function as illustrated in FIGS. 1 to 10. It is made to function as each component which image processing apparatus 170 which was explained related has.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet is used as a recording medium, and is provided to the image processing device 120 and the image processing device 170 as a program via the network. Also good. As described above, the computer controlled by the program functions as the image processing device 120 and the image processing device 170.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10 Image processing system, 100 Imaging device, 102 Imaging part, 104 Imaging moving image compression part, 110 Communication network, 120 Image processing apparatus, 130 Person, 140 Moving body, 150 Monitoring object space, 160 space, 170 Image processing apparatus, 175 image DB, 180 display device, 201 compressed moving image acquisition unit, 202 compressed moving image decompression unit, 203 feature region detection unit, 206 association processing unit, 207 output unit, 210 compression control unit, 220 feature region information storage unit, 230 compression unit, 232 image division unit, 234 fixed value conversion unit, 236 compression processing unit, 250 image acquisition unit, 260 correlation calculation unit, 270 representative feature region determination unit, 301 compressed image acquisition unit, 302 association analysis unit, 310 expansion control unit, 320 Decompression Unit, 322 Decoder, 330 Combining Unit, 340 Output , 350 search instruction acquisition unit, 360 moving image constituent image search unit, 370 moving image output unit, 510 image quality conversion unit, 520 difference processing unit, 522 inter-layer difference processing unit, 530 encoding unit, 601 captured image, 602 captured image, 611 Feature area, 612 Feature area, 800 Search instruction acceptance screen, 801 Representative image, 1505 CPU, 1510 ROM, 1520 RAM, 1530 Communication interface, 1540 Hard disk drive, 1550 Flexible disk drive, 1560 CD-ROM drive, 1570 I / O Chip, 1575 graphic controller, 1580 display device, 1582 host controller, 1584 I / O controller, 1590 flexible disk, 1595 CD-ROM, 1598 Network communication equipment

Claims (25)

An image acquisition unit that acquires a plurality of moving images captured by each of a plurality of imaging devices including a first imaging device and a second imaging device ;
Included in each of the plurality of moving images, including a plurality of first moving image constituent images included in the moving image obtained by the first imaging device and a plurality of second moving image constituent images included in the moving image obtained by the second imaging device. A feature region detection unit that detects a plurality of feature regions from a plurality of video composition images,
Correlation between the plurality of feature regions including the first feature region detected by the feature region detection unit from the first moving image constituent image and the second feature region detected by the feature region detection unit from the second moving image constituent image. A correlation value calculation unit for calculating a value;
A representative feature region that is a feature region representing the plurality of feature regions is determined from the plurality of feature regions having a correlation higher than a predetermined value including the first feature region and the second feature region. A representative feature region determination unit;
The plurality of feature regions having a correlation higher than the predetermined value including the first feature region and the second feature region are detected in the representative feature region information which is information for specifying the representative feature region , An image processing system comprising: an output unit that outputs a plurality of moving image constituent images including the first moving image constituent image and the second moving image constituent image. The image processing system according to claim 1, wherein the representative feature region determination unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to the image content of the feature region. The image processing system according to claim 2, wherein the representative feature region determination unit determines the representative feature region from the plurality of feature regions in which the degree of coincidence of the image contents of the feature region is greater than a predetermined value. The representative feature region determining unit selects a feature region that satisfies a condition relating to a predetermined image content from the plurality of feature regions having a correlation higher than a predetermined value with respect to the image content of the feature region. The image processing system according to claim 2, wherein the image processing system is determined as a feature region. The representative feature region determination unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to the type of object included in the image of the feature region. Image processing system. The representative feature region determination unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a shape of an object included in an image of the feature region. Image processing system. The image according to claim 2, wherein the representative feature region determining unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a color included in the image of the feature region. Processing system. The image processing system according to claim 1, wherein the representative feature region determining unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a shape of the feature region. The image processing system according to claim 1, wherein the representative feature region determination unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a size of the feature region. The image processing system according to claim 1, wherein the representative feature region determining unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a position of the feature region. The image processing system according to claim 10, wherein the representative feature region determination unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a change in the position of the feature region. . The image processing according to claim 11, wherein the representative feature region determining unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a moving direction of the position of the feature region. system. The image processing according to claim 11, wherein the representative feature region determining unit determines the representative feature region from the plurality of feature regions having a correlation higher than a predetermined value with respect to a moving speed of the position of the feature region. system.   The first moving image constituting image in which the first feature region and the second feature region having a correlation higher than the predetermined value are imaged at substantially the same timing by the first imaging device and the second imaging device; When the feature area is detected from the second moving image constituent image, it conforms to a predetermined condition from the first moving image constituent image and the second moving image constituent image captured at substantially the same timing. Selecting means for selecting one moving image constituting image as the moving image constituting image to be associated with the representative feature area information
Further comprising
  The output unit outputs the representative feature region information in association with the one moving image constituent image selected by the selection unit.
The image processing system according to any one of claims 1 to 13.   An image acquisition unit that acquires a plurality of moving images captured by each of a plurality of imaging devices;
  A feature region detection unit for detecting a plurality of feature regions from a plurality of movie composition images included in each of the plurality of movies;
  A representative feature region determining unit that determines a representative feature region that is a feature region representing the plurality of feature regions from among the plurality of feature regions having a correlation higher than a predetermined value with respect to a moving speed of the position of the feature region. When,
  An output unit that outputs representative feature region information, which is information for specifying the representative feature region, in association with a plurality of moving image constituent images having the plurality of feature regions having a correlation higher than the predetermined value;
An image processing system comprising:   A moving image constituent image search unit that searches the plurality of moving image constituent images associated with the representative feature region information indicating a representative feature region that meets a predetermined condition among the plurality of moving image constituent images output from the output unit.
The image processing system according to claim 1, further comprising:   Movie output unit for outputting a movie including the plurality of movie component images searched by the movie component image search unit
The image processing system according to claim 16, further comprising:   A compression unit that compresses a moving image constituent image included in the moving image with different strength between the feature region and a background region other than the feature region.
Further comprising
  The output unit outputs the video compressed by the compression unit.
The image processing system according to claim 1.   The compression unit is
  An image dividing unit that divides a moving image constituent image included in the moving image into the feature region and a background region other than the feature region;
  A compression processing unit that compresses the feature region image that is the image of the feature region and the background region image that is the image of the background region with different intensities, respectively.
The image processing system according to claim 18.   The image dividing unit divides each of a plurality of moving image constituent images included in the moving image into the feature region and the background region,
  The compression processing unit compresses a feature area moving image including a plurality of the characteristic area images and a background area moving image including a plurality of the background area images with different strengths.
The image processing system according to claim 19.   The compression unit is
  From each of the moving image constituent images included in the moving image, an image generating unit that generates a low quality image with a low image quality, and a feature region image having a higher image quality than the low quality image at least in the feature region;
  A difference processing unit that generates a feature area difference image indicating a difference image between the image of the feature area in the feature area image and the image of the feature area in the low-quality image;
  An encoding unit for encoding each of the feature region difference image and the low-quality image;
The image processing system according to claim 18.   The image generation unit generates the low-quality image with reduced resolution from the moving image constituent image included in the moving image,
  The difference processing unit generates the feature region difference image between an image of the feature region in the feature region image and an image obtained by enlarging the image of the feature region in the low-quality image.
The image processing system according to claim 21.   The difference processing unit has a spatial frequency component in which a difference between the feature region image and the enlarged image is converted into a spatial frequency region in the feature region, and the spatial frequency component of the region other than the feature region Generating the feature region difference image with a reduced amount of data
The image processing system according to claim 22.   A program for causing a computer to function as the image processing system according to any one of claims 1 to 23.   An image acquisition stage for acquiring a plurality of moving images captured by each of a plurality of imaging devices including a first imaging device and a second imaging device;
  Included in each of the plurality of moving images, including a plurality of first moving image constituent images included in the moving image obtained by the first imaging device and a plurality of second moving image constituent images included in the moving image obtained by the second imaging device. A feature region detection stage for detecting a plurality of feature regions from a plurality of video composition images,
  Between the plurality of feature regions including the first feature region detected from the first moving image constituting image in the feature region detecting step and the second feature region detected from the second moving image constituting image in the feature region detecting step. A correlation value calculation stage for calculating a correlation value of
  A representative feature region that is a feature region representing the plurality of feature regions is determined from the plurality of feature regions having a correlation higher than a predetermined value including the first feature region and the second feature region. Representative feature region determination stage;
  The plurality of feature regions having a correlation higher than the predetermined value including the first feature region and the second feature region are detected in the representative feature region information which is information for specifying the representative feature region, An output stage for outputting a plurality of moving image composition images including the first moving image composition image and the second moving image composition image;
An image processing method comprising:

Images