JP2011055133A - Image processing system, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an reduced cost image processing system for correcting an optical response of an imaging optical system. <P>SOLUTION: This image processing system includes: a feature quantity extraction portion for respectively extracting feature quantities from images respectively captured at different positions on a plurality of captured images captured by an imaging apparatus; and an image processing portion for generating corrected images obtained by correcting differences of optical responses on an image region basis by an imaging optical system provided in the imaging apparatus from the captured images captured by the imaging apparatus or other captured images based on the feature quantities extracted by the feature quantity extraction portion. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing system, an image processing method, and a program.

There is known an imaging apparatus that corrects an image using correction data corresponding to a combination of a distance between a photographing optical system and a subject and a focal length (for example, see Patent Document 1). In this imaging device, the distance to the subject is detected, the focal length of the photographing optical system is detected, correction data corresponding to the detected distance and the focal length is read, and an object image is read using the read correction data. Correction processing is performed.
JP-A-9-74514

  According to the technique described in Patent Document 1, the degraded image can be restored by using correction data corresponding to the distance to the subject and the focal length. However, according to the technique described in Patent Document 1, a device that detects a distance to a subject and a focal length at the time of imaging must be mounted on the imaging apparatus, compared to an imaging apparatus that does not detect such information. Cost becomes high. In addition, various factors such as a focus position, a zoom position, a stop position, and a distance to a subject can be cited as factors causing image degradation. If an attempt is made to correct a deteriorated image with high accuracy in consideration of these various factors, the cost increases remarkably.

  In order to solve the above problems, in the first aspect of the present invention, in the image processing system, the feature amounts are respectively obtained from images captured at different positions on a plurality of captured images captured by the imaging device. Corrects the difference in optical response for each image area by the imaging optical system of the imaging device from the extracted feature quantity extraction unit and images of the same subject captured at different positions on the multiple captured images captured by the imaging device Based on the feature amount extracted by the correction data calculation unit for obtaining correction data and the feature amount extraction unit, the difference in optical response for each image region by the imaging optical system of the imaging device is determined based on the captured image captured by the imaging device. And an image processing unit that generates a corrected image that is corrected using the correction data and in which a difference in optical response for each image region by the imaging optical system of the imaging apparatus is corrected.

  The image processing unit may generate a corrected image based on the feature amount extracted by the feature amount extraction unit and the positions on a plurality of captured images where the same subject is captured.

  The image processing unit further includes a correction data storage unit that stores correction data in association with the feature amount extracted from images captured at different positions, and the image processing unit uses the feature amount extracted from the captured image captured by the imaging device. A corrected image may be generated using the correction data stored in the correction data storage unit in association with the matching feature amount.

  The correction data calculation unit calculates an optical response for each image area by an imaging optical system of a different imaging device from images of the same subject captured at different positions on a plurality of captured images captured by an imaging device different from the imaging device. The correction data for correcting the difference is calculated by processing that preferentially uses the image located at the center of the captured image of the same subject as the correct image, and the image processing unit captures the image captured by the imaging device. A corrected image may be generated using correction data stored in the correction data storage unit in association with a feature amount that matches the feature amount extracted from the image.

  The correction data calculation unit may calculate the correction data by processing using at least a plurality of captured images obtained by capturing images under different imaging conditions by different imaging devices.

  The correction data calculation unit may calculate the correction data by a process using at least a plurality of captured images obtained by imaging different subjects with different distances in the optical axis direction.

  The image processing unit may generate a corrected image using correction data for correcting the shape of the subject image.

  The feature amount extraction unit may extract feature amounts including edge information from images of the same subject captured at different positions on a plurality of captured images captured by the imaging device.

  The image processing unit may generate a corrected image using correction data for correcting blur of the subject image.

  The feature amount extraction unit may extract a feature amount including a spatial frequency component from images of the same subject captured at different positions on a plurality of captured images captured by the imaging device.

  The correction data may include a high frequency component to be added to the subject image, and the image processing unit may generate a corrected image by adding the high frequency component included in the correction data to the image.

  The correction data may include an image filter to be applied to the subject image, and the image processing unit may generate a corrected image by applying the image filter included in the correction data to the image.

  The image processing unit may generate a corrected image using correction data for correcting the color of the subject image.

  The feature amount extraction unit may extract a feature amount including color information from images of the same subject captured at different positions on a plurality of captured images captured by the imaging device.

  According to a second aspect of the present invention, there is provided an image processing method, wherein a feature amount extraction step of extracting a feature amount from each of images captured at different positions on a plurality of captured images captured by an imaging device, Correction data calculation for obtaining correction data for correcting a difference in optical response for each image area by the imaging optical system of the imaging device from images of the same subject captured at different positions on a plurality of captured images captured by the imaging device Based on the feature value extracted in the stage and the feature quantity extraction stage, the difference in optical response for each image area by the imaging optical system of the imaging device is corrected using the correction data from the captured image captured by the imaging device. And an image processing stage for generating a corrected image in which a difference in optical response for each image region by the imaging optical system of the imaging device is corrected.

  According to a third aspect of the present invention, there is provided a program for an image processing system, wherein a computer extracts a feature amount from images captured at different positions on a plurality of captured images captured by an imaging device. Correction data for correcting a difference in optical response for each image area by the imaging optical system of the imaging device from images of the same subject captured at different positions on a plurality of captured images captured by the imaging device; Based on the feature amount extracted by the correction data calculation unit and the feature amount extraction unit, the correction data is used to calculate the difference in optical response for each image area by the imaging optical system of the imaging device from the captured image captured by the imaging device. And an image processing unit that generates a corrected image in which a difference in optical response for each image region by the imaging optical system of the imaging apparatus is corrected.

  According to a fourth aspect of the present invention, there is provided an image processing system, wherein a first feature amount is extracted from images captured at different positions on a plurality of captured images captured by the first imaging device. The difference in the optical response for each image area by the imaging optical system of the first imaging device is corrected from the image of the same subject captured at different positions on the plurality of captured images captured by the extraction unit and the first imaging device. A correction data calculation unit for obtaining correction data to be corrected, a correction data storage unit for storing a correspondence relationship between the feature amount extracted by the first feature amount extraction unit and the correction data, and one on the captured image captured by the second imaging device A second feature amount extraction unit that extracts a feature amount from each image captured in the partial area, and correction data corresponding to the feature amount extracted by the second feature amount extraction unit is read from the correction data storage unit. From the captured image by the second imaging device imaged, and an image processing unit that the difference of the optical response of each image region of the imaging optical system in which the second imaging device has to generate a corrected image corrected.

  According to a fifth aspect of the present invention, there is provided an image processing method, wherein a first feature amount is extracted from images captured at different positions on a plurality of captured images captured by the first imaging device. The difference in optical response for each image area by the imaging optical system of the first imaging device is corrected from the extraction stage and images of the same subject captured at different positions on the plurality of captured images captured by the first imaging device. A correction data calculation stage for obtaining correction data to be corrected, a correction data storage stage for storing the correspondence between the feature quantity extracted in the first feature quantity extraction stage and the correction data in the correction data storage section, and the second image pickup device Corresponding to the feature quantity extracted in the second feature quantity extraction stage and the second feature quantity extraction stage respectively extracting the feature quantities from the images captured in the partial areas on the captured image. An image that reads correction data from the correction data storage unit and generates a corrected image in which a difference in optical response for each image area by the imaging optical system of the second imaging device is corrected from a captured image captured by the second imaging device. A processing stage.

  According to a sixth aspect of the present invention, there is provided a program for an image processing system, wherein a computer is used to extract feature amounts from images captured at different positions on a plurality of captured images captured by a first imaging device. The first feature amount extraction unit to extract, and the optical for each image region by the imaging optical system of the first imaging device from images of the same subject respectively captured at different positions on the plurality of captured images captured by the first imaging device A correction data calculation unit for obtaining correction data for correcting a difference in response, a correction data storage unit for storing the correspondence between the feature amount extracted by the first feature amount extraction unit and the correction data, and a captured image captured by the second imaging device A second feature amount extraction unit that extracts a feature amount from each of the images captured in the upper partial regions, and correction data corresponding to the feature amount extracted by the second feature amount extraction unit is corrected. The image processing unit functions as an image processing unit that generates a corrected image in which a difference in optical response for each image region by the imaging optical system of the second imaging device is corrected from a captured image captured by the second imaging device. .

  The above summary of the invention does not enumerate all necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of the overall configuration of an image processing system 10 according to an embodiment. As will be described below, the image processing system 10 can function as a video monitoring system. The image processing system 10 is not limited to a video surveillance system, and can function as a system that provides a service for correcting image degradation due to the influence of an imaging optical system on a captured image.

  The image processing system 10 includes a plurality of imaging devices 100a-d, an original image server 120, a communication network 110, an image processing device 170, an image database 175, a learning image database 176, and display devices 180a-d that capture the monitoring space 150. Prepare.

  In the following description, the imaging device 100a, the imaging device 100b, the imaging device 100c, and the imaging device 100d may be collectively referred to as the imaging device 100. Similarly, in the following description, a character indicated by a numerical code may be generically named by omitting a character following the numerical code, such as an English letter at the end.

  The imaging device 100a images the monitoring space 150. The imaging device 100a captures moving objects such as the person 130 and the vehicle 140 in the monitoring space 150 and generates a moving image. The imaging device 100 a supplies a moving image obtained by imaging the monitoring space 150 to the original image server 120. The imaging device 100b-d is provided at a different position from the imaging device 100a. In other respects, the function and operation of the imaging device 100b-d are substantially the same as the function and operation of the imaging device 100a. The description of -d is omitted.

  The original image server 120 sends the moving image supplied from the imaging devices 100 a-d to the communication network 110 toward the image processing device 170. An example of the communication network 110 is an electric communication line such as the Internet. The original image server 120 is provided in the vicinity of the imaging device 100, for example. In another form, the original image server 120 may be provided in the monitoring space 150.

  Further, the original image server 120 controls the imaging operation of the imaging device 100. For example, the original image server 120 controls on / off of an imaging function of the imaging apparatus 100, an imaging rate, and the like. When the image capturing apparatus 100 can capture an image by changing the image capturing condition, the original image server 120 may control the image capturing condition of the image capturing apparatus 100.

  For example, when the image capturing apparatus 100 can capture images while changing the zoom value, the original image server 120 may control the zoom value of the image capturing apparatus 100. When the image capturing apparatus 100 can capture an image by changing the focus position, the original image server 120 may control the focus position of the image capturing apparatus 100. In addition, when the image capturing apparatus 100 can capture images while changing the aperture value, the original image server 120 may control the aperture value of the image capturing apparatus 100. Further, when the imaging direction of the imaging device 100 is variable, the original image server 120 may control the imaging direction of the imaging device 100.

  As an example, the image processing apparatus 170 is provided in a space 165 remote from the monitoring space 150, and acquires the moving images captured by the imaging apparatus 100 from the original image server 120 through the communication network 110. The image processing apparatus 170 performs a correction process on the acquired moving image to generate a corrected moving image. The image processing apparatus 170 stores the corrected moving image in the image database 175. In response to a request from the display device 180, the image processing device 170 provides the corrected moving image stored in the image database 175 in the space 160 different from the monitoring space 150 and the space 165 through the communication network 110. It transmits to the display device 180.

  Note that the image processing apparatus 170 may transmit the corrected moving image to the display apparatus 180 without storing it in the image database 175. Further, the image processing apparatus 170 may store the moving image received from the original image server 120 in the image database 175 without performing image processing. When the corrected moving image is requested from the display device 180, the image processing device 170 performs image processing on the moving image stored in the image database 175 to generate a corrected moving image, and the generated corrected moving image is displayed on the display device. 180 may be transmitted.

  The display device 180 displays the corrected moving image acquired from the image processing device 170. The display device 180 may be provided in a space remote from the space in which the image processing device 170 is provided. The display device 180 may be provided in the vicinity of the monitoring space 150 or in the monitoring space 150.

  The outline of the correction process in the image processing apparatus 170 will be described below. The learning image database 176 stores a plurality of moving images captured by the camera 102. The image processing device 170 is an abbreviation in a frame image obtained under substantially the same imaging conditions (for example, substantially the same focus, zoom, aperture condition) of the same imaging device included in the moving image stored in the learning image database 176. Extracting subjects existing in the same image area, extracting a plurality of feature quantities from the extracted subject image group, obtaining a probability distribution in a feature quantity space expressed in multiple dimensions by learning, The image feature quantity is associated with the probability in the feature quantity space. Further, the image processing device 170 extracts the same subject existing in different image regions from the frame images obtained under substantially the same imaging conditions of the same imaging device, and extracts an object having a similar shape, and tracks the object in the image. From the extracted image of the same subject, the image of the same subject near the center position in the frame image is set as a correct image, and the camera 102 is obtained from the relationship with the image of the same subject in the peripheral portion in the frame. The image is corrected for each imaging condition and image area of the imaging optical system, and correction data for generating an image free from image deterioration such as blur, distortion, and color shift is calculated. The image processing apparatus 170 stores the calculated correction data in association with the probability in the feature amount space of the image feature amount for each previously extracted region.

  The image processing apparatus 170 extracts a feature amount for each image area from the frame image included in the moving image provided from the original image server 120. The image processing apparatus 170 acquires the stored correction data and the probability that the correction data will be applied from the position in the feature amount space represented by the extracted feature amount. A corrected moving image is generated by correcting each frame image based on the probability of application.

  As described above, the image processing apparatus 170 generates a corrected moving image based on the feature amount extracted from the image of the subject captured in different image areas. For this reason, the image processing apparatus 170 can generate a corrected image in which the influence of lens distortion or the like on the frame image is removed even when the optical response of the imaging optical system of the imaging apparatus 100 is different.

  Note that the storage medium 80 stores a program for the image processing apparatus 170. The program stored in the storage medium 80 is provided to an electronic information processing apparatus such as a computer that functions as the image processing apparatus 170 according to the present embodiment. The CPU included in the computer operates according to the contents of the program and controls each unit of the computer. The program executed by the CPU causes the computer to function as the image processing apparatus 170 described with reference to this figure and subsequent figures.

  As the storage medium 80, in addition to a CD-ROM, an optical recording medium such as DVD or PD, a magneto-optical recording medium such as MO or MD, a magnetic recording medium such as a tape medium or a hard disk device, a semiconductor memory, a magnetic memory, etc. It can be illustrated. In addition, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can function as the storage medium 80.

  The storage medium 80 may store a computer program that functions as another component of the image processing system 10. The program can cause the computer to function as at least one of the original image server 120, the display device 180, and the imaging device 100 described with reference to this drawing and the subsequent drawings.

  FIG. 2 shows an example of a block configuration of the image processing apparatus 170. The image processing apparatus 170 includes an original image acquisition unit 210, an identical subject region specification unit 212, a feature amount extraction unit 214a, a feature amount extraction unit 214b, a learning image acquisition unit 226, a correction data calculation unit 228, a correction data storage unit 230, an image A processing unit 218 and an output unit 290 are included.

  The learning image database 176 acquires and stores a plurality of captured images captured by one or more cameras 102 different from the imaging device 100. Specifically, the camera 102 captures a moving image captured under various imaging conditions. Then, the learning image database 176 obtains a plurality of captured images obtained by capturing one or more cameras 102 under a plurality of different imaging conditions. Since the moving image stored in the learning image database 176 is used for learning processing, it is referred to as a learning moving image in the following description, and is distinguished from the moving image captured by the imaging apparatus 100.

  The learning operation of the image processing apparatus 170 will be described below. The learning image acquisition unit 226 acquires a moving image from the learning image database 176.

  The same subject region specifying unit 212 specifies the same subject region, which is a region of the same subject picked up at a different position on the image region, from a plurality of frame images included in the moving image acquired by the learning image acquisition unit 226. For example, the same subject area specifying unit 212 specifies the same subject area by extracting objects having similar shapes. The same subject area specifying unit 212 may specify the same subject area using object tracking in an image using optical flow, mean shift, Kalman filter, or the like. Note that the same subject area specifying unit 212 specifies the same subject area from a plurality of frame images captured under the same imaging conditions.

  For example, the same subject region specifying unit 212 searches for similar image blocks between frames, calculates the velocity field of the image, and tracks the object of the same subject based on the velocity field, thereby identifying the same subject region. May be specified. Further, the same subject area specifying unit 212 calculates a speed field by calculating a spatio-temporal differentiation of the pixel value, and specifies the same subject area by tracking an object of the same subject based on the speed field. May be.

  In addition, the same subject area specifying unit 212 may specify the same subject area by tracking an object of the same subject by searching an image area having a histogram of similar pixel values between frames. Here, examples of the pixel value histogram include a color histogram and a luminance histogram. The color histogram may be a histogram of color components in the RGB color space, or may be a histogram of the HSV color space of hue, saturation, and brightness. The same subject area specifying unit 212 may track the object of the same subject by predicting the future position of the object based on the past position of the object determined as the same subject.

  Due to the influence of lens distortion and the like, even if the object is the same subject, the shape information and color information of the object extracted from the image may differ depending on the position in the image. According to the object tracking described above, the tracking can be performed while sequentially taking into account changes in shape information and color information according to the position in the image. In some cases, the subject area can be appropriately identified.

  The same subject area specifying unit 212 specifies the same subject area corresponding to each position in the frame. Note that the same subject area specifying unit 212 specifies a plurality of identical subject areas with similar angles and shape illumination conditions of the same subject being imaged.

  The feature amount extraction unit 214a extracts the feature amount from the images of the plurality of image regions after normalizing the shooting angle and the illumination condition of each identified subject. Examples of the feature amount include an edge component, a luminance value, a color component, a spatial frequency component, and the like calculated based on the values of the pixel of interest on the image and its surrounding pixels.

  The feature amount extraction unit 214a may extract the edge direction component and the edge strength component as the edge component feature amount. In this case, the feature amount extraction unit 214a may extract the edge direction component and the edge strength component for each color component to obtain the feature amount of the edge component. In addition, the feature amount extraction unit 214a may extract at least one of the average value of luminance values and the luminance distribution in a plurality of pixels as the feature value of the luminance value. In addition, the feature amount extraction unit 214a may extract at least one of an average value and a distribution for each color component in a plurality of pixels as a feature amount of the color component. The feature amount extraction unit 214a may extract a spatial frequency component for each predetermined spatial direction as a feature amount of the spatial frequency component in a plurality of pixels, or may extract a spatial frequency component for each color. In addition, the feature amount extraction unit 214a calculates a gradient vector or a feature amount difference between the partial regions from the feature amounts extracted from the adjacent partial regions, and extracts the gradient vector or the difference as a feature amount. May be.

  The correction data calculation unit 228 has an imaging optical system included in the camera 102 that captures images of a plurality of the same subject areas specified by the same subject area specification unit 212 based on the relationship between the images of the same subject areas. The correction data for correcting the difference in the optical response for each image area is calculated by using the image at the center of the frame as the correct image and integrating the relationship between the plurality of subjects. For example, the correction data calculation unit 228 calculates an image filter that can convert an image of the same subject area specified by the same subject area specification unit 212 into a corresponding correct image by averaging the relationship between a plurality of subjects. To do.

  The correction data storage unit 230 stores the correction data calculated by the correction data calculation unit 228 in association with the feature amount probability distribution obtained by the feature amount extraction unit 214a for each coordinate position of the image.

  Next, the correction operation of the image processing apparatus 170 will be described. The original image acquisition unit 210 acquires a moving image from the original image server 120. Note that the original image acquisition unit 210 may acquire a moving image from the image database 175 as described above.

  The feature amount extraction unit 214b normalizes the illumination conditions from the plurality of frame images included in the moving image acquired by the original image acquisition unit 210, and from the images of the plurality of regions captured at different positions on the image region. , Extract feature quantity. Examples of the feature amount include an edge component, a luminance value, a color component, a spatial frequency component, and the like calculated based on the values of the pixel of interest on the image and its surrounding pixels. Note that the feature quantity extraction unit 214b can extract a feature quantity similar to the above-described feature quantity extracted by the feature quantity extraction unit 214a. As described above, the feature amount extraction unit 214b extracts feature amounts from images of a plurality of regions respectively captured at different positions on a plurality of frame images captured by the imaging device 100.

  The image processing unit 218 reads the correction data stored in correspondence with the probability distribution of the feature amount obtained by the feature amount extraction unit 214 b from the correction data storage unit 230.

  The image processing unit 218 performs image processing on the moving image acquired by the original image acquisition unit 210 using the correction data read from the correction data storage unit 230. Accordingly, the image processing unit 218 can generate a corrected image in which a difference in optical response for each image region by the imaging optical system included in the imaging device 100 is corrected. As described above, the image processing unit 218 generates a corrected image using the correction data associated with the feature amount that matches the feature amount extracted by the feature amount extraction unit 214b.

  The image processing unit 218 supplies the corrected image to the output unit 290. The output unit 290 transmits the corrected image to the display device 180. Note that the output unit 290 may store the corrected image in the image database 175 as described above.

  As described above, the image processing apparatus 170 can generate a filter by learning processing from frame images captured by the camera 102 under various imaging conditions. The image processing apparatus 170 can generate a corrected image using a filter corresponding to the feature amount extracted from the image in the image area of the frame image captured by the imaging apparatus 100. Therefore, even when the optical response of the imaging optical system included in the imaging apparatus 100 is unknown, a corrected image in which the optical distortion of the imaging optical system of the imaging apparatus 100 is corrected can be generated.

  For this reason, the imaging apparatus 100 can correct an image with high accuracy by the image processing apparatus 170 even if it does not have a function of detecting a focus position, a zoom value, and the like. It is possible to obtain a high-quality image in which the above is corrected. In addition, since the imaging apparatus 100 can use an inexpensive imaging optical system with low optical accuracy, the cost can be further reduced.

  The feature quantity extraction unit 214a and the feature quantity extraction unit 214b can function as the first feature quantity extraction unit and the second feature quantity extraction unit in the present invention, respectively. When the functions of the feature quantity extraction unit 214a and the feature quantity extraction unit 214b are implemented in the same image processing apparatus 170, the feature quantity extraction unit 214a and the feature quantity extraction unit 214b may be implemented as the same functional block. In addition, the camera 102 and the imaging device 100 can function as the first imaging device and the second imaging device in the present invention, respectively. The correction data storage unit 230 may store the correction data calculated by the correction data calculation unit 228 in association with information for identifying the imaging device 100.

  FIG. 3 shows an example of the processing flow of the image processing apparatus 170. In this figure, a process in which the image processing apparatus 170 performs image processing on the moving image 310 captured by the image capturing apparatus 100a to generate a corrected moving image 350 will be described.

  Prior to image processing on the moving image 310, an image filter is generated in the image processing device 170. The learning image database 176 stores the learning moving image 300 captured by the camera 102. The same subject area specifying unit 212 has a degree of coincidence larger than a predetermined value among the same subjects at different positions in the frame from among the frame images included in the learning moving image 300 stored in the learning image database 176. .., 362b..., Which coincide with each other, are selected (S360). As a result, learning image sets 364a, 364b,... For generating image filters are extracted.

  The correction data calculation unit 228 generates an image filter 372 from each learning image set by learning processing using the learning image sets 364a, 364b,... (S370). The generated image filter 372 is stored in the correction data storage unit 230 in association with the feature amount.

  Next, image processing for the moving image 310 will be described. The original image acquisition unit 210 acquires a moving image 310 including a plurality of frame images 312. The feature amount extraction unit 214b extracts an image feature amount based on the image of the image area 314 of the moving image 310 (S320), and extracts a feature amount set 322.

  The image processing unit 218 determines one or more image filters to be used for generating an image filter for the moving image 310 based on the feature amount set 322 together with the probability that the image filter is applied. Then, the image processing unit 218 generates an image filter 340 based on the determined image filter and probability (S330).

  The image processing unit 218 generates the corrected moving image 350 by filtering the frame image included in the moving image 310 using the image filter 340 (S345). The corrected moving image 350 is output to the display device 180 or the image database 175 by the output unit 290.

  In the above description, two identical subject areas are specified for simplicity, but three or more identical subject areas may be specified. In order to detect the distortion of the imaging optical system, it is preferable to specify a larger number of the same subject area not only from the vicinity of the center on the image area but also from the peripheral area. As described above, the image processing unit 218 generates a corrected image based on the feature amount extracted by the feature amount extraction unit 214a and the positions on a plurality of captured images where the same subject is imaged. In some cases, an appropriate image filter 340 can be generated in consideration of the above.

  Examples of the image filter include a filter that calculates the pixel value of the target pixel by a weighted average of the pixel values of the pixels around the target pixel. In addition, the image filter may be a frequency filter that emphasizes high frequencies such as edge enhancement, or a geometric filter that performs affine transformation. As described above, the correction data may be an image filter to be applied to the subject image, and the image processing unit 218 generates a correction image by applying the image filter included in the correction data to the image. In addition to the image filter, the correction data in the present invention may be pixel data to be added to the correction target frame image or the spatial frequency component itself. Thus, the correction data may include a high frequency component to be added to the subject image. The image processing unit 218 may generate a corrected image by extracting and amplifying a high-frequency component included in the correction data, for example, with a high-frequency band pass filter and adding the amplified image to the image.

  Note that the feature amount extraction unit 214b may extract an appropriate type of feature amount according to the correction content. For example, when the image processing unit 218 generates a corrected image using correction data for correcting the shape of the subject image, the feature amount extraction unit 214b has different positions on a plurality of captured images captured by the imaging device. A feature amount including edge information may be extracted from an image of the same subject that has been captured. In addition, when the image processing unit 218 generates a corrected image using correction data for correcting the blur of the subject image, the feature amount extraction unit 214b has different positions on a plurality of captured images captured by the imaging device. A feature amount including a spatial frequency component may be extracted from an image of the same subject that has been picked up.

  In addition, the image processing unit 218 may generate a corrected image using correction data for correcting the color of the subject image. In this case, the feature amount extraction unit 214b performs processing on a plurality of captured images captured by the imaging device. A feature amount including color information is extracted from images of the same subject captured at different positions. The feature amount including color information may be a color histogram, a color spatial distribution, or the like.

  Further, the correction data storage unit 230 may store the image filter 340 calculated in S330 in association with information for identifying the imaging device 100a. The correction data storage unit 230 may perform a correction process on the new frame image obtained from the imaging device 100a using the image filter 340 stored in the correction data storage unit 230.

  The image filter may be an image filter that defines a process for replacing the pixel value of the target pixel with a value estimated and calculated in advance by pre-learning from the pixel values of pixels around the target pixel.

  Note that the correction data calculation unit 228 preferably calculates the correction data by learning using at least a plurality of captured images obtained by capturing at least one camera 102 under a plurality of different imaging conditions. In addition, it is desirable that the correction data calculation unit 228 calculates correction data by learning using at least a plurality of captured images obtained by imaging one or more cameras 102 with different distances in the optical axis direction.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

1 is a diagram illustrating an example of an overall configuration of an image processing system 10 according to an embodiment. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 170. FIG. It is a figure which shows an example of the processing flow of the image processing apparatus.

DESCRIPTION OF SYMBOLS 10 Image processing system 80 Storage medium 100 Imaging device 102 Camera 110 Communication network 120 Original image server 150 Monitoring space 160 Space 165 Space 170 Image processing device 175 Image database 176 Learning image database 180 Display device 210 Original image acquisition part 212 Same subject area specification Unit 214 feature amount extraction unit 218 image processing unit 226 learning image acquisition unit 228 correction data calculation unit 230 correction data storage unit 290 output unit 300 video for learning 310 video 350 corrected video 362 image 364 learning image set 372 image filter 312 frame Image 314 image region 322 feature set 340 image filter

Claims (19)

A feature amount extraction unit that extracts feature amounts from images captured at different positions on a plurality of captured images captured by the imaging device;
Correction for obtaining correction data for correcting a difference in optical response for each image area by the imaging optical system of the imaging device from images of the same subject respectively captured at different positions on a plurality of captured images captured by the imaging device A data calculator;
Based on the feature amount extracted by the feature amount extraction unit, the difference in optical response for each image region by the imaging optical system of the imaging device is corrected using the correction data from the captured image captured by the imaging device. An image processing system comprising: an image processing unit that generates a corrected image in which a difference in optical response for each image region by an imaging optical system included in the imaging device is corrected. The image processing according to claim 1, wherein the image processing unit generates the correction image based on the feature amount extracted by the feature amount extraction unit and the positions on the plurality of captured images where the same subject is captured. system. A correction data storage unit that stores the correction data in association with the feature amount extracted from the images captured at different positions;
The image processing unit uses the correction data stored in the correction data storage unit in association with a feature amount that matches a feature amount extracted from a captured image captured by the imaging device, and uses the correction data to store the correction image. The image processing system according to claim 2 to be generated. The correction data calculation unit is configured to calculate, for each image region by an imaging optical system included in the different imaging device, from images of the same subject captured at different positions on a plurality of captured images captured by an imaging device different from the imaging device. Correction data for correcting the difference in optical response is calculated by processing using the image located at the center of the captured image of the same subject with priority as the correct image,
The image processing unit uses the correction data stored in the correction data storage unit in association with a feature amount that matches a feature amount extracted from a captured image captured by the imaging device, and uses the correction data to store the correction image. The image processing system according to claim 3 to be generated. 5. The image processing system according to claim 4, wherein the correction data calculation unit calculates the correction data by a process using at least a plurality of captured images obtained by imaging the different imaging devices under a plurality of different imaging conditions. . 6. The correction data calculation unit according to claim 4 or 5, wherein the correction data calculation unit calculates the correction data by processing using at least a plurality of captured images obtained by imaging the different imaging devices with different distances in the optical axis direction. The image processing system described. The image processing system according to claim 2, wherein the image processing unit generates the corrected image using the correction data for correcting the shape of a subject image. The image processing according to claim 7, wherein the feature amount extraction unit extracts the feature amount including edge information from images of the same subject captured at different positions on the plurality of captured images captured by the imaging device. system. The image processing system according to claim 2, wherein the image processing unit generates the corrected image using the correction data for correcting blur of a subject image. The image according to claim 9, wherein the feature amount extraction unit extracts the feature amount including a spatial frequency component from images of the same subject captured at different positions on the plurality of captured images captured by the imaging device. Processing system. The correction data includes a high frequency component to be added to the subject image,
The image processing system according to claim 9, wherein the image processing unit generates the corrected image by adding a high-frequency component included in the correction data to the image. The correction data includes an image filter to be applied to the subject image,
The image processing system according to claim 9, wherein the image processing unit generates the corrected image by applying an image filter included in the correction data to the image. The image processing system according to claim 2, wherein the image processing unit generates the corrected image using the correction data for correcting the color of the subject image. The image processing according to claim 13, wherein the feature amount extraction unit extracts the feature amount including color information from images of the same subject captured at different positions on the plurality of captured images captured by the imaging device. system. A feature amount extraction stage for extracting feature amounts from images captured at different positions on a plurality of captured images captured by the imaging device;
Correction for obtaining correction data for correcting a difference in optical response for each image area by the imaging optical system of the imaging device from images of the same subject respectively captured at different positions on a plurality of captured images captured by the imaging device The data calculation stage,
Based on the feature amount extracted in the feature amount extraction step, the difference in optical response for each image region by the imaging optical system of the imaging device is corrected using the correction data from the captured image captured by the imaging device. And an image processing step of generating a corrected image in which a difference in optical response for each image region by the imaging optical system included in the imaging device is corrected. A program for an image processing system, comprising:
A feature amount extraction unit that extracts feature amounts from images captured at different positions on a plurality of captured images captured by the imaging device;
Correction for obtaining correction data for correcting a difference in optical response for each image area by the imaging optical system of the imaging device from images of the same subject respectively captured at different positions on a plurality of captured images captured by the imaging device Data calculator,
Based on the feature amount extracted by the feature amount extraction unit, the difference in optical response for each image region by the imaging optical system of the imaging device is corrected using the correction data from the captured image captured by the imaging device. A program that functions as an image processing unit that generates a corrected image in which a difference in optical response for each image region by the imaging optical system of the imaging device is corrected. A first feature amount extraction unit that extracts feature amounts from images captured at different positions on a plurality of captured images captured by the first imaging device;
Correction for correcting a difference in optical response for each image area by an imaging optical system of the first imaging device from images of the same subject captured at different positions on a plurality of captured images captured by the first imaging device. A correction data calculation unit for obtaining data;
A correction data storage unit that stores a correspondence relationship between the feature amount extracted by the first feature amount extraction unit and the correction data;
A second feature amount extraction unit that extracts a feature amount from each image captured in a partial region on the captured image captured by the second imaging device;
The correction data corresponding to the feature amount extracted by the second feature amount extraction unit is read from the correction data storage unit, and the image by the imaging optical system of the second imaging device is obtained from the captured image captured by the second imaging device. An image processing system comprising: an image processing unit that generates a corrected image in which a difference in optical response for each region is corrected. A first feature amount extraction stage for extracting feature amounts from images respectively captured at different positions on a plurality of captured images captured by the first imaging device;
Correction for correcting a difference in optical response for each image area by an imaging optical system of the first imaging device from images of the same subject captured at different positions on a plurality of captured images captured by the first imaging device. Correction data calculation stage for obtaining data,
A correction data storage step of storing a correspondence relationship between the feature amount extracted in the first feature amount extraction step and the correction data in a correction data storage unit;
A second feature amount extraction stage for extracting feature amounts from images respectively captured in partial areas on the captured image captured by the second imaging device;
An imaging optical system included in the second imaging device, which reads out correction data corresponding to the feature amount extracted in the second feature amount extraction step from the correction data storage unit and is captured by the second imaging device. And an image processing step of generating a corrected image in which a difference in optical response for each image region is corrected. A program for an image processing system, comprising:
A first feature amount extraction unit that extracts feature amounts from images captured at different positions on a plurality of captured images captured by the first imaging device;
Correction for correcting a difference in optical response for each image area by an imaging optical system of the first imaging device from images of the same subject captured at different positions on a plurality of captured images captured by the first imaging device. Correction data calculation unit for obtaining data,
A correction data storage unit that stores a correspondence relationship between the feature amount extracted by the first feature amount extraction unit and the correction data;
A second feature amount extraction unit that extracts feature amounts from images captured in partial areas on the captured image captured by the second imaging device;
The correction data corresponding to the feature amount extracted by the second feature amount extraction unit is read from the correction data storage unit, and the image by the imaging optical system of the second imaging device is obtained from the captured image captured by the second imaging device. A program that functions as an image processing unit that generates a corrected image in which a difference in optical response for each region is corrected.

Images