JP2017005699A - Image processing apparatus, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve such a problem that when there is no object reflecting commonly in a plurality of imaging devices, color matching accuracy of the images captured by the plurality of imaging devices may deteriorate.SOLUTION: An image processing device includes determination means for determining any one of first, second and third imaging device as a reference imaging device, from the images of the same object or the same kind of object captured by the first, second and third imaging device, and generation means for generating a second correction image where the color of the object image captured by the third imaging device is matched to the color of the object captured by the reference imaging device. Consequently, deterioration of color matching accuracy of the images captured by the plurality of imaging devices can be suppressed, even when there is no object reflecting commonly in the plurality of imaging devices.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for matching colors of a plurality of images taken by a plurality of imaging devices.

  2. Description of the Related Art Conventionally, a system that searches for image feature amounts of an object (for example, a person) shown in captured images acquired from a plurality of cameras (imaging devices) is known (for example, Patent Document 1). When searching for image features of an object from multiple captured images, colors may not match even though they are the same object due to ambient light, lighting, etc. , The search accuracy of the object was reduced.

  To perform color matching of images captured by multiple cameras, it is assumed that multiple cameras capture the same object, but if multiple cameras are installed remotely, The same object is not always reflected in all. That is, a plurality of cameras may not be able to capture the same object.

  Patent Document 1 discloses a method for performing color matching of images (such as moving image frames) captured by a plurality of cameras. Color matching is performed in order using images of the same object picked up by two adjacent cameras among a plurality of cameras in a predetermined order. When color matching is performed in that order, all the cameras are covered in a round. The order is determined as follows. In this way, color matching between a plurality of cameras is performed by repeating color matching between two adjacent cameras.

JP 2009-49759 A Japanese Patent No. 4715527

  In the method of Patent Literature 1, two adjacent imaging devices are color-matched, and the color-matching reference imaging device is shifted in a fixed order, so that all the imaging devices are compared with the first reference imaging device. Perform color matching. However, as the number of image pickup apparatuses that perform color matching increases, the color matching amount of the image accumulates, and the color matching accuracy may deteriorate.

  The present invention has been made in view of such problems. An object of the present invention is to provide an image processing device that suppresses deterioration in color matching accuracy of images captured by a plurality of imaging devices even when the plurality of imaging devices cannot capture the same object. Moreover, it aims at providing the method and program.

  An image processing apparatus for solving the above-described problems has the following configuration. That is, from the captured images of the same or the same type of object captured by each of the at least one first imaging device, the plurality of second imaging devices, and the at least one third imaging device, the image of the same or the same type of object. From the image of the same or the same type of object imaged by the input means for cutting out the area and inputting the image as the image of the same or the same type of object, and each of the first imaging device and the second imaging device, Identifying means for identifying an image of a first object, identifying an image of a second object from images of the same or the same type of object captured by each of the second captured image and the third image capturing device; Determination to determine any one of the first imaging device, the second imaging device, and the third imaging device as a reference imaging device And when any one of the first imaging device or the second imaging device is determined as the reference imaging device, the image captured by the reference imaging device is based on color information of the image of the first object. A first corrected image is generated by matching the color of the image of the same or same type of object with the color of the image of the same or same type of object captured by each of the first imaging device and the second imaging device. , Based on at least one of the color information of the image of the first object and the color information of the image of the second object, the color of the image of the same or the same type of object captured by the reference imaging device, Generating means for generating a second corrected image in which the colors of the images of the same or same type of object imaged by the third imaging device are combined. .

  Even when a plurality of imaging devices cannot capture the same object, it is possible to suppress deterioration in color matching accuracy of images captured by the plurality of imaging devices.

It is a block diagram which shows the hardware structural example of the image processing apparatus of 1st embodiment. It is a block diagram which shows the function structural example of the image processing apparatus of 1st embodiment. It is a figure which shows the example of the image feature-value used by 1st embodiment. (A) It is a flowchart explaining the color matching process in 1st embodiment. (B) It is a flowchart explaining a color correction parameter calculation process. (A) It is a figure which shows the example of the object table for color correction parameter generation in 1st embodiment. (B) It is a figure which shows which person each imaging device imaged in 1st embodiment. (C) It is a figure which shows an example of the state in which each imaging device color-matches with a reference | standard imaging device in 1st embodiment. It is a block diagram which shows the hardware structural example of the image processing apparatus of 2nd embodiment. It is a flowchart explaining the color correction parameter calculation process in 2nd embodiment. (A) It is a figure which shows the example of the object table for color correction parameter generation in 2nd embodiment. (B) It is a figure which shows which person each imaging device imaged in 2nd embodiment. (C) It is a figure which shows an example of the state in which each imaging device color-matches with a reference | standard imaging device in 2nd embodiment. (D) It is a figure which shows another example of the state in which each imaging device color-matches with a reference | standard imaging device in 2nd embodiment. It is a flowchart explaining the re-selection process of the reference | standard imaging device in 2nd embodiment. (A) It is a figure which shows an example of the state in which each imaging device color-matches with a reference | standard imaging device in 3rd embodiment. (B) It is a figure which shows the example of the frequency | count of color matching frequency, and the frequency | count of application of a color correction parameter.

<First Embodiment>
A configuration of a computer device constituting the image processing apparatus of the present embodiment will be described with reference to the block diagram of FIG. Note that the image processing apparatus may be realized by a single computer apparatus, or may be realized by distributing each function to a plurality of computer apparatuses as necessary. When configured by a plurality of computer devices, they are connected by a local area network (LAN) or the like so that they can communicate with each other. The computer device can be realized by an information processing device such as a personal computer (PC) or a workstation (WS).

  In FIG. 1, a CPU 101 is a central processing unit that controls the entire computer apparatus 100. The ROM 102 is a Read Only Memory that stores programs and parameters that do not need to be changed. A RAM 103 is a Random Access Memory that temporarily stores programs and data supplied from an external device. The external storage device 104 is a storage device such as a hard disk or a memory card that is fixedly installed in the computer device 100. The external storage device 104 may include an optical disk such as a flexible disk (FD) and a Compact Disk (CD) that can be detached from the computer apparatus 100, a magnetic or optical card, an IC card, a memory card, and the like. The input device interface 105 is an interface with an input device 109 such as a pointing device or a keyboard that receives data from a user and inputs data. The output device interface 106 is an interface with the monitor 110 for displaying data held by the computer apparatus 100 and supplied data. The communication interface 107 is a communication interface that is connected to a network line 111 such as the Internet and performs data input / output with the outside. The network camera 112 is an imaging device such as a plurality of surveillance cameras, and is connected to a computer device via a network line 111. The system bus 108 is a transmission path that connects the units so that they can communicate with each other.

  Each operation described later is executed by the CPU 101 executing a program stored in a computer-readable storage medium such as the ROM 102.

[Image processing device]
The image processing apparatus according to the present embodiment is an apparatus that cuts out an image area of an object from image data acquired from an imaging apparatus such as a plurality of cameras, accumulates the image area as an object image, and performs processing using the accumulated object image. . When the images are collated using the image processing apparatus of the present embodiment, the corrected image is matched with the color of the image captured by each imaging apparatus in accordance with the color of the image captured by the reference imaging apparatus (reference camera) serving as a reference. The image is verified after generation.

  Specifically, an image of an object (color correction parameter generation object) suitable for color matching is specified from the images of objects captured by a plurality of imaging devices. Using the color correction parameter generation object image, a color correction parameter that matches the color of the image captured by each image capturing device with the color of the image captured by the reference image capturing device (also referred to as a reference camera) is estimated. At the time of verification, verification is performed after generating a corrected image using the image captured by each imaging device and the color correction parameter.

  An object suitable for generating a color correction parameter (color correction parameter generation object) satisfies a condition such that the similarity of the image of the object is equal to or higher than a predetermined value. For this reason, even if the same object is actually shown in a plurality of imaging devices, it may not be adopted as an object suitable for generating a color correction parameter unless a predetermined condition is satisfied. By selecting a suitable object and performing color matching in this way, the probability of erroneous color matching can be reduced.

  Further, when a color correction parameter for matching a color with an image captured by the reference imaging device cannot be obtained, another imaging device that can match the color with the image captured by the reference imaging device is specified. A color correction parameter for matching a color with an image captured by the other image capturing apparatus is obtained. Then, color matching is realized by matching a color to an image captured by another imaging device and then matching a color to an image captured by the reference imaging device.

  In this embodiment, a person is assumed as an object. However, the object is not limited to a person, and any object suitable for color matching may be used.

  In the following description, as the color matching state of the imaging device, the terms reference imaging device, color-matched imaging device (color-matched camera), and unprocessed imaging device (unprocessed camera) are used. The “reference imaging device” is an imaging device serving as a color matching reference, and the color of an image captured by another imaging device is matched with the color of the image captured by the imaging device. There is only one reference imaging device among a plurality of imaging devices. The “color-matched imaging device” is an imaging device that can obtain color correction parameters in accordance with the reference imaging device. The reference imaging device is also considered as one of the color-matched imaging devices. An “unprocessed imaging device” is an imaging device that is in a state where color matching cannot be performed in accordance with the reference imaging device at that time.

Hereinafter, the functional configuration of the image processing apparatus of the present embodiment will be described with reference to FIG.
The input unit 201 receives input of an object image. Specifically, an image of an object cut out from the image acquired from the network camera 112 is input. Alternatively, the frame number in which the object appears in the video acquired from the network camera 112 and the coordinate information of the image area of the object are input.

  Further, the input unit 201 may accept an input of a captured image and cut out an object from the captured image may be performed by a cut-out unit (not shown) of the image processing apparatus. Alternatively, the image of the object may be cut out in advance in the network camera 112 and only the object image may be transmitted to the image processing apparatus.

  The storage unit 202 stores the image of the object input from the input unit 201. Specifically, the image data of the object is stored in the external storage device 104. In addition, the object ID, the imaging time, and the imaging camera are stored in association with each other as object metadata. Note that the video data obtained from the network camera 112 may be stored instead of the image data of the object, and the frame number where the object appears and the coordinate information of the image area of the object may be stored.

  The collation condition input unit 203 designates an object image used for collation. Specifically, the object image stored in the storage unit 202 is displayed on the monitor 110, and the object image to be collated is designated via the input device 109. Alternatively, a screen for specifying an imaging time, an image feature attribute value, or the like is displayed on the monitor 110, and the object images stored in the storage unit 202 are narrowed down and displayed on the monitor 110 under the conditions specified by the input device 109. You may make it do. Alternatively, the image data stored in the external storage device 104 may be designated as a comparison source, and the object image data stored in the storage unit 202 may be used as a comparison destination. The method for specifying the object image used for collation in the present embodiment is not limited to these.

  When designating the object image to be collated, the object image to be used for collation may be designated as the comparison source image and the comparison target image one by one. Alternatively, a plurality of comparison destination images may be selected for one comparison source image. Alternatively, only one comparison source image may be selected, and all the images of the objects stored in the storage unit 202 as the comparison destination images may be automatically selected. The method for specifying the object image used for collation in the present embodiment is not limited to these.

  The collation result display unit 204 displays a result of collating the images of the objects specified by the collation condition input unit 203. Although the details of the collation process will be described later, the degree of similarity representing the degree of similarity between the object images of the comparison source and the comparison destination is determined by the collation process. The matching result is displayed using this similarity. For example, if there is one image of the object of the comparison source and the comparison destination, the similarity between the images of the object is displayed on the monitor 110. Alternatively, if there is one comparison source object image and a plurality of comparison destination object images, the comparison destination object images are displayed on the monitor 110 in order of similarity. Alternatively, comparison target object images having a degree of similarity equal to or higher than a predetermined level may be classified for each camera and displayed in order of imaging time. In addition, the display method of the collation result of this embodiment is not limited to these.

  The collation unit 205 collates the comparison source object image with the comparison destination object image. Specifically, the collating unit 205 performs collation by comparing the image feature amounts of the comparison source object image and the comparison target object image. In the present embodiment, since a person is assumed as an object, the image feature quantity of a person as shown in FIG. 3 is used as the feature quantity of the object. Note that the image feature amounts shown in FIG. 3 are merely examples, and the present embodiment is not limited to these image feature amounts.

  The calculation method of these image feature amounts is as follows. SIFT (Scale Invariant Feature Transform) is used as an image feature quantity that does not change so much even if the color changes greatly, as a feature quantity that is less dependent on color. Since SIFT is obtained from a luminance image, the dependence on color is low. The SIFT feature value of the facial organ point in FIG. 3 is obtained by detecting the face region of a person, obtaining organ points such as eyes and mouth in the face region, and obtaining SIFT feature values corresponding to each organ point. . In the collation process using this feature amount, the collation unit 205 obtains the sum of the similarity degrees of SIFT feature amounts corresponding to each organ point in the face region in the images of the persons of the comparison source and the comparison destination. Alternatively, the collation unit 205 may determine the similarity using the number of organ points having a predetermined similarity or higher.

  The collation unit 205 obtains the skin color or the hair color by determining a corresponding partial area in advance from the face area obtained by face detection and obtaining a color histogram of the partial area.

  Further, in the collation unit 205, the color of clothes is obtained by determining in advance from the face area obtained by face detection that the area below corresponds to the body and obtaining a color histogram of the area. .

  In addition, the collation unit 205 creates a classifier that can estimate various attributes of a person, such as age and gender, when detecting a facial region image using a technique such as machine learning. Various attributes are obtained using a classifier. Further, when the likelihood of the attribute is obtained, an attribute vector in which the likelihood is arranged for each attribute value is created, and the similarity of the attribute is obtained by the similarity between the attribute vectors.

  Since similarities such as SIFT feature values, color histograms, and attribute vectors are obtained individually, the similarity between the image of the comparison source object and the image of the comparison destination object is obtained by weighting and summing.

  When the collation unit 205 searches for and specifies an object image used for color matching of the captured image of the image capturing apparatus, it is desirable to use an image feature amount that is less dependent on color. For this reason, in the present embodiment, image feature amounts extracted from the luminance image shown in FIG. 3 and person attributes are mainly used. Accordingly, even when the colors of the images of the same object captured by the plurality of imaging devices are different, the probability that the same object captured by the plurality of imaging devices can be determined to be the same is increased.

  On the other hand, after color matching is performed, it is desirable to perform collation using all image feature amounts. Therefore, in this embodiment, all image feature amounts shown in FIG. 3 are used. Thus, by using the image feature quantity depending on the color after color matching, it is possible to prevent deterioration of the collation accuracy of the object compared to the case where color matching is not performed.

  Note that the image feature amount used in the collation unit 205 may be calculated in advance for each object and stored in the storage unit 202. For example, the image feature amount before color matching is calculated and stored when an object image is input to the input unit 201. On the other hand, the feature quantity after color matching is stored in the storage unit 202 when the image feature quantity after color matching is calculated.

  The query list creation unit 206 creates a query list for specifying an image of an object used for color matching. Specifically, several object images are selected as queries from the object images stored in the storage unit 202, priorities are assigned, and a query list that is a query candidate is created. For example, priorities are assigned in the order of new accumulation time. Since the query is also described as an image of the comparison source object, the query list is also described as an image candidate list of the comparison source object.

  Alternatively, when a reference imaging device (reference camera) is determined or there is an imaging device (color-matched camera) for which a color correction parameter for the reference camera has already been obtained, a query may be selected as follows: .

  For example, an object shown in a camera (unprocessed camera) for which a color correction parameter is not obtained may be preferentially selected. As a result, the object captured by the unprocessed camera is always searched, and the color correction parameter generation object can be determined efficiently.

  Alternatively, the video in the unprocessed camera is analyzed, and when the moving direction of the object is directed to the color-matched camera, the object may be preferentially selected. For example, the camera arrangement relationship is stored in which direction of the camera frame is moved and another camera is present. Then, the movement vector of the person in the camera is analyzed to obtain the coordinates and direction when moving out of the frame from the camera, and the arrangement relation is compared with this to determine in which camera direction the person has moved. . At this time, when the person in the unprocessed camera is facing the direction of the color-matched camera, the image of this object is selected as a query. As a result, an image of an object that is likely to be reflected in an unprocessed camera and also in a color-matched camera can be selected as a query, so that a color correction parameter generation object can be identified efficiently.

  The specifying unit 207 determines an image of an object used for color matching between cameras. Specifically, using the query list created by the query list creation unit 206, an image of an object that can be determined to be the same as the query reflected by each camera is identified from the storage unit 202. Then, one image of an object (color correction parameter generation object) suitable for generating a color correction parameter is specified for each camera. For example, when the similarity with the query is greater than or equal to a predetermined value, it is determined that it is suitable for color correction, and is specified as an image of a color correction parameter generation object.

  Note that the method for specifying the image of the color correction parameter generating object in the present embodiment is not limited to these.

  The color correction parameter calculation unit 208 calculates a color correction parameter that is information used for color matching. Specifically, using the images of two objects, color correction parameters are obtained so that the colors of the object images match. When obtaining the color correction parameter, for example, in the case of a person, the color information of the skin area of the face area or the color information of the clothes area is used. Note that the area of the object used for calculating the color correction parameter is not limited to these.

  The corrected image generation unit 209 generates a corrected image in which the color is corrected using the color correction parameter.

  There are the following methods for calculating the color correction parameter and generating a corrected image using the parameter. For example, as shown in Patent Document 1, a plurality of corresponding points in an object may be obtained, and a look-up table may be configured based on the color difference between the corresponding points to be used as a color correction parameter. When generating a corrected image, the corrected image is generated by referring to the lookup table for each pixel of the input image data and replacing it with a different color.

  Alternatively, as shown in Patent Document 2, a color correction matrix may be obtained and used as a color correction parameter. When generating a corrected image, a color correction matrix is applied to each pixel of the input image data to replace it with a different color. Note that the color correction parameter calculation method in the present embodiment is not limited to these.

  In the following description, the color correction parameter may be applied multiple times. At this time, one color correction parameter obtained by combining the color correction parameters may be created in advance. For example, in the case of a lookup table, it can be combined by connecting and referring to the input and output of two lookup tables. In addition, color correction matrices can be combined by multiplying correction matrices in advance.

  In the following description, the degree of similarity between color correction parameters may be obtained, but if it is a lookup table, it can be obtained from the sum of difference values when the same pixel value is given. Further, the color correction matrix can be obtained from the difference value between the elements of the matrix.

  The combination of color correction parameters and the similarity calculation in the present embodiment are not limited to these.

  The reference imaging device determination unit 211 determines a reference imaging device (reference camera) that serves as a reference for color matching. For example, the reference camera may be determined according to designation from the outside. Specifically, a list of network cameras 112 is displayed on the monitor 110, and the network camera 112 selected via the input device 109 is set as a reference camera. For example, it may be possible to designate a network camera that monitors the entrance of a facility as a reference camera. Since the person reflected in the other cameras in the facility passes through the doorway, it is highly likely that the camera at the doorway reflects the same object as many other cameras in common. When such a camera having a high possibility of projecting the same object as that of another camera is selected as the reference camera, color correction parameters suitable for the reference camera can be easily obtained. As a result, the number of color corrections to be applied before matching with the reference camera is reduced, and the color correction error is reduced.

  Alternatively, when no reference camera is given as input, any one of the cameras is selected as the reference camera. Alternatively, the number of persons may be counted for each camera, and the camera in which the most people appear may be determined as the reference camera. The reference camera determination method in the present embodiment is not limited to these.

  The color correction parameter calculation 208 performs control for obtaining a color correction parameter until a predetermined number of cameras capable of color matching with the reference camera are reached.

  Specifically, control is performed so as to obtain a color correction parameter for performing color matching with respect to the reference camera. Alternatively, control is performed so as to obtain a color correction parameter for performing color matching for another camera that can be color-matched to the reference camera. This enables color matching to the reference camera via another camera.

  In the present embodiment, the color correction parameter is obtained so that all cameras can be matched with the reference camera as much as possible. The details of the processing will be described in the color correction parameter calculation processing described in the flowchart FIG.

  In this embodiment, all cameras are color-matched to the reference camera as much as possible. However, when obtaining the color correction parameter via another camera, if the number of times passing through another camera becomes a certain number or more, control may be performed so that the color correction parameter is not obtained between the cameras. This reduces the number of cameras that can perform color matching, but limits the accumulation of correction amounts due to repeated color correction, thereby reducing the color matching error.

[Flow of color matching process]
Next, the flow of color matching processing in the present embodiment will be described using the flowchart of FIG. Hereinafter, the flowchart is realized by the CPU of the image processing apparatus 100 executing a control program. In this process, first, an image of a color correction parameter generating object that is shared between a plurality of imaging devices is specified, and a color correction parameter between cameras is calculated. Then, a corrected image of the object image captured by each imaging device is generated, and an image feature amount of the corrected image of the object is calculated.

  It is assumed that the image feature amount before color matching is calculated in advance from the image of the object stored in the storage unit 202 when the color matching processing is executed. In addition, a list of target cameras for color matching processing is given to this processing. On this list, information on the color matching status of each camera, that is, the status of the reference camera, the color-matched camera, or the unprocessed camera is also given. Normally, all cameras are unprocessed cameras. However, when some of the cameras have already been color-matched and a new camera is added thereto, the reference camera and the color-matched camera are included. When there is a reference camera and a color-matched camera, the color correction parameter is additionally given.

Hereinafter, specific contents of the color matching process will be described with reference to FIG.
In step S401a, a query list is created. Specifically, the query list creation unit 206 selects several object images from the object images stored in the storage unit 202 as a query. Further, a query list is created by rearranging these queries in order of priority. For example, a predetermined number is selected in ascending order of imaging time, and a query list is created in the order of imaging time. Alternatively, when there is a camera that has already been color-matched, the query list may be created with priority given to the image of the object shown in the unprocessed camera. Furthermore, priority may be given to an object heading for a color-matched camera by using a movement vector of the object in the unprocessed camera.

  Step S402a is a loop for sequentially processing the queries in the query list obtained in step S401a, and it is assumed that numbers are sequentially assigned to the queries from 1. In order to refer to this using the variable i, first, i is initialized to 1. Further, when i is equal to or less than the number of queries, the process proceeds to step S403a, and when this is not satisfied, the process exits the loop and proceeds to step S407a.

  In step S403a, an image of an object similar to the i-th query is searched from the storage unit 202 using an image feature amount that does not change so much even if the color changes significantly as a feature amount that has a low dependency on color. To do. Specifically, the collation unit 205 uses the object image as the comparison source as the i-th query, and sets the object image as the comparison destination as the object image stored in the storage unit 202. And collation is performed using the feature-value with low dependence on a color. For example, the feature amount extracted from the luminance image shown in FIG. 3 is used for collation. Finally, the comparison object image and the similarity are associated with each other as a search result.

  In step S404a, it is determined whether or not an image of an object obtained from two or more cameras including an unprocessed camera is included in a search result having a similarity equal to or higher than a predetermined level. Specifically, focusing on only the image of an object having a similarity equal to or higher than a predetermined level, the camera in which the image of each object is captured is specified. When there are two or more specified cameras and an unprocessed camera is included (YES), the process proceeds to step S405a. Otherwise (NO), the process moves to step S406a.

  In step S405a, a color correction parameter generation object is determined and stored. Specifically, an image of an object having a similarity equal to or higher than a predetermined level is classified for each camera that has captured the image of the object. Then, one image of an object having a high similarity is determined for each camera. Images of these objects are stored as color correction parameter generation object images. Alternatively, when there are a plurality of images of objects having a degree of similarity equal to or higher than a predetermined value for a single camera that has captured the images, all of the images may be images of color correction parameter generation objects.

  Step S406a is the end of the query list loop, and 1 is added to i, and the process returns to S402a.

  The processing of step S402a to step S406a is performed by the specifying unit 207, and the image of the color correction parameter generating object is specified by this processing. For example, the results are collected as a color correction parameter generation object table as shown in FIG. This table represents “to which camera the image of the color correction parameter generation object was found” for each query. That is, as illustrated in FIG. 5B, for example, the person of the query Q1 is captured by each of the cameras A, B, C, and D, and the captured images are stored in the storage unit 202. Therefore, it indicates that an image of an object similar to the query Q1 has been found. Although not shown in this drawing, the object ID and the like of each color correction parameter generation object are also stored, and the object image data and the like can be referred to from the storage unit 202 by this object ID.

  In step S407a, the color correction parameter for each camera is calculated using the color correction parameter generation object obtained in steps S402a to S406a. Details of the calculation process will be described later with reference to the flowchart of FIG. As a result of this calculation processing, a color correction parameter group for matching colors with the reference camera is obtained. The graph of FIG. 5C shows that each camera matches the color of the reference camera. The node represents a camera, and the double line node represents a reference camera. For example, FIG. 5C shows that the camera A is determined as the reference camera, and parameters for color correction to the camera A are obtained for the cameras B to D. In addition, it is indicated that a parameter for color correction is required for the camera E to the camera B. Similarly, it is indicated that a parameter for color correction to the camera E is obtained for the cameras F to H.

  In step S408a, a corrected image is generated. Specifically, the corrected image generation unit 209 generates a corrected image by matching the image of the object captured by each camera with the color of the image of the object captured by the reference camera. For example, it is assumed that a color correction parameter group as shown in FIG. At this time, the color of the image of the object captured by the camera G is converted so as to match the color of the captured image of the camera E, and then converted to match the color of the captured image of the camera B. Finally, the image of the camera A is captured. Converted to match the color of the image. In this way, color correction is repeated to generate a corrected image of the captured image captured by each camera so as to match the color of the captured image of the reference camera.

  The corrected image is stored in the storage unit 202 and used in the collation unit 205 when collating the object image. Alternatively, attribute information used in the collation unit 205 is calculated in advance from the corrected image. For example, the image feature amount extracted from the color image shown in FIG. 3 is recalculated. Then, it may be stored in the storage unit 202.

[Color correction parameter calculation processing]
Next, the color correction parameter calculation process will be described with reference to the flowchart of FIG. This process is mainly executed by the color correction parameter calculation unit 208. In this process, a process for obtaining a color correction parameter is performed in order to generate a correction image that matches the color of the reference camera image between the cameras using the image of the color correction parameter generation object.

  When this processing is executed, a color correction parameter generation object table is provided. In addition, this process is given a list of cameras to be corrected. Information on the color matching state of the camera (whether it is in the state of the reference camera, the color-matched camera, or the unprocessed camera) is also given. In addition, when there is a reference camera and a color-matched camera, the color correction parameter is additionally given.

  Hereinafter, specific processing contents will be described with reference to FIG. In the following description, it is assumed that all the cameras are unprocessed cameras and that FIG. 5A is given as the color correction parameter generation object table.

  In step S401b, a reference camera is determined. Specifically, the reference imaging device determination unit 210 determines the reference camera. For example, if a reference camera is given from the input, it is used as the reference camera. When the reference camera is not given, one of the cameras is selected as the reference camera. Here, it is assumed that camera A is selected as the reference camera among cameras A to H shown in FIG.

  In step S402b, the color correction parameter calculation unit 208 lists unprocessed cameras. For example, all cameras B to H other than the reference camera in FIG. 5A are listed as unprocessed cameras.

  In step S403b, it is determined whether there is an unprocessed camera that projects the color-matched camera and the color correction parameter generation object in common. Specifically, when the determination result is YES, the process proceeds to step S404b. In other cases (NO), this process is terminated. For example, in FIG. 5A, when attention is paid to the image of the color correction parameter generation object similar to the query Q1, there are unprocessed cameras B to D that commonly display the reference camera A and the color correction parameter generation object. . Therefore, this determination is YES.

  In step S404b, the color correction parameter calculation unit 208 selects an unprocessed camera that displays the color-matched camera and the color correction parameter generation object in common, and obtains a color correction parameter.

  In step S403b, one of the unprocessed cameras B to D is selected. For example, assume that camera B is selected. Then, the color correction parameter calculation unit 208 uses the image of the camera B as the color of the image of the camera A by using the image of the camera A and the image of the camera B searched by the query Q1 as the image of the color correction parameter generation object. A color correction parameter for matching colors is obtained.

  Thereafter, the camera B is set as a color-matched camera and is not selected again in step S403b.

  By repeating the above processing, as shown in FIG. 5C, the image of the camera B to D is changed to the color of the image of the camera A by using the image of the color correction parameter generation object searched by the query Q1. A color correction parameter for matching colors is obtained. Similarly, a color correction parameter for matching the color of the image of the camera E with the color of the image of the camera B is obtained using the image of the color correction parameter generation object searched by the query Q2. Finally, using the image of the color correction parameter generation object searched in the query Q3, a color correction parameter that matches the colors of the images of the cameras F to H with the color of the image of the camera E is obtained.

  In the example of FIG. 5A, all cameras can be color-matched to the reference camera, but there are cameras that do not project the color correction parameter generation object in common with the reference camera or the color-matched camera. Sometimes the camera will remain as an unprocessed camera.

  At this time, the process of FIG. 4A may be terminated without performing the process of step S408a, and the color matching process may not be performed. Then, the video may be continuously acquired from the camera for a while to increase the number of images of the object in the storage unit 202, and then these processes may be performed again. As a result, color matching is performed only when all cameras can be color-matched.

  Alternatively, in the process of step S408a, only the object image obtained from the color-matched camera may be color-matched, and the object image obtained from the unprocessed camera may not be color-matched. At this time, when the image of the comparison source object and the image of the comparison destination object are input from the collation condition input unit 203 and collation is performed, one of the colors may not be matched. At this time, matching may be performed using a feature quantity independent of color. Alternatively, it is possible to ignore the fact that color matching has not been performed and to perform collation using a characteristic amount depending on the color.

  Note that the processing when an unprocessed camera remains in the present embodiment is not limited to these.

  As described above, the deterioration of the collation accuracy of the image of the object due to the difference in the color of the image of the different imaging devices that captured the same object is suppressed. For example, since the installation environment of a monitoring camera is different, the color of images captured by different monitoring cameras may be different even for the same object due to the influence of shade, sunlight, light source, and the like. Degradation of collation accuracy due to such influence is suppressed.

  In addition, even when there is a camera that does not display an object suitable for generating color correction parameters in common with the reference camera, color matching with another camera can be performed and color matching can be performed with the reference camera via the camera. . For example, in a camera installed in a wide area such as an airport, the same object is rarely captured by all cameras. Therefore, an object suitable for generating the color correction parameter may not be obtained between the cameras. In such a case, it is effective to perform color matching with the reference camera via another camera.

  Further, the corrected image generated using the color correction parameter may be used not only for collation but also for display. For example, the generated correction image may be displayed when the result is displayed on the matching result display unit 204. By doing so, there is no difference in the color of the display image of the same object, and the viewer can comfortably view the image.

Second Embodiment
In the first embodiment, the color correction parameter may be erroneously obtained. For example, in the first embodiment, when an image of a different object is used as an image of a color correction parameter generation object, the color correction parameter is erroneously obtained. In the first embodiment, it is determined that the images of the same object are obtained when the similarity of the collation results of the object images is equal to or greater than a predetermined value. However, the determination is not necessarily erroneous. Further, when color matching is performed using an image of a body region other than the face region used for collation, the image of the body region may have changed. For example, when the same person is imaged, it is assumed that the image of the person can be correctly specified by using the face attribute / feature amount for collation. On the other hand, if the color information of the person's torso area (clothing portion) is used to calculate the color correction parameter, an incorrect color correction parameter is obtained when the clothes are attached or detached.

  Therefore, in this embodiment, an image processing apparatus that creates a plurality of color correction parameter candidates between an unprocessed camera and a color-matched camera and determines a highly reliable color correction parameter will be described.

  The configuration of the image processing apparatus of the present embodiment is basically the same as the configuration of FIG. 2 shown in the description of the first embodiment. However, a color correction parameter determination unit 211 is added. Different parts from the first embodiment will be described below with reference to FIG.

  The color correction parameter determination unit 211 determines a highly reliable color correction parameter from a plurality of color correction parameters between the unprocessed camera and the color-matched camera. Specifically, when the color of a plurality of corrected images of the same camera is determined to be the same using a plurality of color correction parameters when the color is matched with the color of the image of the reference camera, the same color Count the number of color correction parameters that give the correction results. If the ratio of the counted color correction parameters is equal to or greater than a predetermined value, one of the counted color correction parameters is determined.

  When a plurality of identical objects are imaged, the ratio of color correction parameter groups that give the same color correction result for each object is obtained, the color correction parameter group that occupies the largest ratio is identified, and one of them is selected. decide. Thus, if there are more correct color correction parameters than incorrect color correction parameters, the correct color correction parameters can be selected.

  Detailed processing contents of the image processing apparatus 100 of this embodiment will be described later with reference to a flowchart of FIG.

  In this process, the unprocessed camera is changed to a color-matched camera in order. Therefore, the number of color correction parameter candidates between the unprocessed camera and the color-matched camera tends to gradually increase. Therefore, even if an unprocessed camera does not have a sufficient number of color correction parameters at the beginning of the process, sufficient color correction parameters can be obtained as the process proceeds. Therefore, the condition that “the number of color correction parameter candidates is greater than or equal to a predetermined value” has the effect of further improving the probability of selecting the correct color correction parameter.

  Whether or not the color correction parameters give the same color correction result is obtained by obtaining a result (corrected image) obtained by performing color correction on the image captured by the same camera with each color correction parameter, and comparing the results. Thus, it is determined whether or not the same result is given. For example, a difference image between two corrected images may be obtained, and the difference may be determined to be the same if the sum of the difference values is within a predetermined range. Alternatively, the color histograms or the like of two corrected images may be obtained, and it may be determined that the colors of the two corrected images are the same if the histogram similarity is greater than or equal to a predetermined value. The method for determining whether or not the colors of the corrected images are the same in this embodiment is not limited to these.

  In the present embodiment, a highly reliable color correction parameter is determined by selecting one of the color correction parameters that give the same color correction result. However, an average color correction parameter may be generated from a plurality of color correction parameters. Alternatively, a new color correction parameter may be calculated based on a plurality of color correction parameter generation objects that are the basis of the color correction parameter.

  For example, as shown in Patent Document 1, when a lookup table is configured, corresponding points are obtained from a plurality of color correction parameter generation objects, and color correction parameters are obtained based on the corresponding points. Alternatively, an average lookup table may be obtained from a plurality of lookup tables. For example, a lookup table is configured in which values obtained by averaging outputs when all input values are given to each lookup table are associated with the input values.

  Alternatively, as shown in Patent Document 2, when obtaining a color correction matrix, a plurality of colors before and after correction are obtained from a plurality of color correction parameter generation objects, and a color correction matrix satisfying these is obtained by a least square method or the like. Ask by method. Alternatively, an average color correction matrix may be obtained using a plurality of color correction matrices. For example, an average value is obtained for each element of the matrix, and a color correction matrix having this as an element is obtained.

  As described above, the highly reliable color correction parameter determination method in the present embodiment is not limited to the method of selecting any one of them. Further, the method for generating a new color correction parameter from a plurality of color correction parameters and a plurality of color correction parameter generation objects is not limited to these.

  The reference imaging device determination unit 210 determines a camera (reference camera) that is a reference for color matching. In the present embodiment, the reference camera is determined based on the number of the same objects (color correction parameter generation objects) that are shared by the cameras. Specifically, the number of different objects captured in common as the color correction parameter generation object between all two cameras is counted to obtain the common object number. Then, one of the cameras having the largest number of common objects is determined as a reference camera.

  For example, the number of common objects is counted as follows. As illustrated in FIG. 8B, the camera A and the camera B capture each person (object) of the queries Q1, Q2, and Q3, and the captured images are stored in the storage unit 202. As shown in the color correction parameter generation object table shown in FIG. 8A, the number of common objects between the camera A and the camera B is three objects corresponding to the queries Q1, Q2, and Q3, respectively. The number of common objects is obtained as “3”. Similarly, since the number of common objects between the camera B and the camera E is one object corresponding to the query Q4, the number of common objects is “1”. In this way, the number of common objects between all two cameras is obtained, a set of cameras showing the largest number of common objects is specified, and one of them is determined as a reference camera.

  The color correction parameter determination unit 211 can select a correct color correction parameter as the number of color correction parameters calculated by the color correction parameter calculation unit 208 increases. If the number of common objects is large, the color correction parameter calculation unit 208 can calculate a large number of color correction parameters. When the reference imaging device determination unit 210 selects a camera having a large number of common objects as a reference camera, an effect that a correct color correction parameter can be selected can be obtained.

  Note that the number of common objects may be obtained as the number of common objects between all other cameras, not between two cameras. For example, in the object table for color correction parameter generation shown in FIG. 8A, the number of common objects for camera B is the number of common objects because the objects corresponding to queries Q1 to Q4 are also captured by other cameras. “4”. Similarly, the camera E also captures the objects corresponding to the queries Q4 to Q7, so that the number of common objects is “4”. In this case, since it is not necessary to obtain a combination with another camera, the number of common objects can be obtained more easily. However, since the number of common objects is not strictly determined, a small number of common objects between two cameras, such as the camera E, may obtain a high number of common objects. Therefore, the number of common objects between the two cameras may be obtained only for the camera having the large number of common objects. This eliminates the need to determine the number of common objects for all camera combinations, and can be expected to reduce the amount of calculation.

  Alternatively, when the number of common objects between two cameras is obtained, if the number of common objects is the same, the one having the larger number of common objects with all other cameras may be selected. A camera having a large number of common objects with all other cameras indicates that there are many cameras that can be color-matched directly to that camera. Therefore, if such a camera is selected as the reference camera, it is expected that the number of times of applying color correction is reduced. Therefore, it can be expected that the color correction error is reduced.

  Although it is desirable to determine the reference camera as described above, the reference camera may be determined as described above, and the reference camera determination method in the present embodiment is not limited to these.

  The color matching process in the present embodiment is the same as the description of the first example using the flowchart FIG.

  Therefore, the color correction parameter calculation processing in this embodiment will be described with reference to FIG. This process is mainly executed by the color correction parameter calculation unit 208. In this process, a process for obtaining a color correction parameter is performed in order to generate a correction image that matches the color of the reference camera image between the cameras using the image of the color correction parameter generation object.

  When this processing is executed, a color correction parameter generation object table is provided. In addition, this process is given a list of cameras to be corrected. Information on the color matching state of the camera (whether it is in the state of the reference camera, the color-matched camera, or the unprocessed camera) is also given. In addition, when there is a reference camera and a color-matched camera, the color correction parameter is additionally given.

  Hereinafter, specific processing contents will be described with reference to FIG. In the following description, it is assumed that all the cameras are unprocessed cameras and FIG. 8A is given as the color correction parameter generation object table.

  In step S701, a reference camera is determined. Specifically, the reference imaging device determination unit 210 determines the reference camera. For example, if a reference camera is designated from the user input, it is used as the reference camera. When the reference camera is not designated, the camera having the largest number of common objects between the two cameras is determined as the reference camera, as described in the description of the reference imaging device determination unit 210. For example, in FIG. 8A, since the number of common objects between the camera A and the camera B is the maximum of “3”, it is assumed that the camera A which is one of the cameras is determined as the reference camera.

  In step S702, unprocessed cameras are listed, and the number of common objects between each unprocessed camera and the color-matched camera group is obtained. The unprocessed cameras listed are left unprocessed.

  For example, in FIG. 8A, unprocessed cameras B to H are listed. Since the color-matched camera is only the reference camera A, the number of common objects with the camera A is obtained. Specifically, the number of common objects is 3 for cameras B to D, and the number of common objects is 0 for cameras E to H.

  In step S703, it is determined whether there is an unprocessed unprocessed camera among the unprocessed cameras listed in step S702. If there is (YES), the process proceeds to step S704. If not (NO), the process is terminated.

  In step S704, an unprocessed camera having the largest number of common objects is selected from unprocessed unprocessed cameras. In the example of FIG. 8A, the cameras B to D whose number of common objects with the camera A is 3 are unprocessed cameras having the largest number of common objects. Therefore, one is selected from these. For example, assume that camera B is selected. The selected camera is stored as having undergone color matching processing.

  In step S705, the color correction parameter determination unit 212 counts color correction parameter candidates that give the same color correction result when the selected unprocessed camera is matched with the reference camera.

  In the example of FIG. 8A, three color correction parameters can be created between the unprocessed camera B and the reference camera A. Therefore, it is determined whether or not each color correction parameter gives the same color correction result. As described in the description of the color correction parameter determination unit 211, this determination method may be performed by generating and comparing corrected images, or by comparing color correction parameters. By grouping the color correction parameters determined to be the same, a group of parameters that give the same color correction result can be grouped.

  In step S706, the color correction parameter determination unit 211 determines whether the number of color correction parameter candidates is equal to or greater than a threshold and whether the ratio of color correction parameters that give the same color correction result is equal to or greater than the threshold. Specifically, the number of color correction parameter candidates obtained in step S705 and the ratio of parameter candidates that give the same color correction result are obtained.

  For example, there are three color correction parameters between camera A and camera B in FIG. When the number of parameters giving the same color correction result is two in step S705, the ratio is 2/3. When the threshold value for the number of candidates is 3 or more and the threshold value for the ratio is 60% or more in advance, these two conditions are satisfied, so that these two color correction parameters are determined to be correct.

  When the ratio equal to or greater than the threshold is occupied in step S706 (YES), the process proceeds to step S707. Otherwise (NO), the process moves to step S703.

  In step S707, the color correction parameter determination unit 211 determines a color correction parameter to be used when generating a corrected image. In step S706, the color correction parameter group that has occupied a ratio equal to or greater than the threshold value and that has occupied the maximum ratio is identified. One of them is selected and determined as a color correction parameter to be used when generating a corrected image. Alternatively, an average color correction parameter may be generated using a plurality of specified color correction parameter groups.

  In step S708, the camera selected in step S704 is set as a color-matched camera. In the example of FIG. 8A, the camera B is first color-matched.

  In step S709, it is determined whether or not all cameras have been color-matched. When all the colors have been matched (YES), the process is terminated. Otherwise (NO), the process moves to step S702.

  In the example of FIG. 8A, since the cameras C to H are unprocessed, the process continues. Up to this point, the color correction parameter is obtained for the camera B with respect to the reference camera A. Similarly, by applying Steps S702 to S709, color correction parameters can be obtained for the cameras C and D that have a large number of common objects with the color-matched cameras (Camera A and Camera B). A graph showing an example of the color matching state at this time is shown in FIG. A state in which color correction parameters are obtained for the cameras B to D with respect to the reference camera A is shown. At this time, when step S702 to step 709 are applied, the camera having a large number of common objects with the color-matched cameras (cameras A to D) is E, and therefore, a color correction parameter is obtained for the camera E. The processing at this time will be described in detail.

  In step S704, the number of common objects with the color-matched cameras (cameras A to D) is “3” for camera E and “0” for cameras F to H. Therefore, camera E is selected as the processing target. It is.

  In step S705, a color correction result from the unprocessed camera E to the reference camera A is obtained. At this time, color correction parameter candidates exist from the camera E to the cameras B to D as indicated by broken lines in FIG. Therefore, it is determined whether or not the result of color correction on the reference camera via each color correction parameter is the same. Specifically, it is determined whether or not the result of color correction is the same as “camera E → camera B → camera A”, “camera E → camera C → camera A”, “camera E → camera D → camera A”. judge. A corrected image may be generated and compared in each case. Alternatively, when color correction parameters can be connected, the connected color correction parameters may be directly compared.

  Here, as a result, it is assumed that the color correction parameters for the cameras B and C are found to be the same.

  In step S706, it is determined that 2/3 is equivalent, and YES is determined to satisfy the threshold number of candidates (three or more) and the ratio threshold value (60% or more).

  In step S707, it is assumed that the color correction parameter for the camera B is selected from the cameras B and C. As a result, a graph showing another example of the color matching state is as shown in FIG.

  Since the cameras F to H still remain as unprocessed cameras, this processing is continued. However, since the cameras F to H have a small common object number of “1” with the color-matched cameras, NO is determined in step S706, and the color correction parameter is not determined. Therefore, the result of this processing is a graph of the color matching state shown in FIG.

  In the above flowchart of FIG. 7, an unprocessed camera may remain. At this time, when there is a color correction parameter generation object between the unprocessed camera and the color-matched camera, the color correction parameter may be obtained using one of them without considering its reliability. Good.

  Or you may handle similarly to 1st Embodiment. That is, these processing results may be discarded, and video acquisition from the camera may be continued for a while, and the processing may be performed again after increasing the number of objects. Alternatively, the collation may be performed considering that there is an object in a state where color correction cannot be performed.

  Note that the processing when an unprocessed camera remains in the present embodiment is not limited to these.

  With the above processing, even if the color correction parameter may be erroneously obtained from the color correction parameter generation object, the correct color correction parameter can be determined.

<Third Embodiment>
In the second embodiment, when the color is matched with the color of the image of the reference camera, the color correction parameter may be applied a plurality of times. At this time, since the correction amount is accumulated, an error in color matching may increase. Therefore, an image processing apparatus that determines the reference camera so that the error can be reduced by reducing the number of times the color correction parameter is applied will be described.

  The configuration of the image processing apparatus of the present embodiment is the same as the configuration shown in the first embodiment or the second embodiment, but the behavior of the reference imaging device determination unit 210 is different.

  The reference imaging device determination unit 210 determines a camera (reference camera) that is a reference for color matching. In the present embodiment, the reference camera is determined so as to minimize the number of times the color correction parameter is applied. For example, “the sum of the number of times of color matching applied from each camera to the reference camera” is used as the number of times the color correction parameter is applied. Alternatively, in consideration of the appearance frequency of the object in each camera, “the sum of color matching frequencies obtained by multiplying the appearance frequency and the number of color matching” is used as the number of application times of the color correction parameter. Alternatively, a large weight may be set for the camera for which the color correction error is to be reduced, and the number of times of application may be obtained by applying the weight. The method for calculating the number of application times of color correction parameters in the present embodiment is not limited to these.

  Next, reference camera determination processing of the image processing apparatus 100 according to the present embodiment will be described with reference to the flowchart of FIG. Note that this processing is executed immediately after step S407a of the color matching processing described with reference to FIG.

  Steps S901 to S905 are processes for generating a color matching number matrix. For example, when the color matching state of the camera is the state shown in FIG. 10A, the color matching number matrix is obtained as shown in FIG. For example, since the camera A and the camera B are connected with one color correction parameter, the number of times of color matching is 1. Therefore, the corresponding portions of the camera A and the camera B in the matrix are also “1”. Similarly, since there are two color correction parameters between the camera A and the camera E, “2” is obtained.

  In step S906, the number of application times of the color correction parameter applied from each camera to the reference camera is obtained. Specifically, the sum is calculated for each column of the color matching frequency matrix. For example, in the case of FIG. 10B, as shown in FIG. 10C, the total number of times of color matching is obtained for each camera, and this is set as the number of times of applying the color correction parameter.

  Or you may consider the appearance frequency of the object in each camera here. For example, the number of objects observed per unit time is obtained for each camera. For example, the number of object images stored in the storage unit 202 is totaled for each camera, and how many objects appear on average per hour is calculated as the appearance frequency. The appearance frequency obtained for each camera can be obtained by multiplying the component of each column of the color matching frequency matrix shown in FIG. 10B by the appearance frequency for each camera and taking the sum for each column. Good. Alternatively, assuming that FIG. 10B is a matrix, a matrix in which appearance frequencies are arranged in one row may be obtained from the left side.

  In step S907, a color correction parameter is obtained using the camera with the minimum number of applications as a reference camera. For example, in FIG. 10C, since the camera B and the camera E have the minimum value, either one is selected. Here, it is assumed that the camera B is determined as the reference camera. Next, a color correction parameter is obtained using the camera B as a reference camera. Specifically, the color correction parameters are obtained so that the colors of the respective imaging devices match using the image of the color correction parameter generation object used when obtaining each color correction parameter. For example, the color correction parameter is obtained so that the state shown in FIG.

  In the color correction parameter calculation process, only the image of the color correction parameter generation object is determined without determining the color correction parameter. Then, after determining the reference camera, the color correction parameter is obtained in accordance with the color matching direction, and this processing can be realized with a small amount of calculation.

  As described above, since the reference camera is determined so that the number of application times of the color correction parameter is minimized, an increase in color matching error due to accumulation of correction amounts can be suppressed. In addition to the number of application times of the color correction parameter up to the reference camera as the number of application times of the color correction parameter, the reference camera is determined so that the color matching frequency is reduced in consideration of the appearance frequency of the object in each camera. As a result, not only the accumulation of errors can be suppressed, but also the amount of calculation can be expected to be suppressed.

<Other embodiments>
In each of the above embodiments, an example in which a person is used as an object has been described. However, the object is not limited to a person. A car, a poster, a signboard, or the like may be used as the object. Alternatively, a cart such as a supermarket may be used. Further, a mascot character such as a kigurumi may be used. That is, when a person is used as an object as in the above embodiments, the same object (same person) is extracted and color correction is performed using the color of the skin region, clothes, and the like. In contrast, if an object other than a person is used, color correction can be performed with high accuracy even when the same type of object is selected. For example, in the case of red cars of the same type, the color of the exterior of the car is generally the same. By using such color information, color correction can be performed with high accuracy even if the same object is not used.

  When such an object other than a person is used, it is desirable to use an object feature amount used by the object matching unit 205 that is suitable for the object. For example, it is conceivable to use a SIFT feature amount or the like as the image feature amount extracted from the luminance image (corresponding to the upper left in FIG. 3).

  The attribute extracted from the luminance image (corresponding to the upper right in FIG. 3) is determined by the object to be used. For example, in the case of a car or the like, it is conceivable to use a license plate character string. Moreover, if it is a car, a vehicle type etc. can be considered. In addition, it is conceivable to use a character string that appears in the case of a poster / signboard. When these objects are used at the same time, the type of the object may be used.

  Depending on the object, there may be objects of the same shape with different colors. For example, there are the same types of cars with different colors. Therefore, when using such an object, it should be determined that the attribute information is similar and similar. For example, in the case of a car, it is determined that they are similar only when the numbers on the license plates match.

  Further, as an image feature amount extracted from a color image (corresponding to the lower left in FIG. 3), it is conceivable to use a color histogram of an object. Alternatively, a color histogram for each region may be used. For example, in the case of a car or the like, regions such as bonnets, doors, and tires can be defined as regions, so it is conceivable to obtain a color histogram for each region. Similarly, a mascot or the like may be used by defining a region such as a head or a torso.

  As an attribute extracted from the color image (corresponding to the lower right in FIG. 3), for example, in the case of a car, it is conceivable to use the information of the vehicle type specified to the color as the attribute.

  Note that other feature quantities may be used as the feature quantities of the object, and the present invention is not limited to the feature quantities used.

  The color correction parameter calculation unit 208 obtains color correction parameters so that the colors of objects match. In the case of a person, the skin area and the clothes area are used. An area to be used may be determined in advance for an object other than the person. For example, in the case of a mascot or the like, if the type of the mascot or the like is specified, if there is a part with a rich color, it is possible to use the part. Alternatively, the color correction parameter may be obtained using the entire region.

  Note that the color correction parameters may be calculated by methods other than these, and the present invention is not limited to these color correction parameter calculation methods.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 201 Input part 202 Storage part 203 Collation condition input part 204 Collation result display part 205 Collation part 206 Query list creation part 207 Identification part 208 Color correction parameter calculation part 209 Correction image generation part 210 Reference | standard imaging device determination part

Claims (9)

From the captured images of the same or the same type of object captured by each of the at least one first imaging device, the plurality of second imaging devices, and the at least one third imaging device, the image region of the same or the same type of object is obtained. Input means for cutting out and inputting as an image of the same or same type of object;
From the images of the same or the same type of object captured by each of the first imaging device and the second imaging device, an image of the first object is specified, and the second captured image and the third imaging device A specifying means for specifying an image of the second object from the images of the same or the same type of object each imaged;
Determining means for determining any one of the first imaging device, the second imaging device, and the third imaging device as a reference imaging device;
When one of the first imaging device and the second imaging device is determined as the reference imaging device, the same or the same imaged by the reference imaging device based on color information of the image of the first object Generating a first corrected image in which the color of the image of the same type of object captured by each of the first imaging device and the second imaging device is matched with the color of the image of the type of object;
Based on at least one of the color information of the image of the first object and the color information of the image of the second object, the color of the image of the same or same type of object captured by the reference imaging device is changed to the first color. Generating means for generating a second corrected image in which the colors of the images of the same or the same type of object captured by the three imaging devices are combined;
An image processing apparatus comprising: There are a plurality of the second imaging devices that image the first object and the second object, and the color of the image of the first object and the image of the second object for each of the plurality of second imaging devices. Based on the information, among the plurality of second correction images to be generated, when the ratio of the second correction image determined to have the same color is a predetermined value or more,
The image processing apparatus according to claim 1, wherein the generation unit generates any one of the second corrected images determined to have the same color. When the specifying unit specifies the image of the first object and the image of the second object from the images of the same or the same type of object captured by one second imaging device,
The image processing apparatus according to claim 1, wherein the determination unit determines the second imaging apparatus as the reference imaging apparatus.   In the case where the first object is a first person and the second object is a second person, the generating unit is configured to perform the operation based on the color information of a face area or a torso area in the image of the first person. The first corrected image is generated, and the second corrected image is generated based on the color information of the face area or the torso area in the image of the second person. The image processing apparatus according to item.   The said specifying means specifies the image of said 1st object and the image of said 2nd object based on the similarity of the image of the said same or the same kind of object, Any one of Claim 1 thru | or 4 characterized by the above-mentioned. The image processing apparatus according to claim 1.   The specifying unit extracts an image feature amount of the same or the same type of object depending on color information from the first corrected image and the second corrected image generated by the generating unit, and the extracted The image processing apparatus according to claim 1, wherein the image of the same or the same type of object is specified again based on the image feature amount.   The specifying unit extracts an attribute of the same or the same type of object depending on color information from the first corrected image and the second corrected image generated by the generating unit, and the extracted attribute is used as the extracted attribute. The image processing apparatus according to claim 1, wherein the image of the same or same type of object is specified again based on the image. From the captured images of the same or the same type of object captured by each of the at least one first imaging device, the plurality of second imaging devices, and the at least one third imaging device, the image region of the same or the same type of object is obtained. Cutting out and inputting as an image of the same or same type of object,
From the images of the same or the same type of object captured by each of the first imaging device and the second imaging device, an image of the first object is specified, and the second captured image and the third imaging device A specifying step of specifying an image of the second object from the images of the same or the same type of object each imaged;
A determination step of determining any one of the first imaging device, the second imaging device, and the third imaging device as a reference imaging device;
When one of the first imaging device and the second imaging device is determined as the reference imaging device, the same or the same imaged by the reference imaging device based on color information of the image of the first object Generating a first corrected image in which the color of the image of the same type of object captured by each of the first imaging device and the second imaging device is matched with the color of the image of the type of object;
Based on at least one of the color information of the image of the first object and the color information of the image of the second object, the color of the image of the same or same type of object captured by the reference imaging device is changed to the first color. A generation step of generating a second corrected image in which the colors of the images of the same or the same type of object imaged by the three imaging devices are combined;
An image processing method comprising: From the captured images of the same or the same type of object captured by each of the at least one first imaging device, the plurality of second imaging devices, and the at least one third imaging device, the image region of the same or the same type of object is obtained. Cutting out and inputting as an image of the same or same type of object;
From the images of the same or the same type of object captured by each of the first imaging device and the second imaging device, an image of the first object is specified, and the second captured image and the third imaging device A specifying step of specifying an image of the second object from the images of the same or the same type of object each imaged;
A determination step of determining any one of the first imaging device, the second imaging device, and the third imaging device as a reference imaging device;
When one of the first imaging device and the second imaging device is determined as the reference imaging device, the same or the same imaged by the reference imaging device based on color information of the image of the first object Generating a first corrected image in which the color of the image of the same type of object captured by each of the first imaging device and the second imaging device is matched with the color of the image of the type of object;
Based on at least one of the color information of the image of the first object and the color information of the image of the second object, the color of the image of the same or same type of object captured by the reference imaging device is changed to the first color. A generation step of generating a second corrected image in which the colors of the images of the same or the same type of object imaged by the three imaging devices are combined;
A program that causes a computer to execute.

Images