JP5071535B2 - Feature information addition system, apparatus, method and program - Google Patents

Description

  The present invention relates to a feature information addition system, a feature information addition device, a feature information addition method, and a feature information addition program for adding feature information to video data.

  It is common to install surveillance cameras at facilities and stores as a means to ensure the safety of companies and private facilities, or as a means of grasping the characteristics of visitors such as age and gender It has become. Particularly in a wide range of facilities and stores, monitoring and observation by humans are limited, and therefore, it is performed to grasp the site from images taken by a plurality of surveillance cameras.

  As a related technique, for example, Patent Document 1 describes a method of collecting individual features from image information and audio information and performing personal recognition processing.

  In the method described in Patent Document 1, a plurality of types of feature values are collected, and effective feature amounts are identified among them, and personal recognition processing is performed, thereby improving collation accuracy.

JP 2006-293644 A

  However, when introducing a plurality of surveillance cameras, there is an example of introducing a monochrome surveillance camera that is relatively inexpensive considering the introduction cost. In this case, the user of the surveillance camera may not be able to recognize the color characteristics of the person's clothes from the monochrome video, and may not be able to identify the person.

  In addition, a thermographic camera may be introduced as a countermeasure against pandemic. In this case, the thermographic camera can acquire the temperature information of the person, but it is difficult to acquire information for specifying the person. Therefore, real-time monitoring by a person or the like is separately required, and if it is missed, it may be difficult to specify a person.

  As in the above example, the feature information that can be acquired is limited depending on the characteristics of the imaging device, and thus there are cases where the feature information that the user wants to know cannot be obtained.

  If the method described in Patent Document 1 is used, a plurality of types of feature amounts are collected, and effective feature amounts among them are identified and personal recognition processing is performed. It is only checked whether or not the specified person matches a person registered in advance. For this reason, this method cannot be applied to the purpose of supporting the identification of an arbitrary person from a video in which a plurality of persons photographed by the monitoring camera or the like as described above is reflected.

  Therefore, the present invention provides a feature information addition system, a feature information addition device, and a feature information addition device that can support the identification of an arbitrary subject from a photographed video even when the feature information that can be acquired is limited by the characteristics of the imaging device It is an object to provide a feature information addition method and a feature information addition program.

  The feature information addition system according to the present invention includes a feature information extraction unit that extracts feature information of a subject from video data captured by a plurality of shooting units, and a different shooting unit based on the feature information extracted by the feature information extraction unit. If it is determined that the determination unit determines whether or not the subjects included in the captured video data are the same, and the determination unit is the same, the video data captured by one of the different imaging units is included in the other video data. And feature information providing means for assigning feature information extracted from video data photographed by the photographing means.

  The feature information adding apparatus according to the present invention includes a feature information extraction unit that extracts feature information of a subject from video data captured by a plurality of shooting units, and a different shooting unit based on the feature information extracted by the feature information extraction unit. If it is determined that the determination unit determines whether or not the subjects included in the captured video data are the same, and the determination unit is the same, the video data captured by one of the different imaging units is included in the other video data. It is characterized by comprising characteristic information adding means for adding characteristic information extracted from video data taken by the photographing means.

  The feature information adding method according to the present invention extracts feature information of a subject from video data shot by a plurality of shooting means, and based on the extracted feature information, subjects included in video data shot by different shooting means are the same. If it is determined that they are the same, the feature information extracted from the video data photographed by the other photographing means is added to the video data photographed by one of the different photographing means. It is characterized by.

  The feature information addition program according to the present invention includes a feature information extraction process for extracting feature information of a subject from video data photographed by a plurality of photographing means on a computer, and a different photographing means based on the extracted feature information. When it is determined that the subject included in the video data is the same as the determination processing, and the video data captured by one of the different photographing means is captured by the other photographing means. A feature information adding process for adding feature information extracted from video data is executed.

  ADVANTAGE OF THE INVENTION According to this invention, even if it is a case where the characteristic information which can be acquired with the characteristic of imaging device is limited, it can support specifying arbitrary subjects from the image | photographed image | video.

It is a block diagram which shows an example of a structure of the feature information addition system by this invention. It is explanatory drawing which shows an example of the database which the image | video feature DB apparatus 300 memorize | stores. It is a flowchart which shows the process example which a feature information addition system performs. It is a block diagram which shows the minimum structural example of a feature information addition system.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of a feature information addition system according to the present invention. As shown in FIG. 1, the feature information addition system includes a photographing device 100, an information processing device 200, and a video feature DB (database) device 300. In addition, the photographing apparatus 100 and the information processing apparatus 200 can communicate with each other. Further, the information processing apparatus 200 and the video feature DB apparatus 300 can communicate with each other.

  The imaging device 100 is an apparatus that captures video. Specifically, the photographing apparatus 100 is realized by a color camera, a monochrome camera, a thermography camera, or the like. Note that any video format or video transmission format (analog format using a coaxial cable, digital format using an Ethernet cable (Ethernet is a registered trademark), etc.) may be used.

  The imaging device 100 includes an imaging unit 110 and a transmission unit 120.

  The photographing means 110 has a function of photographing a video. The photographing means 110 is realized by a device and mechanism for photographing an image such as a lens or a CCD element.

  The transmission unit 120 has a function of transmitting video data to the outside. The transmission unit 120 is realized by a device and a mechanism for transmitting captured images as data to the outside. Note that any format may be used for the video data transmitted from the transmission unit 120 to the outside.

  Specifically, the information processing apparatus 200 is realized by a personal computer, a server apparatus, or the like that operates according to a program. The information processing apparatus 200 includes a communication unit 210, a video processing unit 220, and a video analysis unit 230.

  Specifically, the communication unit 210 is realized by a CPU and a network interface unit of an information processing apparatus that operates according to a program. The communication unit 210 has a function of performing communication with external and internal units.

  Specifically, the video processing means 220 is realized by a CPU of an information processing apparatus that operates according to a program. The video processing unit 220 includes a management unit 221 and a conversion unit 222.

  When there are a plurality of photographing devices 100, the management unit 221 receives from which photographing device the video data received from each transmitting device 100 via the communication unit 210 in order to identify the photographing device 100. A function of assigning a photographing device ID, which is an ID for identifying whether or not Here, the management unit 221 may assign the ID added to the received video data as it is, or may assign a newly generated ID. For example, the management unit 221 stores the assigned ID in the storage unit and assigns the same ID when received from the same device.

  The conversion unit 222 has a function of converting the video data received from the photographing apparatus 100 into a specific data format, and a function of storing the converted video data in a specific location of a storage unit such as a hard disk device. Yes.

  As the data format conversion processing, for example, the conversion unit 222 converts the video data transmitted by the photographing apparatus 100 as a composite signal into the AVI format that is digital data.

  As processing for storing video data, for example, the conversion unit 222 stores the converted video data in a specific directory of a storage unit on a specific computer. At that time, the conversion means 222 has a function of creating data indicating that the data is stored in a specific directory on a specific computer.

  The video data converted by the conversion unit 222 includes general attached data according to the converted format. For example, when converted to the AVI format, the converted video data includes information such as data time stamp, resolution, and FPS (Frames Per Second).

  Specifically, the video analysis unit 230 is realized by a CPU of an information processing apparatus that operates according to a program. The video analysis unit 230 includes an identification unit 231, a feature extraction unit 232, a collation unit 233, and a writing unit 234.

  The identification unit 231 has a function of assigning a data ID, which is an ID for individually identifying the data, to the video data converted by the conversion unit 222.

  The feature extraction unit 232 has a function of extracting various features from the video data. For example, the feature extraction unit 232 has a function of extracting feature information such as a shooting time from a time stamp of video data or extracting the top of video data as image information.

  Further, for example, the feature extraction unit 232 uses a method described in Japanese Patent Application Laid-Open No. 2007-122097 (a coefficient determination method, a feature extraction method, a system and a program, and a pattern matching method, a system and a program). It has a function of extracting feature information indicating the facial features of the person inside. Further, for example, the feature extraction unit 232 has a function of extracting feature information indicating the color feature of clothes using a method described in Japanese Patent Laid-Open No. 2009-110460 (person image search device).

  The collation means 233 has a function of outputting a command for causing the video feature DB device 300 to transmit video data in order to collate the features of a plurality of video data. The collating unit 233 has a function of collating the characteristics of the received video data, and outputting to the writing unit 234 a command for assigning the characteristics of one video data to the other if the conditions are met. . As conditions, for example, it is possible to set complex information or feature information that needs to match when one feature information is assigned to the other.

  The writing unit 234 has a function of writing data to the video feature DB device 300 based on the command output from the matching unit 233. Specifically, the writing unit 234 transmits a predetermined command together with data to the video feature DB device 300, so that the video feature DB device 300 stores a database (hereinafter referred to as the video feature DB device 300) according to the received command. , Simply referred to as DB).

  Specifically, the image feature DB device 300 is realized by a storage device such as a magnetic disk device or an optical disk device, and stores a database. Note that the video feature DB apparatus 300 may be mounted in the information processing apparatus 200 or may be mounted on an external computer.

  The video feature DB device 300 has a function of adding or deleting data in the database based on a list acquisition command, a write command, or an erase command received from the outside.

  The video feature DB apparatus 300 stores, for example, a database including the photographing device ID 310, the data ID 320, the video data 330, the time 340, the color feature 350, the face feature 360, and the temperature feature 370 shown in FIG. To do. FIG. 2 is an explanatory diagram illustrating an example of a database stored in the video feature DB apparatus 300.

  The photographing device ID 310 is data for distinguishing a plurality of photographing devices 100. The photographing device ID 310 is given by the management unit 221 and written into the database in the video feature DB apparatus 300 by the writing unit 234.

  The data ID 320 is data for distinguishing each data. The data ID 320 is given by the identification unit 231 and is written in the database in the video feature DB device 300 by the writing unit 234.

  The video data 330 is data indicating actual video data, a part of the actual video data, or a location where the actual video data is stored. The video data 330 is created by the conversion unit 222 and written to the database in the video feature DB device 300 by the writing unit 234.

  The time 340 is data indicating the time when the data is acquired. The time 340 is extracted by the feature extraction unit 232 and written into the database in the video feature DB apparatus 300 by the writing unit 234.

  The color feature 350 is data indicating the color feature of a person's clothes in the video. For example, it is expressed in the form of RGB value and ratio of a certain color for each of the upper and lower clothes. The color feature 350 is extracted by the feature extraction unit 232 and written to the database in the video feature DB apparatus 300 by the writing unit 234.

  The face feature 360 is data representing the face feature of a person in the video. The face feature 360 is extracted by the feature extracting unit 232 and written into the video feature DB apparatus 300 by the writing unit 234.

  The temperature feature 370 is data representing the surface temperature of the person in the video, and is expressed in a format such as Celsius. The temperature feature 370 is extracted by the feature extraction unit 232 and written into the database in the video feature DB device 300 by the writing unit 234.

  Note that each means in the video processing means 220 and each means in the video analysis means 230 can communicate with each other for data transfer.

  Next, the operation of the feature information addition system will be described with reference to FIGS. FIG. 3 is a sequence diagram illustrating an example of processing executed by the feature information addition system.

  In the present embodiment, three photographing devices 100 are installed, which are a color camera, a monochrome camera, and a thermography camera, respectively. In addition, the photographing device IDs given by the management unit 221 to the respective cameras are “a001”, “a002”, and “a003”.

  The color camera and the thermography camera photograph the same part. In addition, the monochrome camera is installed at a short distance from the color camera, but does not photograph the same place. Further, it is assumed that no data is stored in the video feature DB device 300 in the initial state. In this embodiment, an example in which three photographing devices are installed will be described. However, the present invention is not limited to this, and the present invention can be applied to any number of photographing devices as long as the number is two or more. is there. The present invention is particularly effective in the case where feature information is mutually complemented in an environment where there are a plurality of different types of photographing devices.

  First, take a picture with a color camera. The photographing unit 110 of the photographing apparatus 100 photographs a color image of a person existing in the photographing range and outputs it as color image data to the transmission unit 120 (step S1 in FIG. 3). In the present embodiment, an example in which the subject is a person will be described. However, the present invention is not limited thereto, and may be an animal, for example.

  Next, the transmission unit 120 transmits the color video data to the communication unit 210 of the information processing apparatus 200 (step S2 in FIG. 3).

  Next, the communication unit 210 outputs the received color video data to the management unit 221 (step S3 in FIG. 3).

  Next, the management unit 221 assigns the photographing device ID 310 to the received color video data and outputs it to the conversion unit 222 (step S4 in FIG. 3). Here, it is assumed that the shooting ID 310 assigned by the management unit 221 is “a001”.

  Next, the conversion means 222 converts the format of the received color video data. In this embodiment, it is assumed that the conversion unit 222 converts the color video data itself into the AVI format. After converting the color video data, the conversion means 222 outputs the color video data converted into the AVI format to the identification means 231 (step S5 in FIG. 3).

  Next, the identification unit 231 gives a data ID 320 to the received color video data. Here, the data ID 320 given by the identification unit 231 is “A001”. After giving the data ID, the identification unit 231 outputs the color video data to the feature extraction unit 232 (step S6 in FIG. 3).

  Next, the feature extraction unit 232 extracts features from the color video data. Features extracted by the feature extraction unit 232 are a time 340 indicating the time when the color video data was captured, a color feature 350 indicating the color features of the clothes above and below the captured person, and a facial feature indicating the characteristics of the captured person's face. 360. After extracting these features, the feature extraction unit 232 outputs the color video data and the extracted features to the collation unit 233 (step S7 in FIG. 3).

  Next, the collation unit 233 outputs a command for obtaining a data list in the video feature DB device 300 to the communication unit 210 in order to collate the received color video data with each data stored in the video feature DB device 300. (Step S8 in FIG. 3). In the present embodiment, it is assumed that the following conditions are set in the collation means. The following conditions are examples, and other conditions may be set.

  First condition: When the time 340 is within a predetermined time (in this example, within 1 minute) and the data of which the facial features 360 match each other, the color feature 350 exists on one side and the color feature on the other side If 350 does not exist, the color feature 350 which exists in the data which does not exist is provided.

  Second condition: When the photographing device ID 310 is “a001” or “a003” and the time ID 340 matches (even if strictly different, for example, the same if the error is within a predetermined range) As for the data of the color feature 350, the face feature 360, and the temperature feature 370 held by each other, the data on the held side is added to the data portion that is not held.

  Next, the communication unit 210 transmits the data list acquisition command received from the collation unit 233 to the video feature DB device 300 (step S9 in FIG. 3).

  Next, the video feature DB device 300 performs an operation of extracting a data list in the DB and transmitting it to the communication unit 210 based on the received data list acquisition command. However, since there is no data in the DB at present and no corresponding data exists, the video feature DB device 300 does not transmit actual data, and only communicates data indicating a state of “no data list”. It transmits to the means 210 (step S10 of FIG. 3).

  Next, the communication unit 210 outputs data representing the state “data list does not exist” to the collation unit 233 (step S11 in FIG. 3).

  Next, since the collation unit 233 receives data indicating a state that “the data list does not exist”, the collation process is not performed. Then, the collation unit 233 determines to write the current video data, the added information, and the extracted features as they are to the video feature DB device, and writes the current video data, the added information, and the extracted features together with the write command. Is output to the insertion means 234 (step S13 in FIG. 3).

  Next, the writing unit 234 outputs an instruction to write the current video data, the added information, and the extracted feature to the video feature DB device 300 (step S14 in FIG. 3).

  Next, the communication unit 210 transmits the current video data, the added information, and the extracted features to the video feature DB device 300 together with the DB write command, and causes the data to be written in the video feature DB device 300 (FIG. 3). Step S15). Specifically, the video feature DB device 300 writes the current video data, the assigned information, and the extracted features in the database according to the write command. At this time, the data existing in the video feature DB device 300 is one of which the data ID 320 is identified by “A001”.

  Next, the process proceeds to step S1, and photographing is performed with a monochrome camera. The photographing unit 110 of the photographing apparatus 100 photographs a monochrome video of a person existing within the photographing range and outputs it as monochrome video data to the transmission unit 120 (step S1 in FIG. 3).

  Next, the transmission unit 120 transmits the monochrome video data to the communication unit 210 of the information processing apparatus 200 (step S2 in FIG. 3).

  Next, the communication unit 210 outputs monochrome video data to the management unit 221 (step S3 in FIG. 3).

  Next, the management unit 221 assigns the photographing device ID 310 to the received monochrome video data, and outputs it to the conversion unit 222 (step S4 in FIG. 3). Here, the shooting ID 310 assigned by the management unit 221 is “a002”.

  Next, the conversion means 222 converts the format of the received monochrome video data into the AVI format (step S5 in FIG. 3).

  Next, the identification unit 231 gives a data ID 320 to the received monochrome video data. Here, the data ID 320 given by the identification unit 231 is “A002”. After assigning the data ID, the identification unit 231 outputs the monochrome video data to the feature extraction unit 232 (step S6 in FIG. 3).

  Next, the feature extraction unit 232 extracts features from the monochrome video data. Here, the features extracted by the feature extraction unit 232 are a time 340 indicating the time when the monochrome video data was captured and a face feature 360 indicating the characteristics of the face of the person. After extracting the features, the feature extracting unit 232 outputs the monochrome video data and the extracted features to the matching unit 233 (step S7 in FIG. 3).

  Next, the collation unit 233 outputs a command for obtaining a data list in the video feature DB device 300 to the communication unit 210 in order to collate the received monochrome video data with each data stored in the video feature DB device 300. (Step S8 in FIG. 3).

  Next, the communication unit 210 transmits the data list acquisition command received from the collation unit 233 to the video feature DB device 300 (step S9 in FIG. 3).

  Next, the video feature DB device 300 extracts a data list in the DB based on the received data list acquisition command, and transmits the data list to the communication unit 210. The data transmitted at the present time is data identified by the data ID 320 of “A001” (step S10 in FIG. 3).

  Next, the communication unit 210 outputs data identified by the data ID 320 of “A001” to the verification unit 233 (step S11 in FIG. 3).

  Next, the collation means 233 has data ID 320 identified by “A001” (hereinafter also simply referred to as “A001”) and data obtained by adding “A002” to the data ID (hereinafter also simply referred to as “A002”). Are collated (step S12 in FIG. 3).

  Specifically, “A001” and “A002” are data in which the time 340 is within one minute and the facial features 360 match. Further, “A001” has a color feature 350, and “A002” has no color feature 350. Therefore, in order to meet the first condition, the matching function 233 assigns the color feature 350 of “A001” to the data identified by “A002” as the color feature of “A002”.

  The collation function 233 determines to write the current video data of “A002”, the added information, the extracted feature, and the newly added color feature to the video feature DB device, and the current video data, the added information, The extracted feature and the newly added color feature are output to the writing means 234 (step S13 in FIG. 3).

  Next, the writing unit 234 outputs an instruction to write the current video data, the added information, the extracted feature, and the newly added color feature to the video feature DB device 300 (step in FIG. 3). S14).

  Next, the communication unit 210 transmits the current video data, the added information, the extracted feature, and the newly added color feature to the video feature DB device 300 together with the DB write command (step S15 in FIG. 3). Then, the video feature DB device 300 writes the current video data, the added information, the extracted features, and the newly added color features in the database according to the received write command. At this time point, the data existing in the video feature DB device 300 is identified by the data ID 320 of “A001” and “A002”.

  Next, the process proceeds to step S1, and photographing is performed with a color camera and a thermography camera. The photographing means 110 of each photographing device 100 takes a color image and a surface temperature image of a person existing within the photographing range, and outputs the color image data and the surface temperature image data to the transmitting means 120 (step in FIG. 3). S1).

  Next, the transmission unit 120 transmits the color video data and the surface temperature video data to the communication unit 210 of the information processing apparatus 200 (step S2 in FIG. 3).

  Next, the communication unit 210 outputs the color video data and the surface temperature video data to the management unit 221 (Step S3 in FIG. 3).

  Next, the management unit 221 gives a photographing device ID 310 to the received color video data. Here, the photographing device ID 310 assigned by the management unit 221 is “a001”.

  Similarly, the management unit 221 assigns a photographing device ID 310 to the surface temperature video data. Here, the photographing device ID given by the management unit 221 is “a003”. Thereafter, the management unit 221 outputs both video data to the conversion unit 222 (step S4 in FIG. 3).

  Next, the conversion means 222 converts the format of both received video data to the AVI format (step S5 in FIG. 3).

  Next, the identification unit 231 gives a data ID 320 to both received video data. Here, the data ID 320 given by the identification means 231 is “A003” for the color video data and “A004” for the surface temperature video data. After giving the data ID, the identification unit 231 outputs the video data to the feature extraction unit 232 (step S6 in FIG. 3).

  Next, the feature extraction unit 232 extracts features from the video data. Here, the feature extraction unit 232 extracts a time 340 indicating the time when the video data was captured from the color video data and the surface temperature video data. Also, the feature extraction unit 232 extracts the color feature 350 from the color video data, and extracts the temperature feature 370 from the surface temperature video data. After extracting the features, the feature extracting unit 232 outputs each video data and the extracted features to the matching unit 233 (step S7 in FIG. 3).

  Next, the collation unit 233 outputs a command for obtaining a data list in the video feature DB device 300 to the communication unit 210 in order to collate the received video data with each data stored in the video feature DB device 300. (Step S8 in FIG. 3).

  Next, the communication unit 210 transmits the data list acquisition command received from the collation unit 233 to the video feature DB device 300 (step S9 in FIG. 3).

  Next, the video feature DB apparatus 300 extracts a data list in the DB based on the received data list acquisition command, and transmits it to the communication unit 210 (step S10 in FIG. 3). The data transmitted to the video feature DB device 300 at this time is data whose data ID 320 is identified by “A001” and “A002”.

  Next, the communication unit 210 outputs data identified by the data ID 320 of “A001” to the verification unit 233 (step S11 in FIG. 3).

  Next, the collation unit 233 collates data identified by the data ID 320 of “A001”, “A002”, “A003”, and “A004” (step S12 in FIG. 3).

  Specifically, in the data with the data ID 320 of “A003” and “A004”, the photographing device IDs 310 are “a001” and “a003”, respectively, and the time IDs 340 match. Therefore, in order to meet the second condition, the matching unit 233 holds the data of the color feature 350, the face feature 360, and the temperature feature 370 held by each other with respect to the data portions that are not held. Side data is given.

  Further, the collation unit 233 gives the temperature feature 370 to the data whose data ID 320 is “A003”, and gives the color feature 350 to the data whose data ID 320 is “A004”.

  Further, the collation unit 233 determines to write the video data, the added information, the extracted feature, and the newly added feature to the video feature DB device 300 regarding the data with the data ID 320 of “A003” and “A004”. Thereafter, the collating unit 233 outputs the video data, the added information, the extracted feature, and the newly added feature to the writing unit 234 (step S13 in FIG. 3).

  Next, the writing unit 234 outputs an instruction to write the video data, the added information, the extracted feature, and the newly added feature to the video feature DB device 300 to the communication unit 210 (step S14 in FIG. 3). .

  Next, the communication unit 210 transmits the video data, the added information, the extracted features, and the newly added features to the video feature DB device 300 together with the database write command. Then, the video feature DB apparatus 300 writes the video data, the added information, the extracted features, and the newly added features in the database according to the write command (step S15 in FIG. 3). At this time, the data existing in the video feature DB device 300 is identified by the data ID 320 of “A001”, “A002”, “A003”, or “A004”.

  Thereafter, for example, when the user performs an operation of reproducing the video data, the information processing apparatus 200 extracts the video data to which the feature information is added from the video feature DB apparatus 300 according to the operation of the user and reproduces it.

  As described above, in the present embodiment, when the feature information that can be obtained is obtained from the person shown in the video captured by the photographing device and the feature information that can be determined to be the same person matches, Is added as feature information. Thereby, it is possible to acquire feature information of a person who should not be originally acquired due to the characteristics of the video. As an example, regarding the feature information of the color feature of a person's clothing, even if it is a monochrome image that does not originally have this information, other specific features in the color image having the clothing color feature are collated and the same If so, it is possible to add the color feature of the clothes to the monochrome image side.

  The effect of the present invention is that characteristic information that cannot be acquired from the characteristics of the imaging device can be acquired from video data captured from the video device. The reason is that feature information not originally acquired can be given by collating each feature information given to a plurality of video data.

  From the above, the present invention makes it possible to acquire feature information that cannot be acquired by itself, by collating with other images, from the characteristics of the imaging device, with respect to the characteristics of the object captured by the imaging device, mainly the characteristics of the person. Can do.

  Further, the present invention has the following background and problems. In recent years, surveillance cameras have been used in many facilities and stores, and scenes where it is necessary to find a specific person in the video have definitely spread. In particular, the method of discovering a specific person based on the color of clothes is very important when it is necessary to find a specific person based on ambiguous appearance characteristics such as sighting information on the suspect in the case. It is an effective means.

  However, when the video imaged by the surveillance camera is not a color video but a monochrome video, it is difficult to distinguish the color of clothes from the monochrome video, so that there is a problem that it is difficult to exert the effect.

  In the real world, there are still many surveillance cameras that can shoot only monochrome images due to cost reasons, and there are many scenes where technology for finding a specific person based on the color of clothes cannot be used effectively. As described above, there are examples other than the color camera and the monochrome camera in which the person to be discovered cannot be found due to the characteristics of the photographing device.

  With respect to the above problems, the present invention makes it possible to complement and give feature information to each other, so that it is possible to distinguish the color feature of clothes even from a monochrome image, and efficient identification A person can be discovered.

  Next, the minimum configuration of the feature information addition system according to the present invention will be described. FIG. 4 is a block diagram illustrating a minimum configuration example of the feature information addition system. As shown in FIG. 4, the feature information addition system includes a feature information extraction unit 10, a determination unit 20, and a feature information addition unit 30 as minimum components.

  In the minimum configuration feature information adding system shown in FIG. 4, the feature information extraction unit 10 extracts subject feature information from video data captured by a plurality of imaging units. Next, based on the feature information extracted by the feature information extraction unit 10, the determination unit 20 determines whether or not the subject included in the video data captured by the different imaging unit matches. Next, when it is determined that the determination unit 20 matches, the feature information adding unit 30 adds the feature information extracted from the other video data to one video data of different photographing units.

  Therefore, according to the feature information addition system with the minimum configuration, the feature information of the subject extracted from the video captured by the other imaging unit can be added, and thus the feature information that can be extracted is limited by the characteristics of the imaging unit. Even in this case, it is possible to assist in identifying an arbitrary subject from the video.

  In the present embodiment, the characteristic configuration of the characteristic information addition system as shown in the following (1) to (5) is shown.

  (1) The feature information addition system is a feature information extraction unit (each of which extracts feature information of a subject (for example, a person) from video data captured by a plurality of imaging units (for example, a color camera, a monochrome camera, a thermography camera, etc.)). For example, based on the feature information extracted by the feature information extraction unit 232 and a determination unit that determines whether or not the subjects included in the video data captured by different imaging units are the same If it is determined that the determination means is the same (for example, realized by the collating means 233), the video data captured by one of the different imaging means is extracted from the video data captured by the other imaging means. Feature information providing means (for example, realized by collating means 233 or writing means 234) for providing the feature information Characterized in that it contains.

  (2) In the feature information addition system, the feature information extraction unit extracts time feature information (for example, time 340) indicating the shooting time from the video data as the feature information, and the determination unit extracts the time extracted by the feature information extraction unit. Based on the feature information, it may be configured to determine whether or not the subjects included in the video data photographed by different photographing means are the same.

  (3) In the feature information addition system, the feature information extraction unit extracts facial feature information (for example, a facial feature 360) indicating the facial feature of the subject as a person from the video data as the feature information, and the determination unit includes the feature information. Based on the facial feature information extracted by the extracting unit and the time feature information, it may be configured to determine whether or not the subjects included in the video data captured by the different capturing units are the same.

  (4) In the feature information addition system, the feature information extraction unit includes, as the feature information, color feature information indicating the color feature of the subject from video data captured by one of the different imaging units (for example, a color camera). (For example, the color feature 350) is extracted, and the feature information providing unit is the same as the subject included in the video data captured by the other capturing unit (for example, a monochrome camera) out of the capturing unit whose determination unit is different from the subject. If it is determined that the color feature information is added to the video data captured by the other imaging means (for example, the feature information is added and stored in the video feature DB device 300).

  (5) In the feature information addition system, the feature information extraction unit includes, as the feature information, temperature feature information indicating temperature characteristics of the subject from video data captured by one of the different imaging units (for example, a thermography camera). (For example, the temperature feature 370) is extracted, and the feature information adding unit is the same as the subject included in the video data captured by the other capturing unit (for example, a monochrome camera) among the capturing units whose determination unit is different from the subject. If it is determined that the temperature characteristic information is obtained, the temperature characteristic information may be added to the video data captured by the other imaging unit.

  The present invention can be applied to a use for supporting an operation of identifying an arbitrary person from a video taken by a surveillance camera or the like.

DESCRIPTION OF SYMBOLS 10 Feature information extraction means 20 Judgment means 30 Feature information addition means 100 Imaging equipment 110 Imaging means 120 Transmission means 200 Information processing apparatus 210 Communication means 220 Video processing means 221 Management means 222 Conversion means 230 Video analysis means 231 Identification means 232 Feature extraction means 233 verification means 234 writing means 300 video feature DB device

Claims (8)

Feature information extracting means for extracting feature information of each subject from video data taken by a plurality of photographing means;
Determination means for determining whether or not the subjects included in the video data captured by the different imaging means are the same based on the feature information extracted by the feature information extraction means;
If it is determined that the determination unit is the same, a feature information adding unit that adds feature information extracted from video data captured by the other imaging unit to video data captured by one of the different imaging units. A feature information addition system characterized by including The feature information extraction means extracts time feature information indicating the shooting time from the video data as feature information,
The feature information addition system according to claim 1, wherein the determination unit determines whether or not the subjects included in the video data captured by the different imaging units are the same based on the time feature information extracted by the feature information extraction unit. . The feature information extraction means extracts facial feature information indicating the facial feature of the subject as a person from the video data as feature information,
The determination unit determines whether the subjects included in the video data captured by the different imaging units are the same based on the face feature information extracted by the feature information extraction unit and the time feature information. The feature information addition system described. The feature information extraction unit extracts color feature information indicating the color feature of the subject from the video data captured by one of the different imaging units as the feature information,
When the determination unit determines that the subject is the same as the subject included in the video data captured by the other photographing unit among the different photographing units, the feature information providing unit captures the video captured by the other photographing unit. The feature information addition system according to any one of claims 1 to 3, wherein the color feature information is added to data. The feature information extracting means extracts temperature feature information indicating the temperature characteristics of the subject from the video data taken by one of the different shooting means as the feature information,
When the determination unit determines that the subject is the same as the subject included in the video data captured by the other photographing unit among the different photographing units, the feature information providing unit captures the video captured by the other photographing unit. The feature information addition system according to any one of claims 1 to 4, wherein the temperature feature information is added to data. Feature information extracting means for extracting feature information of each subject from video data taken by a plurality of photographing means;
Determination means for determining whether or not the subjects included in the video data captured by the different imaging means are the same based on the feature information extracted by the feature information extraction means;
If it is determined that the determination unit is the same, a feature information adding unit that adds feature information extracted from video data captured by the other imaging unit to video data captured by one of the different imaging units. A feature information adding device characterized by comprising: Extract feature information of each subject from video data shot by multiple shooting means,
Based on the extracted feature information, it is determined whether or not the subjects included in the video data captured by different imaging means are the same,
If it is determined that they are the same, the feature information extracted from the video data photographed by the other photographing means is added to the video data photographed by one of the different photographing means. . On the computer,
Feature information extraction processing for extracting feature information of each subject from video data taken by a plurality of photographing means;
A determination process for determining whether or not the subjects included in the video data captured by different imaging means are the same based on the extracted feature information;
If it is determined that they are the same, a feature information adding process for adding feature information extracted from the video data photographed by the other photographing means to the video data photographed by one of the different photographing means is executed. Feature information addition program.

Images