JP2012124658A - System and method for detecting specific person - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make precision of detection to be high, namely, to reduce missing and erroneous detection in a specific person detection system and a specific person detection method.SOLUTION: The specific person detection system which has means detecting an arbitrary character from a photographed image and means obtaining similarity from feature collation of a list of the specific persons and the character and detecting the specific person and which is constituted of an imaging device, a detection device and terminal device includes: means obtaining a photographing condition of the character; means obtaining the photographing condition of the most similar person; means determining directionality of the next photographing condition from comparison of both the conditions; means controlling the imaging device in accordance with determination; and means detecting the specific person by using the photographed image under the plurality of conditions, which are obtained from the control.

Description

  The present invention relates to a video monitoring system, and more particularly, to a system and method for improving detection accuracy for specifying a specific person from an image input from an imaging device or the like.

  Conventionally, video surveillance systems have been installed in facilities visited by an unspecified number of people such as hotels, buildings, convenience stores, financial institutions, dams and roads for the purpose of crime prevention and accident prevention. In such a video surveillance system, a person to be monitored is photographed by a surveillance imaging device, and the photographed video is transmitted to a monitoring center such as a management office or a security room, and a resident supervisor monitors it. However, depending on the purpose and necessity, attention is given or video is stored.

In recent years, the number of such video surveillance systems has increased along with worsening security and increasing social unrest, and the price of surveillance imaging devices being reduced. There are cases where it is installed on streets and on residential roads in residential areas.
However, the spread of installation locations of video surveillance systems is becoming increasingly apparent as the burden of visually checking the system operator's surveillance video.
From such a background, a system and technology for automatically detecting the occurrence of a specific event in an image in real time using an image recognition technology has been developed (for example, see Patent Document 1). One of them is a specific person detection system.

A specific person detection system is a system operator that automatically detects a specific person who has been registered in advance as a detection target (hereinafter referred to as a specific person) when it appears in the surveillance video. It is a system with a function to inform
The detection of the specific person is performed by calculating the similarity between the two person images by collating the person feature amount between the image of the specific person and the image of the person (character) appearing in the monitoring video, and the similarity is a predetermined value. When it is above, it carries out by determining with the same person.
Such a specific person detection system contributes to reducing the burden on the system operator.

JP 2010-171705 A

On the other hand, the specific person detection system targets video shot by a monitoring imaging device installed under a wide range of conditions such as a shooting angle of view, image resolution, and lighting environment. For this reason, it is difficult to increase the accuracy of detection, that is, to reduce missed detection and false detection, which is a big problem.
As a countermeasure against this, in the company application (Japanese Patent Application No. 2010-190862, filed on August 27, 2010), the image registration of a specific person, which was conventionally performed only before the start of system operation, is performed even during operation. There has been proposed a method for improving detection accuracy by increasing the number of registered images per person and the variety of shooting conditions and increasing the number of collation patterns.
The disclosed technique described in the above-mentioned literature is effective as a method that contributes to accuracy improvement.
However, the specific person detection system is actually required to further improve the detection accuracy.
The specific person detection system and the specific person detection method of the present invention have been made in view of such a situation, and an object thereof is to increase detection accuracy, that is, to reduce oversight and false detection.

  In order to achieve the above object, the present invention has means for detecting an arbitrary character from a photographed image, and means for detecting a specific person by obtaining a similarity based on a feature comparison between a list of specific persons and the characters. In the specific person detection system composed of the imaging device, the detection device, and the terminal device, the means for obtaining the shooting condition of the character, the means for obtaining the shooting condition of the most similar person, Means for determining directionality, means for controlling the imaging apparatus according to the determination, and means for detecting a specific person using captured images obtained under a plurality of conditions obtained from the control are provided.

  That is, in order to achieve the above object, a specific person detection system according to the present invention is a specific person detection system including an imaging device, a detection device, and a terminal device, and the detection device is an image input from the imaging device. Detecting means for detecting a person, a specific person detecting means for detecting a specific person from a feature comparison between a person detected by the detecting means and a list of specific persons stored in advance, and detecting the specific person, and the detecting means The first shooting condition calculation means for obtaining the first shooting condition of the person detected by the camera, the second shooting condition calculation means for obtaining the second shooting condition for the most similar person, the first shooting condition, and the From the comparison of the second imaging conditions, directionality determining means for determining the directionality of the next imaging conditions, control means for controlling the imaging device based on the determination of the directionality of the imaging conditions, It is obtained by a means for performing a specific person detected using photographic images of a plurality of conditions obtained from the control means.

  In order to achieve the above object, a specific person detection method of the present invention is a specific person detection method in a specific person detection system including an imaging device, a detection device, and a terminal device, and is input from the imaging device. A person is detected from the image, a similarity is obtained from a feature comparison between the detected person and a list of specific persons stored in advance, the specific person is detected, and the detected first person is photographed. Obtaining a condition, obtaining a second photographing condition for the most similar person, determining a directionality of the next photographing condition from a comparison between the first photographing condition and the second photographing condition, and determining the directionality of the photographing condition. Based on this determination, the image pickup apparatus control means controls the image pickup apparatus, and a specific person is detected by using captured images obtained under a plurality of conditions obtained from the image pickup apparatus control means.

  According to the present invention, it is possible to obtain higher detection accuracy in the specific person detection system.

It is a block diagram which shows the structure of one Example of the specific person detection system of this invention. It is a block diagram which shows the structure of one Example of the detection apparatus of the specific person detection system of this invention. It is a figure which shows one Example of the recording data of the specific person recording part in the detection apparatus of the specific person detection system of this invention. It is a flowchart for demonstrating one Example of the detection process procedure which the detection apparatus of the specific person detection system of this invention performs. It is a flowchart for demonstrating one Example of the detection process procedure which the detection apparatus of the specific person detection system of this invention performs. It is a flowchart for demonstrating one Example of the detection process procedure which the detection apparatus of the specific person detection system of this invention performs. It is a figure which shows one Example of the data of the determination result storage part in the detection apparatus of the specific person detection system of this invention. In the detection apparatus of the specific person detection system of this invention, it is a figure for demonstrating one Example of the content of a series of processes with respect to the 1st image when this person appears. In the detection apparatus of the specific person detection system of this invention, it is a figure for demonstrating one Example of the content of a series of processes with respect to the 2nd image when this person appears. The figure explaining the content of a series of processes with respect to the 3rd image when a person appears.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of each figure, the same reference numerals are assigned to components having a common function, and the description is omitted to avoid duplication as much as possible.

The configuration of an embodiment of the specific person detection system of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of an embodiment of the specific person detection system of the present invention. Reference numeral 100 denotes a network, 101 denotes an imaging device, 102 denotes a detection device, and 103 denotes a terminal device.
As shown in FIG. 1, an embodiment of the specific person detection system of the present invention is configured such that an imaging device 101, a detection device 102, and a terminal device 103 are connected to a network 100 and can communicate with each other. For simplification of description, in the embodiment of FIG. 1, the imaging device 101, the detection device 102, and the terminal device 103 are each configured as a single unit, but these can be connected to a network by a plurality of units. It is.

In FIG. 1, a network 100 is a communication line such as a dedicated line or an intranet, the Internet, or a wireless LAN that connects devices (the imaging device 101, the detection device 102, and the terminal device 103) to each other and performs data communication.
The image pickup apparatus 101 includes a zoom lens capable of perspective control and focus control, an image pickup device such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor), and an A / D (Analog to Digital) circuit for digitally converting an analog signal. , Temporary storage memory such as RAM (Random Access Memory), data transmission bus, timing circuit, external input / output interface, power supply circuit, pan / tilt head, visible light and near infrared LED (Light Emitting) It is a device such as a surveillance camera provided with illumination such as a diode.
The imaging device 101 converts incident light that has passed through a lens into an electrical signal by an imaging device, digitally converts the electrical signal by an A / D circuit, and stores it as image data in a temporary storage memory. The stored image data is output from the external input / output interface to the network in response to an external video request input to the external input / output interface or an instruction from the timing circuit. Similarly, in accordance with an external control command input to the external input / output interface, the direction of shooting, the perspective of the lens, the focus, the amount of illumination, and the like are changed.

The detection device 102 is a device such as a computer including an arithmetic circuit such as a CPU, a temporary storage memory such as a RAM, a recording medium such as an HDD (Hard Disk Drive), a data transmission bus, an external input / output interface, a power supply circuit, and the like. is there.
The detection device 102 stores the image data from the imaging device input to the external input / output interface via the network in the temporary storage memory. The detection apparatus 102 performs various calculations related to specific person detection including the method of the present invention on the stored image data using an arithmetic circuit. Further, the detection apparatus 102 stores in its recording medium a set of software including the method of the present invention, an OS (operation system), a set of images of a specific person (person feature amount), and the like. Further, the detection device 102 outputs the result of specific person detection to the network from the external input / output interface.

The terminal device 103 is an arithmetic circuit such as a CPU, a temporary storage memory such as a RAM, a recording medium such as an HDD, a data transmission bus, an external input / output interface, a power circuit, a screen such as a liquid crystal display, and a user input such as a keyboard and a mouse. It is a device such as a computer provided with a device. Further, the terminal device 103 may be configured by adding, for example, a buzzer, a voice announcement device, an LED light emitting device, or the like.
The terminal device 103 stores the image data from the imaging device 101 and the specific person detection result input from the detection device 102 input to the external input / output interface via the network 100 in a temporary storage memory. Further, the terminal device 103 uses an arithmetic circuit to convert the stored image data and the specific person detection result into a form suitable for display, and displays the screen. The terminal device 103 stores a set of software, an OS, and the like on the recording medium. Further, a user operation on the terminal device 103 is performed by a user input / output device.

Details of the detection system of FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of an embodiment of the detection device of the specific person detection system of the present invention. 2, devices having the same reference numerals as those in FIG. 1 indicate the same devices. 200 is a specific person recording unit, 201 is an image receiving unit, 202 is a person detection unit, 203 is a shooting condition calculation unit, 204 is a person feature amount calculation unit, 205 is a specific person primary determination unit, 206 is a determination result storage unit, 207 is a specific person secondary determination unit, 208 is a detection notification unit, 209 is an imaging condition determination unit, and 210 is a control instruction unit.
As illustrated in FIG. 2, the detection apparatus 102 includes a specific person recording unit 200, an image reception unit 201, a person detection unit 202, a photographing condition calculation unit 203, a person feature amount calculation unit 204, a specific person primary determination unit 205, a determination The processing unit includes a result storage unit 206, a specific person secondary determination unit 207, a detection notification unit 208, a shooting condition determination unit 209, and a control instruction unit 210.

In FIG. 2, a specific person recording unit 200 records an image of a person, a person feature amount, a person ID (first and last name, identification information given to the person), and the like when registering an image of the specific person. Part, for example, a database. The data to be recorded is given in advance, and the way of giving it may be whatever. The mode of recorded data in the specific person recording unit will be described later.
The image receiving unit 201 is a processing unit that performs image input / output from the outside of the detection apparatus 102. In this embodiment, the image receiving unit 201 receives input image data from the imaging apparatus 101 via the network 100.

The person detection unit 202 performs person detection using image recognition technology on the input image data input by the image reception unit 201, determines the presence of a character, and if there is a person, its image area coordinates A processing unit that performs output.
The imaging condition calculation unit 203 is a processing unit that calculates imaging conditions for the input image data input by the image reception unit 201 using an image processing technique. The shooting conditions are, for example, the brightness of the person, the clarity of the reflection, the apparent size of the person, and the like.

The person feature amount calculation unit 204 is a processing unit that performs feature amount calculation on the input image data input by the image reception unit 201 using an image recognition technique.
The specific person primary determination unit 205 is a processing unit that primarily determines whether or not the character detected by the person detection unit 202 is a specific person.
The determination result storage unit 206 is a processing unit that stores and stores the results determined by the specific person primary determination unit 205. A mode of stored data in the determination result storage unit 206 will be described later.

The specific person secondary determination unit 207 determines whether or not there is a specific person for the character detected by the person detection unit 202 based on a plurality of primary determination results determined by the specific person primary determination unit 205. It is a processing part to determine.
The detection notification unit 208 is a processing unit that inputs / outputs a result notification when a specific person is detected. In this embodiment, the detection notification unit 208 transmits specific person detection result data to the terminal device 103 via the network 100. Do.

The shooting condition determination unit 209 is a processing unit that determines the directionality of the next shooting condition.
Here, the directionality of the shooting condition refers to how to change the shooting condition (increase or decrease the parameter) of the next image. For example, the brightness of the above-mentioned person is brightened or darkened, and for example, the clarity of the above-mentioned person's reflection is made clear or blurred. The apparent size is either increased or decreased. In either case, the option of maintaining the current state is also included.

  The control instruction unit 210 is a processing unit that inputs and outputs control instructions in accordance with the determination result of the imaging condition determination unit. In this example, the control instruction unit 210 transmits a control instruction to the imaging apparatus.

  With reference to FIG. 3, an embodiment of recorded data in the specific person recording unit of the detection apparatus of the specific person detection system of the present invention will be described. FIG. 3 is a diagram showing an example of recorded data of the specific person recording unit 200 in the detection apparatus 102 of the specific person detection system of the present invention (see FIG. 2). Reference numerals 300 to 309 denote records. 310 is a record number cell, 311 is a person ID cell, 312 is a feature amount cell, and 313 is a person image cell.

In FIG. 3, the data records information of one photographing as one record. This example shows a state in which a total of 10 shootings for four specific persons, that is, 10 records are recorded, and is composed of records 300 to 309.
Each record 300 to 309 includes a record number cell 310, a person ID cell 311, a feature amount cell 312, and a person image cell 313.
The record number cell 310 is a data area for storing a record number. The record number is a number used for managing records, and is, for example, a numerical value of a serial number (continuous integer) uniquely assigned to each record.
The person ID cell 311 is a data area for storing a person ID of a specific person. In this embodiment, an example in which a character string as a person name is stored is shown, but an integer value or the like may be used as long as it is identification information given to a person.

The feature quantity cell 312 is a data area for storing a person feature quantity of a specific person. In the present embodiment, for ease of explanation, an example in which a decimal value as a one-dimensional value is stored is shown, but it is needless to say that a multidimensional value may be used.
The person image cell 313 is a data area for storing an image of a specific person. In this embodiment, an example in which the image itself is stored is shown, but instead, the address value of the data area in which the image is separately stored may be stored in hexadecimal or the like.

With reference to FIG. 4, an embodiment of the flow of detection processing executed by the detection device of the specific person detection system of the present invention will be described. FIG. 4 is a flowchart for explaining an embodiment of the detection processing procedure executed by the detection device of the specific person detection system of the present invention. FIG. 4 is divided into FIG. 4 (a), FIG. 4 (b), and FIG. 4A describes Step 400 to Step 408, FIG. 4B illustrates Step 409 to 417, and FIG. 4C illustrates Step 418 to 422.
The processing in FIG. 4 is executed by the detection device 102 unless otherwise specified.
Although not shown in FIG. 4, the initial value of the detection counter is “0”.

In the image reception standby step 400, the image reception unit 201 stands by until image data is input from the imaging device 101. When the image receiving unit 201 detects an incoming call (image reception), the image receiving unit 201 proceeds to a process of an image receiving step 401.
In an image receiving step 401, the image receiving unit 201 receives an image transmitted from the imaging device 101.
In the person detection step 402, the person detection unit 202 performs person detection on the received image data. For example, the person detection is performed by detecting a moving object based on a difference from the background image, and further detecting the person by the area, shape, moving speed, or the like of the moving object image area, or a person image that has been learned and prepared in advance. This is performed by a known technique such as a method of searching for the presence or absence of an image region having a feature pattern using an in-image search. In the present invention, any method may be used, and even if the person detection is detection for the entire person, detection is performed for a partial image region such as a face that is a representative part of a person. There may be. If a person is detected as a result of detection (detection), the process proceeds to detection counter value addition step 403. If not detected (non-detection), the process proceeds to secondary determination necessity determination step 418 (via T3). Proceed to the process of FIG.

In the detection counter value addition step 403, the person detection unit 202 adds 1 to the detection counter value and proceeds to the process of the person image region calculation step 404.
In the person image area calculation step 404, the person detection unit 202 calculates the image area of the person, and proceeds to the process of the apparent size calculation step 405.
In the apparent size calculation step 405, the shooting condition calculation unit 203 calculates the apparent size of the person for the character image area calculated in the human image area calculation step, and performs the process in the brightness calculation step 406. Proceed to As the apparent size of the person, for example, an area on the image, that is, a product value of the number of pixels in the vertical direction and the number of pixels in the horizontal direction of the image region, or the like is used.
In the brightness calculation step 406, the shooting condition calculation unit 203 calculates the brightness of the person for the character image area calculated in the person image area calculation step 404, and performs the processing of the clarity of image calculation step 407. move on. For example, the average brightness of the image area is used as the brightness of the person.
In the reflection clarity calculation step 407, the shooting condition calculation unit 203 calculates the clarity of the person's reflection for the character image area calculated in the person image area calculation step 404, and in the feature amount calculation step 408. Proceed to processing. For example, the amount of high-frequency components in the image area is used for clarity of the reflection of a person. The high frequency component can be obtained by transformation such as Fourier transformation on the image region.
In the feature amount detection step 408, the person feature amount calculation unit 204 calculates the person feature amount from the image area of the character, and proceeds to the process of the detection counter value determination step 409 (the process of FIG. 4B via T2). Go to). The human feature amount calculated here includes, for example, the shape and direction of the outline of the person, the color of the hair and skin, the gait, the shape and direction of the outline of the face, the size of the main components such as eyes, nose, and mouth. An image feature amount in which the shape, the arrangement relationship, and the like are digitized can be mentioned. In the present embodiment, these plural feature quantities are handled as one vector.

  In the detection counter value determination step 409, the specific person primary determination unit 205 evaluates the detection counter value. When the detection counter value is “1”, the process proceeds to the process of all collation step 410, and when the detection counter value is other than “1”, the process proceeds to the process of narrowing collation step 411.

  In all verification step 410, the specific person primary determination unit 205 performs all verification. Specifically, the character feature amounts of the characters calculated in the feature amount detection step 408 are collated in order (similar to all the person feature amounts of each specific person previously recorded in the specific person recording unit 200). The feature amount having the highest similarity is searched for. The person having this feature amount is set as the most similar person. After the processing, the process proceeds to the apparent size calculation step 412. Here, the similarity is a numerical value indicating the closeness between images (image feature amounts), so-called similarity, and a calculation method thereof is, for example, “Expression model for large-scale image collection” (Tsugaike) Reference can be made to a paper such as Tsuji et al., Journal of the Japan Photography Society, 2003, Vol. 66, No. 1, P93-P101). In the present embodiment, the similarity is a positive number that increases as the degree of similarity increases. For example, the similarity is a value obtained by adding a predetermined constant to the distance (norm) between feature amounts and taking the reciprocal.

  In the narrowing collation step 411, the specific person primary determination unit 205 performs narrowing collation. Specifically, the character feature amounts of the characters calculated in the feature amount detection step 408 are all of the person feature amounts of each specific person previously recorded in the specific person recording unit 200 in the all matching step 410. Only the person feature amount having the person ID of the most similar person obtained by the collation is sequentially collated (similarity calculation), and the feature amount with the highest similarity (the most similar person) is found. After the processing, the process proceeds to the apparent size calculation step 412. The person ID is acquired from the record whose detection counter value is “1” from the storage data of the determination result storage unit 206. A mode of stored data in the determination result storage unit 206 will be described later.

In the apparent size calculation step 412, the imaging condition calculation unit 209 calculates the apparent size of the person for the image region of the most similar person found in the all verification step 410 or the narrowing verification 411, and the brightness. The process proceeds to the calculation step 413. The apparent size of the person is the same as that in the appearance size calculation step 405 of the character. For example, the area on the image, that is, the product value of the number of pixels in the vertical direction and the number of horizontal pixels in the image area, etc. Use.
In the brightness calculation step 413, the shooting condition calculation unit 209 calculates the brightness of the person for the image region of the most similar person found in the all matching step 410 or the narrowing matching step 411, and calculates the clarity of the reflection. Proceed to step 414. The brightness of the person is the same as in the brightness calculation step 406 of the character. For example, the average brightness of the image area is used as the brightness of the person.
In the projection clarity calculation step 414, the imaging condition calculation unit 203 calculates the clarity of the projection of the person for the image area of the most similar person found in the all matching step 410 or the narrowing matching step 411. The process proceeds to determination result storage step 415. The clarity of the person's reflection is the same as in step 407, and for example, the amount of high frequency components in the image area is used. The high frequency component can be obtained by transformation such as Fourier transformation on the image region.

  In the primary determination result storage step 415, the determination result storage unit 206 stores and stores the results obtained so far. Here, the data to be stored include the current detection counter value, the person ID of the most similar person found in the all matching step 410, and the most similar person found in the all matching step 410 or the narrowing matching step 411. Similarity, apparent size calculation step 405 to visual clarity calculation step 407, apparent size, brightness, and visual clarity for the characters, and apparent size calculation step 412 It includes the apparent size, brightness, and clarity of the image of the most similar person calculated in the clarity calculation step 414. A mode of stored data in the determination result storage unit 206 will be described later.

In the directionality determination step 416, the shooting condition determination unit 209 determines the directionality of the next shooting condition based on the shooting condition for the character and the shooting condition for the most similar person, and performs the processing of the control instruction transmission step 417. move on. Here, both of the two photographing conditions are already stored in the determination result storage unit 206 by the previous processing step.
The directionality is determined as follows, for example. Assume that the shooting conditions are composed of the brightness Y of the person, the clarity F of the person's reflection, and the apparent size S of the person, and the shooting conditions C (t1) of the characters are represented by {Y (t1), F ( t1), S (t1)}, and the shooting condition C (t0) for the most similar person is {Y (t0), F (t0), S (t0)}. Note that t0 and t1 are times.
If Y (t1) <Y (t0), it is determined in the direction of increasing the brightness of the person. On the other hand, if Y (t1) ≧ Y (t0), it is determined that the brightness of the person is darkened.
If F (t1) <F (t0), the determination is made in the direction of increasing the clarity of the person's reflection. On the other hand, when F (t1) ≧ F (t0), it is determined to reduce the clarity of the person's reflection.
If S (t1) <S (t0), it is determined to increase the apparent size of the person. On the other hand, if S (t1) ≧ S (t0), it is determined to reduce the apparent size of the person.

Next, in the control instruction transmission step 417, the control instruction unit 210 creates a control instruction based on the next imaging condition calculated in the previous processing step, and the created control instruction data is transmitted to the imaging apparatus via the network 100. 101. For example, if it is determined in the previous processing step that the brightness of the person is increased, the control instruction unit 210 transmits control instruction data for increasing the amount of illumination. Further, for example, the control instruction unit 210 transmits control instruction data for increasing sensitivity of the image sensor. Similarly, if it is determined in the previous processing step that the brightness of the person is darkened, the control instruction unit 210 transmits control instruction data for reducing the amount of illumination. For example, the control instruction unit 210 transmits control instruction data for reducing the sensitivity of the image sensor.
The increase / decrease amount of the illumination and the increase / decrease amount of the imaging element sensitivity included in the control instruction data are determined in accordance with the specifications of the imaging device 101, are given as preset values of the detection device 102, and Y (t1 ) And Y (t0) are given in multiple stages.

Similarly, if it is determined in the previous processing step that the direction of the person's reflection is to be improved, the control instruction unit 210 transmits control instruction data for focusing. Similarly, if it is determined in the previous processing step that the direction of the person's reflection is lowered, the control instruction unit 210 transmits control instruction data for defocusing.
The focus increase / decrease amount included in the control instruction data is determined in accordance with the specification of the imaging apparatus 101, is given as a preset value of the detection apparatus 101, and is the difference between F (t1) and F (t0). It is given in multiple stages depending on the amount.

Similarly, if it is determined in the previous processing step that the direction of the person's apparent size is to be increased, the control instruction unit 210 transmits zoom-up control instruction data. Similarly, if it is determined in the previous processing step that the direction is to reduce the apparent size of the person, the control instruction unit 210 transmits zoom-down control instruction data.
The zoom increase / decrease amount included in the control instruction data is determined in accordance with the specification of the imaging apparatus 101, is given as a preset value to the detection apparatus 102, and is the difference between S (t1) and S (t0). It is given in multiple stages depending on the amount. In addition, when the person moves out of the shooting range due to zooming up, the control instruction unit 210 transmits control instruction data for changing the shooting direction so that the person falls within the shooting range.
After completing the transmission of the control instruction data, the control instruction unit 210 returns the process to the image reception standby step 400.

In the secondary determination necessity determination step 418, the determination result storage unit 206 determines whether the secondary determination is necessary. The determination is made based on the detection counter value at that time. When the detection counter value is “0”, it is determined that the secondary determination is not necessary, and the processing (in FIG. 4A via T1) is returned to the image reception standby step 400. If the detection counter value is other than “0”, it is determined that secondary determination is necessary, and the process proceeds to the secondary determination step 419 for a specific person.
In the specific person secondary determination step 419, the specific person secondary determination unit 207 performs the specific person secondary determination. The specific person secondary determination obtains a result having the highest similarity among the one or more specific person primary determination results stored in the determination result storage unit, and the similarity is a certain value (threshold). In this relationship, when the similarity is equal to or greater than a threshold value, the character is determined to be a specific person. As a result of the determination, if it is determined that the person is a specific person, the process proceeds to a specific person detection information transmission step 420. In other cases, the process proceeds to the detection counter value initialization step 421.

In the specific person detection information transmission step 420, the detection notification unit 208 transmits a specific person notification to the terminal device 103, and the process proceeds to the detection counter value initialization step 421. The transmission data includes a specific person ID, a person image thereof, and the like.
In the detection counter value initialization step 421, the determination result storage unit 206 sets the detection counter value to “0”, and proceeds to the processing of the primary determination all deletion step 422.
In the primary determination all deletion step 422, the determination result storage unit 206 deletes all the stored primary determination results, and returns to the image reception waiting step 400 (via T1 in FIG. 4A). Although not shown in the present embodiment, before returning to step 400, the control instruction unit 210 transmits control instruction data for initializing the imaging apparatus 101 to the imaging apparatus 101 as necessary. Also good.

  With reference to FIG. 5, an example of storage data in the determination result storage unit of the detection device in an embodiment of the specific person detection system of the present invention will be described. FIG. 5 is a diagram illustrating an example of data of the determination result storage unit in the detection device of the specific person detection system of the present invention. 500 to 509 are records, 510 is a record number cell, 511 is a detection counter value cell, 512 is a person ID cell, 513 is a similarity cell, 514 is an apparent size cell of a character, 515 is a brightness cell of the character Reference numeral 516 denotes an appearance clarity cell of the character, 517 denotes an apparent size cell of the most similar person, 518 denotes a brightness cell of the most similar person, and 519 denotes a clarity cell of the most similar person.

In FIG. 5, the data stores one primary determination result as one record. In the embodiment of FIG. 5, the primary determination result for 10 times at the maximum, that is, the state where 10 records can be stored and all the records are empty, is composed of records 500 to 509. Is done.
The record includes a record number cell 510, a detection counter value cell 511, a person ID cell 512, a similarity cell 513, an appearance size cell 514, a character brightness cell 515, and a character clarity cell. 516, an apparent size cell 517 of the most similar person, a brightness cell 518 of the most similar person, and an image clarity cell 519 of the most similar person.

  A record number cell 510 is a data area for storing a record number. The record number is a number used for managing records, and is, for example, a continuous integer value uniquely assigned to each record.

The detection counter value cell 511 is a data area for storing the detection counter value. An integer value is stored as the detection counter value.
The person ID cell 512 is a data area for storing the person ID of the most similar person. Although the example of FIG. 5 shows an example in which a character string as a person name is stored, an integer value or the like may be used as long as it is identification information given to a person.
The similarity cell 513 is a data area for storing the similarity. The example of FIG. 5 shows an example in which decimal values are stored.

The appearance size cell 514 of the character is a data area for storing the appearance size of the character. In the embodiment of FIG. 5, an integer value is stored with the apparent size as the number of pixels indicating the area on the image.
The character brightness cell 515 is a data area for storing the brightness of the character. In the embodiment of FIG. 5, an example is shown in which an integer value is stored with the brightness as an average value of luminance.
Character clarity cell 516 of the character is a data area for storing the clarity of the character image. In the embodiment of FIG. 5, an example is shown in which an integer value indicating the amount of high-frequency component is stored for clarity of reflection.

The apparent size cell 517 of the most similar person is a data area for storing the apparent size of the most similar person. In the embodiment of FIG. 5, an integer value is stored with the apparent size as the number of pixels indicating the area on the image.
The brightness cell 518 of the most similar person is a data area for storing the brightness of the most similar person. In the embodiment of FIG. 5, an example is shown in which an integer value is stored with the brightness as an average value of luminance.
The clarity of the reflection of the person of the most similar person is a data area for storing the clarity of the reflection of the person of the most similar person. In the embodiment of FIG. 5, an example is shown in which an integer value indicating the amount of high-frequency component is stored for clarity of reflection.

Here, the operation from the appearance of one person to the exit will be described more specifically with reference to FIGS. Here, it is assumed that this character is a detection target, that is, a specific person Mr. A.
For example, the data shown in FIG. 3 is recorded in the specific person recording unit 200 in advance. For simplification of explanation, the person's shooting condition is only the apparent size of the person. In addition, it is assumed that there are a total of two images taken from the appearance to the exit, and no person is shown on the third image.

FIG. 6 is a diagram for explaining the contents of a series of processes for the first image when this person appears in the detection apparatus of the specific person detection system of the present invention. The same reference numerals as those in FIG. 3 and FIG. 600 is an image, 601 is an image area, 602 is a detection counter value, 603 is an apparent size, 604 is a person feature, 605 to 614 is a matching process, 615 to 624 are similarities, 625 is the most similar person ID, 626 Is an image, 627 is an image area, 628 is an apparent size, and 629 is storage data.
FIG. 6A shows an image 600, and FIG. 6B is a diagram for explaining the contents of a series of processes for the first image of the detection apparatus 102. 6C is an enlarged view of the image 626 in the person image cell 313 of the record 307, and FIG. 6D is a view showing the storage data 629 as described in FIG. As shown in FIG. 6C, the person image cell 313 stores an apparent size 628 in addition to the image data 626.

In FIG. 6, an image 600 shows the first frame after the appearance of the person, and this image 600 is input from the imaging device 101 to the image receiving unit 201 of the detection device 102 via the network 100.
The image receiving unit 201 of the detection device 102 acquires this input image (see processing steps 400 to 401 in FIG. 4A). Then, the person detection unit 202 of the detection apparatus 102 performs processing such as person detection on this image (see processing steps 402 to 404 in FIG. 4A). An image area 601 indicates the person image area obtained above. The detection counter value 602 at this time is “1”.
Next, the imaging condition calculation unit 203 of the detection apparatus 102 calculates the apparent size of the person with respect to the human image region 601 (see the apparent size calculation step 405 in FIG. 4A). The apparent size 603 indicates a value calculated here.

  Next, the person feature amount calculation unit 204 of the detection apparatus 102 calculates a person feature amount for the person image region 601 (see the feature amount detection step 408 in FIG. 4B). The person feature quantity 604 indicates a value calculated here. In general, a multidimensional quantity is used as the human feature quantity. However, in this embodiment, an example using a one-dimensional decimal value is shown for the sake of simplicity of explanation.

Next, the specific person primary determination unit 205 of the detection apparatus 102 uses the person feature quantity 604 to perform all verification on the recording data of the specific person recording unit 200 (all verification step in FIG. 4B). 410.). The arrows of the collation processes 605 to 614 indicate that collation is performed with each record of the recording data, and the similarities 615 to 624 indicate the similarity calculated in each collation. As a result of all verification, the most similar person is determined to be a person recorded in the record 306.
The most similar person ID 625 indicates the person ID recorded in the record 306. For the similarity, a vector scalar value in the human feature space, which is a multidimensional quantity, is often used. However, in this embodiment, the absolute value of the difference between the human feature quantities is used to simplify the explanation. Here is an example.

  Next, the imaging condition calculation unit 209 of the detection apparatus 102 extracts a person image from the person image cell 313 of the record 306, and calculates the apparent size of the person (apparent size calculation step 412 in FIG. 4B). reference.). An image 626 shows the extracted person image. An image area 627 indicates a person image area on the image 626. The apparent size 628 indicates the value calculated here.

  Next, the determination result storage unit 206 of the detection apparatus 102 stores the detection counter 602 in the detection counter value cell 511 of the storage data record 500 in the determination result storage unit. Similarly, the most similar person ID 625 is stored in the person ID cell 512 of the record 500, the similarity 521 is stored in the similarity cell 513 of the record 500, and the apparent size 603 is displayed in the apparent size cell 514 of the character in the record 500. The apparent size 628 is stored in the apparent size cell 517 of the most similar person in the record 500 (see the primary determination result storage step 415 in FIG. 4B). The stored data 629 indicates the state of the stored data at that time.

  Next, the shooting condition determination unit 209 of the detection apparatus 102 determines the directionality of the next shooting condition from the apparent size 603 and the apparent size 628 (see the directionality determination step 416 in FIG. 4B). ). Here, since the value of the apparent size data 628 of the most similar person is larger than the apparent size 603 of the character, it is determined to increase the apparent size.

Next, the control instruction unit 210 of the detection apparatus 102 transmits a zoom-up instruction to the imaging apparatus 101 (see the control instruction transmission step 417 in FIG. 4B). In response to this control instruction, the imaging apparatus 101 side performs zoom-up control.
The series of processes related to the image 600 is as described above.

FIG. 7 is a diagram for explaining an example of the contents of a series of processes for the second image when this person appears in the detection apparatus of the specific person detection system of the present invention. The same reference numerals as those in FIGS. 3 and 5 indicate that they are the same, and the same reference numerals as those in FIG. 6 indicate that they are the same. 700 is an image, 701 is an image area, 702 is a detection counter value, 703 is an apparent size, 704 is a person feature, 705 to 707 are collation processes, 708 to 710 are similarities, 711 is an apparent size, 712 Is stored data.
FIG. 7A shows an image 700, and FIG. 7B is a diagram for explaining the contents of a series of processing for the second image of the detection apparatus 102. FIG. 7C is an enlarged view of the image 626 in the person image cell 313 of the record 307, and FIG. 7D is a diagram showing the storage data 712 as described in FIG. As shown in FIG. 7C, in the person image cell 313, in addition to the image data 626, an apparent size 711 is stored.

In FIG. 7, an image 700 shows the second frame when the person appears, and the image 700 is input from the imaging device 101 to the image receiving unit 201 of the detection device 102 via the network 100. Since the image 700 has been instructed to zoom up first, the person appears larger than the image 600.
The image receiving unit 201 of the detection device 102 acquires this input image (see processing steps 400 to 401 in FIG. 4A). Then, the person detection unit 202 of the detection apparatus 102 performs processing such as person detection on this image (see processing steps 402 to 404 in FIG. 4A). An image area 701 indicates the person image area 701 obtained above. The detection counter value at this time is “2”.
Next, the shooting condition calculation unit 203 of the detection apparatus 102 calculates the apparent size of the person with respect to the human image region 701 (see the apparent size calculation step 405 in FIG. 4A). The apparent size 703 indicates the value calculated here.

  Next, the person feature amount calculation unit 204 of the detection apparatus 102 calculates a person feature amount for the person data area 701 (see the feature amount detection step 408 in FIG. 4B). The person feature amount 704 indicates a value calculated here. In general, a multidimensional quantity is used as the human feature quantity. However, in the present embodiment, an example using a one-dimensional decimal value is shown for the sake of simplicity of explanation.

Next, the specific person primary determination unit 205 of the detection apparatus 102 uses the most similar person ID 625 and the person feature amount 704 to narrow down and match the recorded data of the specific person recording unit 200 (FIG. 4 ( (Refer to the narrowing collation step 411 of b).) The arrows of the collation processes 705 to 707 indicate that the record data is collated with each record having the same value as the person ID 625, and the similarities 708 to 710 indicate the similarity calculated in each collation.
As a result of the narrowing and collation, it is determined that the most similar person is a person recorded in the record 306.
Here, the record of the most similar person may be the same as or different from the previous record. In this embodiment, the same case is shown.

  Next, the imaging condition calculation unit 209 of the detection apparatus 102 extracts a person image from the person image cell 313 of the record 306 and calculates the apparent size of the person (see step 412 in FIG. 4B). An image 626 shows the extracted person image. An image area 627 indicates a person image area on the image 626. The apparent size 711 indicates the value calculated here. In this example, the apparent size 711 is the same value as the apparent size 628 shown in FIG.

  Next, the determination result storage unit 206 of the detection apparatus 102 stores the detection counter value 702 in the detection counter value cell 511 of the storage data record 501 in the determination result storage unit. Similarly, the most similar 710 is in the similarity cell 513 of the record 501, the apparent size 703 is in the apparent size cell 514 of the character in the record 501, and the apparent size 711 is in the appearance of the most similar character in the record 501. (Refer to the primary determination result storage step 415 in FIG. 4B). The stored data 712 indicates the state of stored data at that time.

  Next, the shooting condition determination unit 209 of the detection apparatus 102 determines the directionality of the next shooting condition from the apparent size 703 and the apparent size 711 (see the directionality determination step 416 in FIG. 4B). ). Here, since the apparent size 711 of the most similar person is smaller than the apparent size 703 of the characters, the determination is made in the direction of decreasing the apparent size.

Next, the control instruction unit 210 of the detection apparatus transmits a zoom-down instruction to the imaging apparatus (see the control instruction transmission step 417 in FIG. 4B). In response to this control instruction, the imaging apparatus 101 side performs zoom-down control.
The series of processes relating to the image 700 is as described above.

FIG. 8 is a diagram for explaining an example of the contents of a series of processes for the third image when this person appears in the detection apparatus of the specific person detection system of the present invention. The same reference numerals as those in FIGS. 3 and 5 indicate that they are the same, and the same reference numerals as those in FIGS. 6 and 7 indicate that they are the same. In addition, 800 is an image and 801 is storage data.
8A shows an image 800, and FIG. 8B is a diagram for explaining the contents of a series of processes for the third image of the detection apparatus 102. FIG. 8C shows the storage data 712 as described in FIG. 5 (same as FIG. 7D). Also in FIG. 8B, as shown in FIG. 7C, the person image cell 313 stores apparent size data in addition to the image data.

In FIG. 8A, an image 800 shows the third frame when this person appears, and this image is input to the detection device. The previous person has already left and is an image in which no one is shown.
The image receiving unit 201 of the detection apparatus 102 acquires the input image (see processing steps 400 to 401 in FIG. 4A). Then, the person detection unit 202 of the detection apparatus 102 performs processing such as person detection on this image (see processing steps 402 to 404 in FIG. 4A). Since the previous person has already left, a person image area cannot be obtained. The detection counter value at this time remains “2”.

Next, the person detection unit 202 of the detection apparatus 102 performs the specific person secondary determination based on the current detection counter value “2” (see processing steps 418 to 419 in FIG. 4C).
The determination result storage unit 206 of the detection apparatus 102 stores the similarity stored in the similarity cell 513 of the record 500 of the storage data 712 stored in the determination result storage unit 206 and the similarity 513 of the record 501. The similarity is compared, and the similarity stored in the similarity 513 of the record 501 (the value at the similarity 710 in FIG. 7B is “0.4”) is set as the highest similarity. Further, this highest similarity is compared with a threshold value (in this embodiment, “0.5”), and since it is equal to or greater than the threshold value, it is determined as a specific person. If it is less than the threshold, it is determined that the person is not a specific person.

Next, the detection notification unit 208 of the detection apparatus 102 transmits a specific person notification to the terminal apparatus 103 (see the specific person detection information transmission step 420 in FIG. 4C).
Next, the person detection unit 202 of the detection apparatus 102 sets the detection counter value to “0” and deletes all stored data. (FIG. 8 (c) stored data 801).

  As described above, the processing operation from the appearance of one person to the exit has been described with reference to FIGS. 6 to 8, but only the apparent size of the person has been described as the shooting condition of the character. However, the shooting condition of the character is obtained using at least one of the brightness of the person, the clarity of the reflection of the person, or the apparent size of the person, and the shooting condition of the most similar person is further obtained. It is also possible to determine the direction of the next shooting condition from the comparison of the two conditions, control the imaging device according to the determination, and perform specific person detection using the shot images obtained under the multiple conditions obtained from the control. Is self-explanatory.

  According to the above-described embodiment, it is possible to obtain higher detection accuracy in the specific person detection system.

Similarly, the imaging device and the detection device are shown to operate one by one, but a single detection device can be used for a plurality of imaging devices, and vice versa. It is also possible to make a plurality of detection devices correspond to the imaging device.
Similarly, each of the detection device and the terminal device is shown to operate in the opposite direction, but it is also possible to correspond to a plurality of detection devices with one terminal device, and conversely, one A plurality of terminal devices can correspond to the detection device.

Similarly, the imaging device and the detection device are shown as separate devices, but the same device may be used.
Similarly, the detection device and the terminal device are shown as separate devices, but the same device may be used.
Similarly, the shooting condition of the most similar person is calculated every time, but it may be calculated when the specific person is pre-registered and recorded in the specific person recording unit.
Similarly, all changes in imaging conditions are shown as mechanical control for the imaging apparatus, but electronic pseudo control for digital data in the imaging apparatus or detection apparatus may be used.
Similarly, although the specific person detection system is shown as a target, the detection may be performed not only on a specific person but also on a detection system such as a specific vehicle or another specific object.

  The present invention has been described in detail with the embodiments. However, the present invention is not limited to the above-described embodiments, and any person who has ordinary knowledge in the technical field to which the present invention belongs can modify the present invention based on the spirit and spirit of the present invention. Of course, the invention which can be changed is included.

  100: Network 101: Imaging device 102: Detection device 103: Terminal device 200: Specific person recording unit 201: Image reception unit 202: Person detection unit 203: Shooting condition calculation unit 204: Person feature amount Calculation unit 205: Specific person primary determination unit 206: Determination result storage unit 207: Specific person secondary determination unit 208: Detection notification unit 209: Shooting condition determination unit 210: Control instruction unit 300-309 : Record, 310: Record number cell, 311: Person ID cell, 312: Feature amount cell, 313: Person image cell, 500 to 509: Record, 510: Record number cell, 511: Detection counter value cell, 512: Person ID Cell, 513: similarity cell, 514: apparent size cell of characters, 515: brightness of characters Cell 516: Clarity cell of the appearance of the character 517: Apparent size cell of the most similar person 518: Brightness cell of the most similar person 519: Clarity cell of the reflection of the most similar person 600: Image, 601: image area, 602: detection counter value, 603: apparent size, 604: person feature, 605-614: matching process, 615-624: similarity, 625: most similar person ID, 626: image 627: Image area, 628: Apparent size, 629: Storage data, 700: Image, 701: Image area, 702: Detection counter value, 703: Apparent size, 704: Human feature, 705 to 707: Collation processing, 708 to 710: similarity, 711: apparent size, 712: storage data, 800: image, 801: storage data

Claims (2)

In a specific person detection system composed of an imaging device, a detection device, and a terminal device,
The detection device obtains similarity from a feature check between a detection unit that detects a person from an image input from the imaging device, a person detected by the detection unit, and a list of specific persons stored in advance, A specific person detecting means for performing person detection; a first photographing condition calculating means for obtaining a first photographing condition of the person detected by the detecting means; and a second photographing for obtaining a second photographing condition for the most similar person. Based on the condition calculation means, directionality determination means for determining the directionality of the next imaging condition based on the comparison between the first imaging condition and the second imaging condition, and the directionality determination of the imaging condition A specific person detection system comprising: control means for controlling an imaging device; and means for detecting a specific person using a plurality of captured images obtained from the control means. In a specific person detection method in a specific person detection system composed of an imaging device, a detection device, and a terminal device,
Detecting a person from an image input from the imaging apparatus, obtaining a similarity from a feature comparison between the detected person and a list of specific persons stored in advance, detecting the specific person, and detecting the detected person The first shooting condition of the person is obtained, the second shooting condition for the most similar person is obtained, and the directionality of the next shooting condition is determined from the comparison between the first shooting condition and the second shooting condition. The image pickup apparatus control means controls the image pickup apparatus based on the determination of the directionality of the image pickup conditions, and the specific person is detected using the picked-up images obtained from the plurality of conditions obtained from the image pickup apparatus control means. Specific person detection method.

Images