JP5349405B2 - Detection system, signal processing method thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a detection system, a signal processing method thereof, and a program which make it possible to detect subjects for each subject even if a plurality of subjects exist. <P>SOLUTION: In a detection system including a position storing unit 144 capable of storing information on position including detection range in an image of a plurality of subjects, and a subject position determination unit 17 for determining a position of a subject including the detection range by retrieving position and size of the subject in an entire image from detection information obtained from a determination unit 16 and capable of storing detection information including the detection range regarding the subject in a position storing unit, an analysis processing unit 14 obtains a signal for each predetermined scan face cycle from an imaging apparatus, calculates time average of signal level difference from the signal level difference of signals between a plurality of different scan faces, performs analysis processing for performing a predetermined calculation based on a value of time average, and performs the analysis processing selectively for an image in the designated detection position range in detection information when detection information is stored in the position storing unit 144. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a detection system that detects the state of a subject using an imaging device or the like, a signal processing method thereof, and a program, for example.

For example, an imaging apparatus used in a nighttime crime prevention system shown in Patent Document 1 has been proposed.
This imaging apparatus has a signal processing unit, and the light source has a frequency higher than 100 Hz or 120 Hz (when a commercial power source of 50 Hz or 60 Hz is used as the power source, the brightness of the light source fluctuates at 100 Hz or 120 Hz that is twice that). Modulate with high frequency. Further, the signal processing unit has a detection unit for detecting the high frequency modulation signal.
However, this imaging apparatus needs to have a frame rate higher than its modulation frequency.

By the way, a standard called NTSC (National Television Standards Committee) system or PAL (Phase Alternating Line) system is adopted for imaging apparatuses which are widely spread in general.
Since an image pickup apparatus adopting such a standard has a low frame rate, for example, it cannot be used as an image pickup apparatus for a crime prevention system as shown in Patent Document 1.

Therefore, a detection system that can detect a light source or the state of a subject irradiated with the light source and can be applied to a crime prevention system adopting a standard such as NTSC has been proposed (see Patent Document 2).
This detection system detects the state of the subject of the frequency component of the light source.

Japanese Patent No. 3019309 JP 2008-141251 A

However, in the above-described detection system, when there are a plurality of subjects detected from the video captured by the imaging device, it is difficult to determine the position of each subject and its detection information.
Moreover, it is difficult to measure the information of the detected position range with time.

Conventionally, as a position detection method, a detection method using comparison based on a background difference is known.
However, with this method, particularly when a plurality of subjects are detected, it is easy to detect moving objects and floating objects other than the target subject, and it is difficult to determine the target subject.

  The present invention provides a detection system capable of detecting a light source or the state of a subject irradiated with the light source, and capable of detecting each subject even when there are a plurality of subjects, and a signal processing method and program therefor There is.

A detection system according to a first aspect of the present invention includes a light source, an imaging device that images the light source or a subject irradiated with the light source, an analysis processing unit that performs an analysis process on a signal acquired from the imaging device, A determination unit that detects and determines the state of the light source or the subject based on an analysis result of the analysis processing unit, a position storage unit that can store information about positions including detection ranges in the images of a plurality of subjects, and the determination unit From the obtained detection information, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, the detection information including the detection range related to the subject is acquired, and the detection information is A subject position determination unit that can be stored in the position storage unit as information on the image, and the analysis processing unit acquires the signal from the imaging device for each predetermined scanning plane period. The time average of the signal level difference is obtained from the signal level difference of the signal between a plurality of different scanning planes, and an analysis process for performing a predetermined calculation based on the value of the time average is performed. the calculation result is output to the determination unit, when the detection information is stored in the position storage unit selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information.

The signal processing method of the detection system according to the second aspect of the present invention includes an imaging step of capturing an image of a light source or a subject irradiated with the light source by the imaging device, and acquiring the signal from the imaging device for each predetermined scanning plane period. An analysis processing step for obtaining a time average of the signal level difference from the signal level difference of the signal between a plurality of different scanning planes, and further performing an analysis process for performing a predetermined calculation based on the value of the time average, and the analysis A determination step for detecting and determining the state of the light source or subject based on the analysis result of the processing step, and a detection range by searching for the position and size of the subject in all images from the detection information obtained in the determination step. The position of the subject to be included is determined, detection information including a detection range related to the subject is acquired, and the detection information is stored in the position storage unit as information about the position. Has a Utsushitai position determining step, and in the analysis processing step, selectively if the detected information is stored in the position storage unit for image detection knowledge range specified in the above detection information Includes a step of performing the above analysis processing.

According to a third aspect of the present invention, an image pickup process for picking up a light source or a subject irradiated with the light source with an image pickup device, and acquiring the signal for each predetermined scan plane period, and acquiring the signal between a plurality of different scan planes. The time average of the signal level difference is obtained from the signal level difference, and further, an analysis process for performing a predetermined calculation based on the value of the time average, and detecting the state of the light source or subject based on the analysis result of the analysis process From the determination process for determination and the detection information obtained by the above determination process, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, and the detection information including the detection range for the subject And a subject position determination process for storing the detection information in the position storage unit as information on the position. In the analysis process, the detection information is stored in the position storage unit. If it is憶is a program for executing the signal processing of the detection system including a process for selectively above analysis process to the image of the test knowledge range specified in the detection information to the computer.

  According to the present invention, it is possible not only to detect the state of a light source or a subject irradiated to the light source, but also to detect for each subject even when there are a plurality of subjects.

It is a figure which shows the structural example of the detection system which concerns on embodiment of this invention. It is a figure which shows an example of the information preserve | saved in the position memory | storage part which concerns on this embodiment. It is a figure which shows notionally the state before the position determination of the to-be-photographed object in the to-be-photographed object position determination part which concerns on this embodiment, and the state after position determination. It is a figure which shows an example of the (x, y) width table information of the detection range referred in order to perform position determination in the to-be-photographed object position determination part which concerns on this embodiment. It is a flowchart for demonstrating specifically the detection position determination process in the to-be-photographed object position determination part which concerns on this embodiment, Comprising: It is a figure which shows the process in the case of performing a position determination process. It is a flowchart for demonstrating specifically the detection position determination process in the to-be-photographed object position determination part which concerns on this embodiment, Comprising: It is a figure which shows a process when not performing a position determination process. It is a figure for demonstrating the search method of the object of one light source or reflected light. It is a figure for demonstrating the fine adjustment process of a detection range. It is a figure which shows an example for demonstrating the structure of CCD which concerns on this embodiment. It is a figure for demonstrating the time series of CCD of FIG. It is a figure which shows the example of an arrangement | sequence of the pixel of a single plate complementary color filter type CCD. It is a figure which shows an example of the combination of the color signal in odd field OFD and even field EFD. It is a timing chart for demonstrating the signal processing method of the luminance signal of the light source analysis process part which concerns on this embodiment. It is a figure which shows the waveform W6 about the frequency component contained in the light source. It is a figure which shows an example of the relationship between duty ratio D and square sum SACBD. It is a flowchart figure for demonstrating a series of operation | movement outline | summary of the detection system which concerns on this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example of a detection system according to an embodiment of the present invention.

The detection system 10 includes a light source 11, an imaging device 12, a luminance signal extraction unit 13, a light source analysis processing unit 14, a filter processing unit 15, a determination unit 16, and a subject position determination unit 17.
The light source analysis processing unit 14 includes a first calculation unit (A) 141, a second calculation unit (B) 142, a calculation processing unit (time average square sum calculation processing unit) 143, and a position storage unit 144.
The subject position determination unit 17 includes a position search processing unit 171 and a position search information storage unit 172.

  The light source 11 illuminates the subject with a predetermined luminance. The luminance of the light source 11 is variable, and the luminance within the charge accumulation time of the imaging element included in the imaging device 12 changes at a period 4n times the field period of the imaging device 12. Here, n = 1, 2, 3,.

The imaging device 12 used in the detection system 10 according to the present embodiment employs an imaging device having the following specifications.
As an example of the imaging device constituting the imaging apparatus 12, a single-plate complementary color filter and a field storage type interline transfer CCD (Charge Coupled Device) image sensor (hereinafter simply referred to as a CCD) are used.
As an example, the television system of the imaging device 12 employs the NTSC system, the scanning system employs interlace, the scanning frequency is 15.734 KHz, and the vertical frequency is 59.94 Hz.
The imaging device 12 having such a configuration images the subject illuminated by the light source 11 and outputs a signal obtained by the imaging to the luminance signal extraction unit 13.
Although a CCD image sensor is illustrated here, a CMOS image sensor can also be applied.

The luminance signal extraction unit 13 extracts a luminance signal from the input signal and outputs the luminance signal to the first calculation unit 141 and the second calculation unit 142 of the light source analysis processing unit 14.
The luminance signal extracted by the luminance signal extraction unit 13 is adjusted to a signal level optimized for calculation.
The signal level needs to be a signal level that does not overflow in the output values of the first calculation unit 141, the second calculation unit 142, and the calculation processing unit 143. Therefore, the luminance signal extraction unit 13 includes a circuit that adjusts the luminance signal level.
The adjustment value of the luminance signal level may have a table capable of mode switching when there are several modes. The mode may be a video signal standard such as NTSC or PAL, or a frequency mode of the imaging apparatus.

In the light source analysis processing unit 14, the first calculation unit 141 obtains the time average of the level difference of the luminance signal in the m-th and (m + 2) -th fields in the same area of the imaging device.
The output result A of the first calculation unit 141 is output to the calculation processing unit 143.
The second calculation unit 142 obtains a time average of the level difference of the luminance signal in the (m + 1) th and (m + 3) th fields in the same region of the input luminance signal.
The output result B of the second calculation unit 142 is output to the calculation processing unit 143.
The details of the operations of the first calculation unit 141 and the second calculation unit 142 will be described later.
The first calculation unit 141 and the second calculation unit 142 perform calculation processing in accordance with calculation range designation information for analyzing the image stored in the position storage unit 144.

The output result A and the output result B output from the first calculation unit 141 and the second calculation unit 142, respectively, are input to the calculation processing unit 143.
The arithmetic processing unit 143 obtains a square sum value (A ² + B ² ) of the output result and outputs it to the filter processing unit 15.
The arithmetic processing unit 143 performs arithmetic processing according to the calculation range designation information for analyzing the image stored in the position storage unit 144.

The position storage unit 144 stores, for example, detection information on the positions and sizes of a plurality of subjects detected with high accuracy by removing unnecessary moving objects from the signal component of the light source, which is determined by the subject position determination unit 17.
As described above, the light source analysis processing unit 14 includes the position storage unit 144 that can specify a calculation range for image analysis of the first calculation unit 141 and the second calculation unit 142.
The position storage unit 144 can store a plurality (n) pieces of position information.
Therefore, the detection processing of the light source analysis processing unit 14 is performed in the detection of the light source or the subject irradiated with the light source. One piece of analysis processing is performed within one frame.
Alternatively, if n analysis processes cannot be performed within the time of one frame, the analysis process in one frame is performed in one n range, and after the completion, the analysis in the second range from the next frame is performed. A method of performing processing in order may also be used.
An image in the n designated ranges obtained by the analysis process can be acquired as a detected image if there is a light source or a subject irradiated with the light source.
Since the detection range is not specified in the initial state in which position information or the like is not stored in the position storage unit 144, the light source analysis processing unit 14 performs light source analysis processing on the entire image instead of selective detection processing.

Further, when the position storage unit 144 is a nonvolatile memory or a setting file, the detection position can be restored without being affected by a power failure. In addition, it has a flag for selecting whether or not to determine the subject position.
A part of the position storage unit 144 has a volatile memory and can be used for temporary storage.
The position storage unit 144 has a function of storing information such as the position coordinates and size of the subject, the frequency detection sensitivity (signal level) of the subject, and the individual number of the subject.

  FIG. 2 is a diagram illustrating an example of an information table stored in the position storage unit 144 according to the present embodiment.

In FIG. 2, “No” indicates the individual number of the subject. “X, y” indicates the start position coordinates of the subject, and “xe, ye” indicates the end position coordinates of the subject. “N” indicates a number corresponding to the number of subjects. “Level” indicates a signal level (signal intensity) which is frequency sensitivity of the subject.
In this example of the table TBL1, the subject of “No. 1” is the first subject with the start position coordinates “10, 10” and the end position coordinates “20, 20”, and the signal level is relatively high. “40”.
The subject of “No. 2” is the second subject whose start position coordinates are “30, 30” and end position coordinates are “50, 60”, and its signal level is relatively “43”.
The “No. 3” subject is the third subject whose start position coordinates are “100, 100” and end position coordinates are “120, 120”, and its signal level is relatively “50”.
The subject of “No. 4” is the fourth subject having a start position coordinate of “300, 300” and an end position coordinate of “325, 315”, and its signal level is relatively “60”.

As described above, the light source analysis processing unit 14 of the present embodiment includes the position storage unit 144 that is a memory that can store information on the positions of a plurality of subjects, and the position storage unit 144 has coordinates of an image area that indicates the subject position. It has subject detection information such as value and subject number.
Source analyzing unit 14, when the detection position coordinates stored in the position storage section 144 is present, has a function of analyzing an image of the detection knowledge range designated by the coordinates (detection).
The subject position determination unit 17 stores the detection information in the position storage unit 144.

The light source analysis processing unit 14 is, for example, two modes that are designated by a control system (not shown) or automatically switched according to the presence or absence of detection information in the position storage unit 144, specifically, an all-pixel detection mode and a subject position detection mode. It is possible to operate with.
In the all-pixel detection mode, the light source analysis processing unit 14 does not specify a detection range in the initial state where position information or the like is not stored in the position storage unit 144. Perform analysis processing.
Based on the detection result in the all-pixel detection mode, the subject position determination unit 17 described later detects information regarding the positions of a plurality of subjects, and the detection information is stored in the position storage unit 144.
For example, when this detection information is stored in the position storage unit 144, the light source analysis processing unit 14 switches from the all-pixel detection mode to the subject position detection mode.
The subject position detection mode, the light source analyzing unit 14, based on the detection position coordinates or the like stored in the position storage unit 144, analyzes the image of the test knowledge range designated by the coordinates (detection).
Since the light source analysis processing unit 14 is provided, the detection system 10 can detect only the position of the light source or the subject irradiated with the light source, and does not detect the position excluded as the subject. Is also effective and can be detected with high accuracy.

In the initial state described above, the filter processing unit 15 performs noise reduction on the entire image by filter processing and outputs the noise to the determination unit 16.
The filter processing unit 15 performs noise reduction on the n detected images in the light source analysis processing unit 14 or eliminates a portion that has not been detected by the filter processing, and outputs the result to the determination unit 16.

In the above-described initial state, the determination unit 16 is output from the arithmetic processing unit 143 and based on the square sum value (A ² + B ² ) of the entire image that has been reduced in noise by the filter processing unit 15. For example, it is determined whether the subject imaged is stationary or moving. Details of the operation of the determination unit 16 will be described later.
Based on the square sum value (A ² + B ² ) of the detected image that is output from the arithmetic processing unit 143 and from which the filter processing unit 15 has eliminated noise and has not been detected, the determination unit 16 For example, it is determined whether the subject imaged at 12 is stationary or moving. Details of the operation of the determination unit 16 will be described later.

  The subject position determination unit 17 stores the position search processing unit 171 that searches for the position of the subject from all the images from the detection state obtained by the determination unit 16 and the position search result of the position search processing unit 171. A search information storage unit 172 is included.

FIGS. 3A and 3B are diagrams conceptually showing states before and after the position of a subject in the subject position determination unit according to the present embodiment.
As shown in FIG. 3A, the subjects OBJ1 and OBJ2 before the position determination are searched, and as shown in FIG. The position is determined, and the range becomes the analysis target of the light source analysis processing unit 14 and the detection target of the determination unit 16.
Detection information such as the range after the position determination is stored in the position storage unit 144 of the light source analysis processing unit 14. In the present embodiment, as described above, there are a subject position detection mode in which only the detection of the range is possible and an all image detection mode for all images.
For example, in an application where it is necessary to detect only a subject at a specific position, the operation is performed in the subject position detection mode in the range stored in the position storage unit 144.
In that case, as described above, in order to set a specific position, the subject position determination unit 17 first searches the subject target position and stores it in the position storage unit 144, and thereafter only the detection of the stored position is performed. Applicable for use.

The subject position determination unit 17 determines the detection range from the size of the subject.
In the subject position determination unit 17, the detection range is determined using the first mode when the (x, y) width of the detection range DTRG is fixed to one and the (x, y) width according to the size of the subject. Second, a table value detection range DTRG having a size exceeding (or below) the current (x, y) width is selected from a plurality of tables prepared in advance with a (x, y) width. There are two modes:
The (x, y) width table of the detection range DTRG is stored in the position search information storage unit 172, for example.
The subject position determination unit 17 can select either the first mode or the second mode, and has a configuration having these functions.
This detection range is determined from a table in which the size of the subject is prepared in advance and the detection range is determined, so that the size and position of the subject can be detected at high speed.

  FIG. 4 is a diagram illustrating an example of (x, y) width table information of the detection range referred to for position determination in the subject position determination unit 17 according to the present embodiment.

In FIG. 4, “No” indicates the number of (x, y) width information of the subject detection range. “X, y” indicates the size information of the object detection range DTRG, and sBlK indicates the number of search blocks (number of pixels).
In the example of the (x, y) width table TBL2, the detection range DTRG of “No. 1” is “10 × 10” and the number of blocks is “2”.
The detection range DTRG of “No. 2” is “30 × 30” and the number of blocks is “4”.
The detection range DTRG of “No. 3” is “100 × 100” and the number of blocks is “8”.
The detection range DTRG of “No. 4” is “300 × 300” and the number of blocks is “12”.
The position search processing unit 171 of the subject position determination unit 17 refers to the (x, y) width table TBL2, and searches, for example, in ascending order from the smaller “No” or in descending order from the larger “No”. The detection range most suitable for the target subject is determined.

The subject position determination unit 17 is configured to be able to finely adjust the detection range DTRG from the size of the subject.
For fine adjustment of the detection range, from the state where the detection range DTRG (x, y) is determined, the fine adjustment value table prepared in advance or the fine adjustment formula [x1 = x * sP1, y1 = y * sP2] x, y) The configuration has a function of finely adjusting the width.
Here, “sP1” and “sP2” are adjustment parameters and take values of, for example, 1 <sP1, sP2 <2.
With this function, even if a small shake or movement occurs in a subject that can be detected within the normal detection range, it can be detected without departing from the subject range.
This is particularly effective when the subject is a light source.

Next, the detection position determination process in the subject position determination unit 17 according to the present embodiment will be specifically described with reference to FIGS.
5 and 6 are flowcharts for specifically explaining the detection position determination process in the subject position determination unit 17 according to the present embodiment, and FIG. 5 is a flowchart in the case of performing the position determination process. FIG. 6 is a flowchart when the position determination process is not performed.
FIG. 7 is a diagram for explaining a method of searching for a subject of one light source or reflected light.
FIG. 8 is a diagram for explaining detection range fine adjustment processing.
The following processing is performed mainly by the position search processing unit 171. Here, the processing will be described as processing by the subject position determination unit 17.

The subject position determination unit 17 determines whether or not to perform position determination processing (ST1).
When the subject position determination unit 17 determines that the position determination process is to be performed in step ST1, the subject position determination unit 17 performs detection (search) with reference to the (x, y) width table TBL2 stored in the position search information storage unit 172. The (x, y) width of the range DTRG is determined (ST2).
Then, for example, the subject position determination unit 17 clears the storage area for information on the detected position in the position storage unit 144 of the light source analysis processing unit 14 (ST3).
Next, the subject position determination unit 17 investigates the number of pixels of the target frequency with the (x, y) width of the detection range DTRG (ST4).

The process in step ST4 will be described with reference to FIGS.
The subject position determination unit 17 refers to the (x, y) width table TBL2 and searches for a subject of one light source or reflected light by a method as shown in FIGS.
First, as shown in FIG. 7A, the minimum unit of search is one block BLK.
FIG. 7A shows a 2 × 2 case as an example of a vertical × horizontal search pixel spix.
The detection intensity of one block is a value obtained by dividing the sum of all pixels by 2 × 2.
Here, the search is started.

As shown in FIG. 7B, one horizontal line LLN search process is performed.
In this case, the search block sblk = 2, and the 2 × 2 block is searched for the horizontal line (x direction) LLN.
When the search block sblk is 2, since it is vertical (y) 2 and horizontal (x) 2, the number of detected blocks is counted.
If the block BLK is detected by the blocks indicated by reference numerals 2, 3, and 4, the count value is 3.
In the case of a table in which the number of search blocks sblk is large, the subject position determination unit 17 repeats the process with reference to the next number sblk in the detection range table until the count value becomes 1.
Information on the detected horizontal line LLN is stored in the position search information storage unit 172.
The subject position determination unit 17 searches the entire process for every 2 × 2 blocks in the above process.
Next, one vertical line search process is performed.

As shown in FIG. 7C, one vertical line VLN search process is performed.
The subject position determination unit 17 searches the detection line of the horizontal line LLN stored in the position search information storage unit 172 with a block width indicated by the number of search blocks sblk in the vertical direction.
The search method is the same as the search of the horizontal line LLN. In the case of a table in which the number of search blocks sblk is large, the process is repeated with reference to the next number sblk in the detection range table until the count value becomes 1.
The detected vertical line VLN is stored in the position search information storage unit 172. The subject position determination unit 17 searches the entire process for every 2 × 2 blocks in the above process.
Next, a search process for the detection range DTRG is performed.

As shown in FIG. 7D, the detection process of the detection range DTRG is performed.
A continuous vertical and horizontal block detected by the horizontal line LLN and the vertical line VLN from the above processing is defined as one range.
The total average value of the detection intensities of the blocks in the range is obtained.
Next, the detection range DTRG is determined.
Also in this case, continuous vertical and horizontal blocks that can be detected by the horizontal line LLN and the vertical line VLN from the above processing are set as one range.
The total average value of the detection intensities of the blocks in the range is obtained.
Then, the subject position determination unit 17 determines the detection range [(x, y) − (xe, ye)] DTRG.
Then, fine adjustment is performed within the determination range with the magnification of the fine adjustment parameters sP1 and sP2.
The above processing can be performed simultaneously with a plurality of light sources.

The subject position determination unit 17 performs fine adjustment processing of the detection range DTRG by a method as shown in FIGS.
The subject position determination unit 17 finely adjusts the detection intensity in each determination range, investigates the detection intensity in four directions or eight directions for each detection range, and finally determines the detection intensity in a place with the highest detection intensity.
In this example, the detection intensity on one block is investigated in FIG. 8 (B), the detection intensity on the right of one block is investigated in FIG. 8 (C), and the detection intensity below one block is examined in FIG. 8 (D). Then, in FIG. 8E, the detection intensity on the left of one block is investigated.
Then, in FIG. 8F, the final detection range DTRG is determined at the place where the detection intensity is the highest.
In other words, the place where the subject OBJ is located at substantially the center of the detection range DTRG is determined as the final detection range DTRG.

After examining the number of blocks of the target frequency in step ST4, the subject position determination unit 17 determines whether or not the number of blocks of the target frequency is greater than 0 (ST5).
If it is determined in step ST5 that the number of blocks of the target frequency is greater than 0, the subject position determination unit 17 stores the detected position information obtained by searching the position storage unit 144 of the light source analysis processing unit 14 (ST6).
Then, the number “No” in the table TBL1 of the position storage unit 114 is incremented by 1 (ST7).
The subject position determination unit 17 designates the next detection range DTRG (x, y) (ST8).
If the subject position determination unit 17 determines in step ST5 that the number of blocks of the target frequency is 0, the process proceeds to step ST8 without performing steps ST6 and ST7.
The subject position determination unit 17 repeats the processes of steps ST5 to ST8 until the detection of the target image is completed (ST9).
In step ST8, when the detection of the target image is completed, areas with numbers consecutive in the (x, y) direction are integrated from the detected position information in the position storage unit 144 (ST10), and the number of detected positions is acquired (ST11). .

If the subject position determination unit 17 determines not to perform the position determination process in step ST1, the subject position determination unit 17 proceeds to the process from step ST12 in FIG.
The subject position determination unit 17 acquires the designated number from the position storage unit 144 (ST12).
Here, the subject position determination unit 17 determines whether or not there is a designated number (ST13).
If it is determined in step ST13 that there is a designated number, the subject position determining unit 17 acquires (x, y) width information of the detection range FTRG in the position storage unit 144 from the designated number (ST14), and the subject in the detection range DTRG is acquired. A state is detected (searched) (ST15).
Then, the number “No” in the table TBL1 of the position storage unit 114 is incremented by 1 (ST16).
The subject position determination unit 17 repeats the processes of steps ST14 to ST16 until the detection of the target image is completed (ST9).
If the subject position determination unit 17 determines that there is no designated number in step ST13, the subject position determination unit 17 detects (searches) the subject state of all pixels without performing the processing of steps ST14 to ST17.

Next, other configurations and functions of the detection system according to the present embodiment will be described in detail.
First, the configuration of the CCD of the imaging device 12 according to the present embodiment will be described.

  FIG. 9 is a diagram showing an example for explaining the structure of the CCD according to the present embodiment.

The CCD 20 in FIG. 9 is an interline transfer, and includes a photodiode PD21, a vertical transfer CCD22, a horizontal transfer CCD23, and an amplifier 24.
The photodiodes PD21 are arranged in a matrix. The photodiodes PD21 arranged in the vertical line direction are connected to a vertical transfer CCD 22 for transferring charges for each column. The end of each vertical transfer CCD 22 is connected to a horizontal transfer CCD 23 that transfers charges to the amplifier. An amplifier 24 is connected to the output side of the horizontal transfer CCD 23.

The video scanning method is interlaced, and one screen is interlaced scanning, and is composed of an odd field and an even field.
First, light enters the photodiode PD21, and charges are accumulated in the photodiode PD21 during the charge accumulation time. During this time, the photodiode PD21 and the vertical transfer CCD 22 are disconnected.
When the charge accumulation time ends, the photodiode PD21 and the vertical transfer CCD 22 are brought into conduction, and the accumulated charge is transferred to the vertical transfer CCD 22. Immediately after this, the photodiode PD21 and the vertical transfer CCD 22 are disconnected, and the next charge accumulation is started in the photodiode PD21. The charges transferred to the vertical transfer CCD 22 are transferred to the horizontal transfer CCD 23 for each horizontal line and input to the amplifier 24.
The frequency until charge is transferred from the vertical transfer CCD 22 to the horizontal transfer CCD 23 for each horizontal line is the horizontal scanning frequency of 15.734 Hz of the CCD 20. When all the charges of the vertical transfer CCD 22 are transferred to the horizontal transfer CCD 23, the vertical transfer CCD 22 and the photodiode PD 21 are again brought into conduction, and the charges of the photodiode PD 21 are transferred to the vertical transfer CCD 22. In the case of a field accumulation CCD, charges are accumulated in the photodiode PD21 by photoelectric conversion, and the transfer frequency until this charge is transferred from the photodiode PD21 to the vertical transfer CCD 22 is 59.94 Hz.

  FIG. 10 is a diagram for explaining the time series of the CCD 20 of FIG.

As shown in FIG. 10, the required time until charge is accumulated in the photodiode PD21 by photoelectric conversion is ΔT1, and the required time until the charge is transferred from the photodiode PD21 to the vertical transfer CCD 22 is ΔT2.
As can be seen from FIG. 10, the light energy incident on the CCD 20 is sampled at the charge accumulation period ΔT = ΔT1 + ΔT2 = (1 / 59.94) seconds while being integrated during the charge accumulation time ΔT1.

As shown in FIG. 1, a luminance signal is extracted by a luminance signal extraction unit 13 from a captured image signal captured by the imaging device 12, and the luminance signal is input to the light source analysis processing unit 14.
Here, a method for reading out pixels from the CCD 20 (see FIG. 9) according to the present embodiment will be described.

FIG. 11 is a diagram showing an arrangement example of pixels of a single-plate complementary color filter type CCD.
FIG. 12 is a diagram illustrating an example of a combination of color signals in the odd field OFD and the even field EFD.

The color filter of the pixel is composed of Ye (yellow), Cy (cyan), Mg (magenta), and G (green), and is arranged as shown in FIG. Pixels are read by adding the upper and lower pixels. The combination to be added is shifted by one column between the odd field OFD and the even field EFD. Specifically, in the n-th line of the odd field OFD, (C11 + C21), (C12 + C22), (C13 + C23), (C14 + C24), (C15 + C25),... In the n-line of the even field EFD, (C21 + C31), (C22 + C32), (C23 + C33), (C24 + C34), (C25 + C35),...
Accordingly, color signals are output in the odd field OFD and the even field EFD as shown in FIG.
In any case, the same color pattern of a combination of Ye, Cy, Mg, and G is repeated in a two-pixel cycle.
In other words, the color signal appears superimposed on a frequency of two pixel periods or more. Therefore, if this color signal is passed through a low-pass filter having a cutoff frequency of two pixel periods, the color signal is lost and only a luminance signal is obtained.
Therefore, luminance information is sampled at a cycle of two pixels.

A projection area REG illustrated by a circle in FIG. 11 shows a state in which an image of a subject by a light source is projected. It is assumed that the pixels C35, C36, C45, C46, C55, and C56 are completely in the projection region REG and are uniformly irradiated with light.
In the odd field OFD, the luminance information is read by a combination of C35, C36, C45, and C46 of the horizontal line (n + 1), and in the even field EFD, by a combination of C45, C46, C55, and C56 of the (n + 1) line of the horizontal line. Is done.

  As described above, the luminance signal among the signals from the imaging device 12 is output to the light source analysis processing unit 14. This luminance signal is input to the first calculation unit 141 and the second calculation unit 142 and subjected to predetermined processing.

  Next, a luminance signal processing method performed by the first calculation unit 141 and the second calculation unit 142 will be described with reference to FIG.

  FIG. 13 is a timing chart for explaining the signal processing method of the luminance signal of the light source analysis processing unit according to the present embodiment.

FIG. 13A is a diagram showing interline scanning of the imaging device 12, and shows a state of either the even field EFD or the odd field OFD. FIGS. 13B to 13E respectively show waveforms W1, W2, W3, and W4 representing changes in luminance signal level processed by the light source analysis processing unit 14, and FIG. 13F changes at a constant cycle. It is a figure which shows the waveform W5 of the sine wave to do.
The odd field OFD and even field EFD scan one frame. That is, as shown in FIG. 13A, A and B, C and D are one frame scan.
In the following description, f represents frequency, t represents time, θ represents phase difference, and ω satisfies (ω = 2πf). Note that π is the circumference ratio.

Waveforms W1 and W2 illustrated in FIGS. 13B and 13C are waveforms for deriving functions of waveforms indicated by a waveform W3 and a waveform W4 described later.
A waveform W3 illustrated in FIG. 13D is a time development distribution of a function for obtaining the level difference AC when the luminance level difference between the luminance signal of the A field and the luminance signal of the C field is the level difference AC. Indicates.
A waveform W4 shown in FIG. 13E is a function time for obtaining the level difference BD when the difference in luminance level between the luminance signal of the B field and the luminance signal of the D field is the level difference BD. The development distribution is shown.
Waveform W3 is derived from waveform W1 and waveform W2, and is obtained by adding waveform W1 and waveform W2 and dividing by two. Waveform W4 is derived from waveform W1 and waveform W2, and is obtained by subtracting waveform W1 from waveform W2 and dividing it by 2.

At this time, a time average SAC that is a first time average of the level difference AC is calculated by the first calculation unit 141 shown in FIG. In addition, a time average SBD that is a second time average of the level difference BD is calculated by the second calculation unit 142.
Specifically, the time average SAC is calculated from the luminance level difference AC between the A field and the C field by a combination of C35, C36, C45, and C46.
Similarly, the time average SBD is calculated from the level difference BD between the B field and the D field by a combination of C45, C46, C55, and C56.

The calculation method of the time average is described.
The time average SAC of the level difference AC between the A field and the C field is calculated by multiplying the waveform W1 by the waveform W3 shown in FIG.
Similarly, the time average SBD of the level difference BD between the B field and the D field is shown in FIG. 13 (E) in a waveform representing the temporal change of light irradiated to the pixel by the combination of C45, C46, C55, and C56. The time average SBD is calculated by multiplying the waveform W4 shown.

First, a method for calculating the time average SAC will be specifically described.
The waveform W3 is expressed by a mathematical expression. First, the waveform W1 and the waveform W2 can be expressed by the following Fourier series.

  Here, it is assumed that the waveforms W1 and W2 have the same period f2. From equation (1) and equation (2), waveform W3 can be expressed as equation (4).

  Incidentally, the waveform W5 having the period f1 shown in FIG. 13F can be represented by a sine wave as shown in the equation (5).

  When the waveform W3 represented by the equation (4) is multiplied by the sine wave W5 represented by the equation (5), the equation (7) is obtained.

Next, the time average of the equation (7) from time 0 to time T is taken. Of the terms shown on the right side of the equation (7), the term including time t is an AC signal, so the time average is zero.
Therefore, only when (ω ₁ − (2n−1) ω ₂ = 0), the constant cos θ ₁ and the constant sin θ ₁ remain, and the time average SAC is expressed by the equation (8).

  In this way, the time average SAC is obtained by the first calculation unit 141. Similarly, the time average SBD is obtained by the second calculation unit 142 and is expressed by equation (9).

Now, the sum of squares (S _AC ² + S _BD ² ) of the time average SAC and SBD expressed by the equations (8) and (9) is expressed by the equation (10).

From this equation (10), when the frequency component (f ₁ = (2n−1) f ₂ ) is included in the light incident on the CCD 20 (see FIG. 9), it is expressed by the equation (10). Waveform components are detected.

Next, frequency components included in the light source 11 will be considered.
FIG. 14 is a diagram showing a waveform W6 for frequency components included in the light source 11. As shown in FIG. The light source 11 emits light at the luminance level L1 at the frequency f3 for the time τ.

This waveform W6 is developed into a Fourier series. The waveform W6 is a periodic function of a period (T ₃ = 1 / f ₃ ). When (ω ₃ = 2πf ₃ ), the waveform W6 is represented by a general formula of Fourier series as shown in Expression (11).

The coefficients a ₀ , a _n , and b _{n in the} equation (11) are obtained from the waveform W6 as in the equations (12) to (14).

  Therefore, the Fourier series of the waveform W6 is expressed by equation (15).

Therefore, when the blinking period of the light source 11 is set to four times the field period, that is, when (f ₃ = f ₂ ), the frequency coincides with the odd term from the expressions (7) and (15), and the time average SAC And the sum of the squares of SBD is given by equation (16).

  When the duty ratio of lighting of the light source 11 is D, it is expressed by equation (17).

    Therefore, the square sum SACBD of the time average SAC and SBD expressed by the equation (16) is expressed by the equation (18) when the equation (17) is used.

  By the way, the term (19) on the right side of the following equation (18) converges.

  The term (19) on the right side of the equation (18) takes a value as shown in Table 1 with respect to the duty ratio D. Table 1 is shown below.

  Based on Table 1, when the duty ratio D is taken on the horizontal axis and the square sum SACBD of the time average SAC and SBD is taken on the vertical axis, the relationship between the duty ratio D and the square sum SACBD is as shown in FIG. become.

From FIG. 15, it can be seen that the square sum SACBD is maximized at the duty ratio D = 0.5.
Therefore, the sum of squares SACBD expressed by the equation (18) is expressed by the following equation.

[Equation 15]
S _AC ² + S _BD ² = 0.08333L ₁ ² (20)

  As shown in the equation (20), the arithmetic processing unit 133 detects the luminance of the light source 11 (see FIG. 1) and outputs the detection result (square sum SACBD) to the determination unit 16 via the filter processing unit 15. To do. This detection result is used for determining the state of the subject by the determination unit 16.

The light source 11 according to the present embodiment does not depend on a specific light source. Therefore, it is considered whether the luminance can be detected for other light sources.
Incandescent light bulbs and fluorescent lamps that are widely used as light sources are 100 Hz in regions where the power supply frequency is 50 Hz and 120 Hz in regions where 60 Hz. The field frequency of an NTSC television is 59.94 Hz, and the field frequency of a monitor used for a personal computer is 60 Hz or more so as not to flicker.

  The field frequency of the NTSC television is 59.94 Hz, and if the light source is caused to emit light at a quarter period, the frequency f2 of the luminance level difference is as follows.

[Equation 16]
f ₂ = 59.94 / 4 = 14.985 Hz (21)

  From the equations (7) and (15), the signal component is detected when the odd multiple of the frequency f2 matches the integer multiple of the frequency f3 of the light source 11.

  Table 2 is a table of values indicating the relationship between the light emission frequency of different light sources and the frequency in the difference in luminance signal level.

F1 in Table 2 is the frequency of the sine wave of Formula (5), f2 is the frequency shown in Formula (21), and f3 is the frequency of the light source 11, the frequency of illumination in the 50 Hz region, and the illumination in the 60 Hz region, respectively. Frequency, NTSC television field frequency, and monitor field frequency used in personal computers.
According to Table 2, up to m = 30, f3 is 100, 120, 59.94 Hz, and there is nothing that satisfies (n × f ₃ = (2m−1) × f ₂ ).

For example, regarding a monitor of a personal computer, if the field frequency is 60 Hz or more, there is a monitor that is scanned at 74.925 Hz as a possibility that the largest output is detected in the signal processing output of the detection system 10. When
That is, when f ₃ = 74.925 Hz, as shown in Table 2, (5 × f ₂ ), (15 × f ₂ ), (25 × f ₂ ), and (1 × f ₃ ), ( 3 × f ₃ ), (5 × f ₃ ). The signal level detected at this time is expressed by the following equation.

したがって、（２２）式で示される信号レベルは光源１１の１／２５のレベルであり、図１に図示していない信号処理で別に除去できる。

Therefore, the signal level expressed by the equation (22) is 1/25 of that of the light source 11 and can be removed separately by signal processing not shown in FIG.

  As described above, the detection system 10 detects the state of the light source or the subject irradiated on the light source without depending on the light emission frequency of the light source.

  Hereinafter, a series of operations of the detection system according to the present embodiment will be described with reference to FIG.

  FIG. 16 is a flowchart for explaining an outline of a series of operations of the detection system according to the present embodiment.

In the present embodiment, first, the luminance of the light source 11 within the charge accumulation time of the imaging device 12 is changed by 4n times the field period of the imaging device 12 (ST11).
Next, a luminance signal is acquired by the luminance signal extraction unit 13 every n fields in the field unit from the imaging device 12 (ST12), and this luminance signal is output to the first calculation unit 141 and the second calculation unit 142.
The first arithmetic unit 141 obtains the time average SAC of the level difference AC of the luminance signal level in the projection area REG of the mth and (m + 2) th fields. Further, the second arithmetic unit 142 obtains the time average SBD of the level difference BD of the luminance signal level in the projection area REG of the (m + 1) th and (m + 3) th fields (ST13).
These time averages SAC and SBD are output to the arithmetic processing unit 143.
Next, the arithmetic processing unit 133 obtains the square sum SACBD of the time average SAC and SBD (ST14), and outputs the result to the determination unit 16 via the filter processing unit 15.
The determination unit 16 determines the state of the subject according to the value of the square sum SACBD (ST15). The determination result is supplied to the subject position determination unit 17.
The subject position determination unit 17 determines whether or not to determine the subject position (ST16).
Here, when determining the subject position, the subject position determination unit 17 detects the state of the subject and acquires information about the detection position (ST17), and the detected position information is stored in the position storage unit 144 of the light source analysis processing unit 14. Remembered.
When the subject position is not determined, the subject position determination unit 17 acquires the specified position (specified number) from the position information stored in the position storage unit 144 (ST18), and detects the state of the subject at the specified position. (ST19).
Then, when performing the step ST13 and ST14 in the light source analyzing unit 14, based on the detection position coordinates or the like stored in the position storage unit 144, one or more images of test knowledge range designated by the coordinates Analysis (detection) is performed.
By this processing of the light source analysis processing unit 14, the present detection system 10 can detect only the position of the light source or the subject irradiated with the light source, and does not detect the position excluded as the subject. It is effective and can be detected with high accuracy.

In the arithmetic processing unit 143 according to the present embodiment, the square sum SACBD of the time average SAC and SBD is obtained, and the sum of the time average SAC and SBD (S _AC + S _BD ) is determined by the determination unit 16. It can also be used.

  As described above, according to the present embodiment, the background noise of the captured image can be removed, and the state of the subject irradiated with the light source or the light source can be detected.

According to the detection system according to the present embodiment, in the detection of the light source or the state of the subject irradiated to the light source, detection when there are a plurality of subjects can be performed for each subject, and tracking and capturing of the subject can be performed. By doing so, there is no need to detect the position of an unnecessary object that is not a subject, and the effect of detecting the state of the subject at high speed and with high accuracy can be obtained.
Moreover, since it does not depend on the light source, it does not depend on the specifications of the imaging device, and for example, a generally available camera can be used as the imaging device of the present detection system.

Furthermore, this detection system uses a plurality of light sources and can transmit signals to the signal processing unit in parallel.
Alternatively, it is possible to provide a plurality of light source colors and perform wavelength multiplexing transmission of signals.
It is also possible to process the signal by appropriately blinking the light source.

[Second Embodiment]
Next, a second embodiment according to the present invention will be described.

In the second embodiment, the field accumulation / interlace imaging device 12 according to the first embodiment is replaced with a frame accumulation / interlace imaging device. At the same time, the luminance change period of the light source 11 according to the first embodiment is changed from 4n times the field period to 4n times the frame period. Accompanying this change in the change period, the luminance signal acquisition period is changed from every n fields to every 2n fields.
As described above, by changing the luminance change cycle of the light source 11 and the luminance signal acquisition cycle, the same effects as those of the first embodiment can be obtained.
Therefore, according to the present embodiment, the background noise of the captured image can be removed, the state of the light source or the subject irradiated with the light source can be detected, and even when there are a plurality of subjects, detection can be performed for each subject.

[Third Embodiment]
Next, a third embodiment according to the present invention will be described.

In the third embodiment, the field accumulation / interlace scanning imaging device 12 according to the first embodiment is replaced with a frame accumulation / non-interlace scanning imaging device.
As described above, even when a non-interlaced scanning imaging apparatus is used, the same effect as that of the first embodiment can be obtained.
Therefore, according to the present embodiment, the background noise of the captured image can be removed, the state of the light source or the subject irradiated with the light source can be detected, and even when there are a plurality of subjects, detection can be performed for each subject.

Note that the method described above in detail can be formed as a program according to the above-described procedure and executed by a computer such as a CPU.
Further, such a program can be configured to be accessed by a recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a floppy (registered trademark) disk, or the like, and to execute the program by a computer in which the recording medium is set.

  DESCRIPTION OF SYMBOLS 10 ... Detection system, 11 ... Light source, 12 ... Imaging device, 13 ... Luminance signal extraction part, 14 ... Light source analysis process part, 141 ... 1st calculating part (A) , 142... Second calculation unit (B), 143... Calculation processing unit, 144... Position storage unit, 15... Filter processing unit, 16. Unit, 171 ... position search processing unit, 172 ... position search information storage unit.

Claims (9)

A light source;
An imaging device for imaging the light source or a subject illuminated by the light source;
An analysis processing unit that performs analysis processing on a signal acquired from the imaging device;
A determination unit that detects and determines the state of the light source or subject based on the analysis result of the analysis processing unit;
A position storage unit capable of storing information on positions including detection ranges in images of a plurality of subjects;
From the detection information obtained by the determination unit, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is acquired, A subject position determination unit capable of storing detection information as information about the position in the position storage unit,
The analysis processing unit
The signal is acquired from the imaging device every predetermined scanning plane period, a time average of the signal level difference is obtained from a signal level difference of the signal between a plurality of different scanning planes, and further, based on the value of the time average Perform an analysis process to perform a predetermined calculation, and output the calculation result of the analysis process to the determination unit,
Detection system when the detection information is stored in the position storage unit for selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information. The analysis processing unit
When the detection information is not stored in the position storage unit or in the all-image detection mode, the analysis processing is performed on the entire image,
When the detection information to the position storage section is stored, and when the subject position detection mode, according to claim 1, wherein performing the analysis processing for the image of the test knowledge range specified in the detection information detection system . The subject position determination unit
In the determination of the detection range, the size of the subject is measured, and a table whose size exceeds the current coordinate (x, y) width or below the coordinate (x, y) width is prepared beforehand. The detection system according to claim 1 or 2, wherein a detection range of values is selected. The subject position determination unit
After searching all the lines in the x and y directions for every predetermined number of search blocks, the detection range search process is performed, and the continuous vertical and horizontal blocks detected by the line search are regarded as one range. The detection system according to claim 3, wherein the detection range is determined by obtaining a total average value of detection intensities of the blocks. The subject position determination unit
The detection system according to claim 3 or 4, wherein the determined detection range can be finely adjusted. The analysis processing unit
A first calculation unit for obtaining a first time average of the signal level difference in the m (m = 1, 2, 3,...) And (m + 2) -th field in the same region;
A second computing unit for obtaining a second time average of the signal level difference in the (m + 1) th and (m + 3) th fields in the same region;
The detection system according to claim 1, further comprising: an arithmetic processing unit that calculates a square sum of the first time average and the second time average. The analysis processing unit
A first calculation unit for obtaining a first time average of the signal level difference in the m (m = 1, 2, 3,...) And (m + 2) th frames in the same region;
A second calculation unit for obtaining a second time average of the signal level difference in the (m + 1) th and (m + 3) th frames in the same region;
The detection system according to claim 1, further comprising: an arithmetic processing unit that calculates a square sum of the first time average and the second time average. An imaging step of imaging with an imaging device a light source or a subject illuminated by the light source;
A signal is acquired from the imaging device every predetermined scanning plane period, a time average of the signal level difference is obtained from a signal level difference of the signal between a plurality of different scanning planes, and a predetermined average is obtained based on the time average value. An analysis process step for performing an analysis process for performing an operation;
A determination step of detecting and determining the state of the light source or subject based on the analysis result of the analysis processing step;
From the detection information obtained in the determination step, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is acquired, Subject position determination step for storing detection information in the position storage unit as information on the position,
In the analysis processing step,
Signal processing method of the detection system when the detection information is stored in the position storage unit, including the step of selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information. An imaging process in which an imaging device captures an image of a light source or a subject illuminated by the light source;
A signal is acquired every predetermined scanning plane period, a time average of the signal level difference is obtained from a signal level difference of the signal between a plurality of different scanning planes, and a predetermined calculation is performed based on the time average value. Analysis processing,
A determination process for detecting and determining the state of the light source or subject based on the analysis result of the analysis process;
From the detection information obtained in the determination process, the position and size of the subject in all the images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is acquired. Subject position determination processing for storing detection information in the position storage unit as information on the position,
In the above analysis process,
A computer signal processing of the detection system including processing when the detection information is stored in the position storage unit for selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information The program to be executed.

Images