JP5349413B2 - Detection system, signal processing method thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a detection system, a signal processing method thereof and a program capable of detecting each subject even if plural subjects exist. <P>SOLUTION: A detection system comprises: a position storage unit 144 which can store information on positions including detection areas of plural subjects in an image; a subject position determination unit 17 which searches the positions and sizes of the subjects in the whole image from detection information obtained by a determination unit 16 to determine the positions of the subjects including the detection areas and can store the detection information including the detection areas about the subjects in the position storage unit; and a jamming action determination unit 18 for performing at least one kind of detection of positional displacement detection of an imaging device based on the state of detection area positions obtained by the subject position determination unit, subject no-response detection, and subject successive response detection. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a detection system that detects the state of a subject using an imaging device or the like, a signal processing method thereof, and a program, for example.

For example, an imaging apparatus used in a nighttime crime prevention system shown in Patent Document 1 has been proposed.
This imaging apparatus has a signal processing unit, and the light source has a frequency higher than 100 Hz or 120 Hz (when a commercial power source of 50 Hz or 60 Hz is used as the power source, the brightness of the light source fluctuates at 100 Hz or 120 Hz that is twice that). Modulate with high frequency. Further, the signal processing unit has a detection unit for detecting the high frequency modulation signal.
However, this imaging apparatus needs to have a frame rate higher than its modulation frequency.

By the way, a standard called NTSC (National Television Standards Committee) system or PAL (Phase Alternating Line) system is adopted for imaging apparatuses which are widely spread in general.
Since an image pickup apparatus adopting such a standard has a low frame rate, for example, it cannot be used as an image pickup apparatus for a crime prevention system as shown in Patent Document 1.

Therefore, a detection system that can detect a light source or the state of a subject irradiated with the light source and can be applied to a crime prevention system adopting a standard such as NTSC has been proposed (see Patent Document 2).
This detection system detects the state of the subject of the frequency component of the light source.

Japanese Patent No. 3019309 JP 2008-141251 A

However, in the above-described detection system, when there are a plurality of subjects detected from the video captured by the imaging device, it is difficult to determine the position of each subject and its detection information.
Further, in the detection system, it is difficult to measure the information of the detected position range with time change.
In the above detection system, it is not easy to detect mischief or obstruction.

Conventionally, as a method of detecting a disturbing action, a method of detecting by comparison based on a background difference and a detecting method of a disturbing action that is detected by contrast of an image are known.
In addition, it is known to detect an interference action by determining no signal by cutting a cable of an output signal.

When there is a disturbing action of the captured image by spraying or blindfolding on the lens of the image capturing apparatus by these methods, there is a disadvantage that there are many false detections because the change from the difference image is not constant.
Also, in mischievous actions such as shifting the position of the imaging device, it is difficult to detect when there are many moving objects in the background or in a place with large light and darkness, and high accuracy detection even in judgments based on natural background changes such as shaking of trees Is difficult.

  The present invention is capable of detecting the state of a light source or a subject irradiated with the light source, as well as being able to detect each subject even when there are a plurality of subjects, and capable of detecting a disturbing action. A system, a signal processing method thereof, and a program are provided.

A detection system according to a first aspect of the present invention includes a light source, an imaging device that images the light source or a subject irradiated with the light source, an analysis processing unit that performs an analysis process on a signal acquired from the imaging device, A determination unit that detects and determines the state of the light source or the subject based on an analysis result of the analysis processing unit, a position storage unit that can store information about positions including detection ranges in the images of a plurality of subjects, and the determination unit From the obtained detection information, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, the detection information including the detection range related to the subject is acquired, and the detection information is and the object position determination unit that can be stored in the position storage unit as the information regarding the position of the imaging apparatus according to the state of the position of the detection range obtained by the object position determination unit A disturbing action determination unit that performs at least one detection determination among detection detection, subject non-response detection, and subject continuous response detection, and the analysis processing unit performs the above-described analysis processing unit from the imaging device every predetermined scanning plane period Obtaining a signal, obtaining a time average of the signal level difference from the signal level difference of the signal between a plurality of different scanning planes, and performing an analysis process for performing a predetermined calculation based on the value of the time average, the calculation result of the analysis is output to the determination unit, when the detection information is stored in the position storage unit selectively the analysis processing for the image of the test knowledge range specified in the above detection information I do.

A signal processing method of a detection system according to a second aspect of the present invention includes an imaging step of imaging a light source or a subject irradiated with the light source with an imaging device, a signal acquired from the imaging device every predetermined scanning plane period, and a plurality of signals An analysis processing step of performing an analysis process for obtaining a time average of the signal level difference from the signal level difference of the signal between different scanning planes, and further performing a predetermined calculation based on the value of the time average; A determination step for detecting and determining the state of the light source or subject based on the analysis result of the step, and a detection range including a detection range by searching for the position and size of the subject in all images from the detection information obtained in the determination step A subject that determines the position of the subject, acquires detection information including a detection range related to the subject, and stores the detection information in the position storage unit as information about the position A position determining step, positional deviation detection image pickup device according to the state of the position of the detection range obtained by the object position determination step, the subject no response detection, and sabotage determination to perform at least one of the detection determining of the subject continuous response detection has a step, and in the analysis processing step, when the detection information is stored in the position storage unit selectively the analysis processing for the image of the test knowledge range specified in the above detection information The step of performing is included.

According to a third aspect of the present invention, an image pickup process for picking up a light source or a subject irradiated with the light source with an image pickup device, a signal is acquired for each predetermined scan plane period, and the signal of the signal is obtained between a plurality of different scan planes. Calculates the time average of the signal level difference from the level difference, and further performs analysis processing for performing a predetermined calculation based on the value of the time average, and detects and determines the state of the light source or subject based on the analysis result of the analysis processing And the detection information obtained by the determination process, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is obtained. obtained by the object position determination process stored in the position storage unit the detection information as information related to the position, shooting by the state of the position of the detection range obtained by the object position determination process Interfering action determination processing for performing at least one detection determination among device position shift detection, subject non-response detection, and subject continuous response detection. In the analysis processing, the detection information is stored in the position storage unit. when stored in a program for executing the signal processing of the detection system including a process for selectively above analysis process to the image of the test knowledge range specified in the detection information to the computer.

  According to the present invention, not only can the state of a light source or a subject irradiated on the light source be detected, but also a plurality of subjects can be detected for each subject, and a disturbing action can be detected.

It is a figure which shows the structural example of the detection system which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the information preserve | saved in the position memory | storage part which concerns on this embodiment. It is a figure which shows notionally the state before the position determination of the to-be-photographed object in the to-be-photographed object position determination part which concerns on this embodiment, and the state after position determination. It is a figure which shows an example of the (x, y) width table information of the detection range referred in order to perform position determination in the to-be-photographed object position determination part which concerns on this embodiment. It is a flowchart for demonstrating specifically the detection position determination process in the to-be-photographed object position determination part which concerns on this embodiment, Comprising: It is a figure which shows the process in the case of performing a position determination process. It is a flowchart for demonstrating specifically the detection position determination process in the to-be-photographed object position determination part which concerns on this embodiment, Comprising: It is a figure which shows a process when not performing a position determination process. It is a figure for demonstrating the search method of the object of one light source or reflected light. It is a figure for demonstrating the fine adjustment process of a detection range. It is a figure for demonstrating the position shift determination which concerns on this embodiment. It is a figure which shows the non-occurrence | occurrence | production and generation | occurrence | production of the position shift using a detection range and a capture range with a waveform. It is a figure which shows the detection waveform corresponding to the state transition of a position shift. It is a figure which shows the detection waveform corresponding to the state transition at the time of interruption | blocking generation | occurrence | production by a shield. It is a figure for demonstrating the continuous abnormal response determination which concerns on this embodiment. In this embodiment, it is a figure which shows the non-occurrence | occurrence | production and generation | occurrence | production of the position shift using a detection range and a capture range, and a continuous abnormal response detection with a waveform. In this embodiment, it is a figure which shows the non-occurrence | occurrence | production and generation | occurrence | production of the position shift using a detection range and a capture range, and the detection state of a continuous abnormal response detection as a table | surface. It is a figure which shows an example for demonstrating the structure of CCD which concerns on this embodiment. It is a figure for demonstrating the time series of CCD of FIG. It is a figure which shows the example of an arrangement | sequence of the pixel of a single plate complementary color filter type CCD. It is a figure which shows an example of the combination of the color signal in odd field OFD and even field EFD. It is a timing chart for demonstrating the signal processing method of the luminance signal of the light source analysis process part which concerns on this embodiment. It is a figure which shows the waveform W6 about the frequency component contained in a light source. It is a figure which shows an example of the relationship between duty ratio D and square sum SACBD. It is a flowchart figure for demonstrating a series of operation | movement outline | summary of the detection system which concerns on this embodiment. It is a figure which shows the structural example of the detection system which concerns on the 2nd Embodiment of this invention. It is a figure which shows the structural example of the detection system which concerns on the 3rd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example of a detection system according to the first embodiment of the present invention.

The detection system 10 includes a light source 11, an imaging device 12, a luminance signal extraction unit 13, a light source analysis processing unit 14, a filter processing unit 15, a determination unit 16, a subject position determination unit 17, and an obstruction act determination unit 18.
The light source analysis processing unit 14 includes a first calculation unit (A) 141, a second calculation unit (B) 142, a calculation processing unit (time average square sum calculation processing unit) 143, a position storage unit 144, and a capture position. A storage unit 145 is included.
The subject position determination unit 17 includes a position search processing unit 171 and a position search information storage unit 172.

  The light source 11 illuminates the subject with a predetermined luminance. The luminance of the light source 11 is variable, and the luminance within the charge accumulation time of the imaging element included in the imaging device 12 changes at a period 4n times the field period of the imaging device 12. Here, n = 1, 2, 3,.

The imaging device 12 used in the detection system 10 according to the present embodiment employs an imaging device having the following specifications.
As an example of the imaging device constituting the imaging apparatus 12, a single-plate complementary color filter and a field storage type interline transfer CCD (Charge Coupled Device) image sensor (hereinafter simply referred to as a CCD) are used.
As an example, the television system of the imaging device 12 employs the NTSC system, the scanning system employs interlace, the scanning frequency is 15.734 KHz, and the vertical frequency is 59.94 Hz.
The imaging device 12 having such a configuration images the subject illuminated by the light source 11 and outputs a signal obtained by the imaging to the luminance signal extraction unit 13.
Although a CCD image sensor is illustrated here, a CMOS image sensor can also be applied.

The luminance signal extraction unit 13 extracts a luminance signal from the input signal and outputs the luminance signal to the first calculation unit 141 and the second calculation unit 142 of the light source analysis processing unit 14.
The luminance signal extracted by the luminance signal extraction unit 13 is adjusted to a signal level optimized for calculation.
The signal level needs to be a signal level that does not overflow in the output values of the first calculation unit 141, the second calculation unit 142, and the calculation processing unit 143. Therefore, the luminance signal extraction unit 13 includes a circuit that adjusts the luminance signal level.
The adjustment value of the luminance signal level may have a table capable of mode switching when there are several modes. The mode may be a video signal standard such as NTSC or PAL, or a frequency mode of the imaging apparatus.

In the light source analysis processing unit 14, the first calculation unit 141 obtains the time average of the level difference of the luminance signal in the m-th and (m + 2) -th fields in the same area of the imaging device.
The output result A of the first calculation unit 141 is output to the calculation processing unit 143.
The second calculation unit 142 obtains a time average of the level difference of the luminance signal in the (m + 1) th and (m + 3) th fields in the same region of the input luminance signal.
The output result B of the second calculation unit 142 is output to the calculation processing unit 143.
The details of the operations of the first calculation unit 141 and the second calculation unit 142 will be described later.
The first calculation unit 141 and the second calculation unit 142 perform calculation processing in accordance with calculation range designation information for analyzing the image stored in the position storage unit 144.

The output result A and the output result B output from the first calculation unit 141 and the second calculation unit 142, respectively, are input to the calculation processing unit 143.
The arithmetic processing unit 143 obtains a square sum value (A ² + B ² ) of the output result and outputs it to the filter processing unit 15.
The arithmetic processing unit 143 performs arithmetic processing according to the calculation range designation information for analyzing the image stored in the position storage unit 144.

The position storage unit 144 stores, for example, detection information on the positions and sizes of a plurality of subjects detected with high accuracy by removing unnecessary moving objects from the signal component of the light source, which is determined by the subject position determination unit 17.
As described above, the light source analysis processing unit 14 includes the position storage unit 144 that can specify a calculation range for image analysis of the first calculation unit 141 and the second calculation unit 142.
The position storage unit 144 can store a plurality (n) of pieces of position information, and the capture range position storage unit 145 has a capacity for storing a plurality (n) of pairs in the position storage unit 144.
Therefore, the detection process of the light source analysis processing unit 14 is performed in the detection range of the image specified by the position storage unit 144 (for example, the range Mn = [x1, y1−x2, y2]) in the detection of the light source or the object irradiated to the light source. n analysis processes are performed within one frame.
Alternatively, if n analysis processes cannot be performed within the time of one frame, the analysis process in one frame is performed in one n range, and after the completion, the analysis in the second range from the next frame is performed. A method of performing processing in order may also be used.

The captured position storage unit 145 is paired with an analysis process based on the stored information of the position storage unit 144, and MHn with respect to an image detection range (Mn = [x1, y1-x2, y2]) designated by the position storage unit 144. Stores information for performing n analysis processes in the range of = [xh1, yh1-xh2, yh2] in one frame that is the same as the process in the range of the position storage unit 144.
The capture range position of the capture position storage unit 145 is a range of x1 ≧ xh1, y1 ≧ yh1, x2 ≦ xh2, y2 ≦ yh2, and the determination of the capture range position is based on the coefficient by referring to the value of the position storage unit 144 A method of calculating automatically or determining each image range in advance may be used.
An image in the n designated ranges obtained by the analysis process can be acquired as a detected image if there is a light source or a subject irradiated with the light source.
Since the detection range is not specified in the initial state in which position information or the like is not stored in the position storage unit 144, the light source analysis processing unit 14 performs light source analysis processing on the entire image instead of selective detection processing.

Further, when the position storage unit 144 is a nonvolatile memory or a setting file, the detection position can be restored without being affected by a power failure. In addition, it has a flag for selecting whether or not to determine the subject position.
A part of the position storage unit 144 has a volatile memory and can be used for temporary storage.
The position storage unit 144 has a function of storing information such as the position coordinates and size of the subject, the frequency detection sensitivity (signal level) of the subject, and the individual number of the subject.

  FIG. 2 is a diagram illustrating an example of an information table stored in the position storage unit 144 according to the present embodiment.

In FIG. 2, “No” indicates the individual number of the subject. “X, y” indicates the start position coordinates of the subject, and “xe, ye” indicates the end position coordinates of the subject. “N” indicates a number corresponding to the number of subjects. “Level” indicates a signal level (signal intensity) which is frequency sensitivity of the subject.
In this example of the table TBL1, the subject of “No. 1” is the first subject with the start position coordinates “10, 10” and the end position coordinates “20, 20”, and the signal level is relatively high. “40”.
The subject of “No. 2” is the second subject whose start position coordinates are “30, 30” and end position coordinates are “50, 60”, and its signal level is relatively “43”.
The “No. 3” subject is the third subject whose start position coordinates are “100, 100” and end position coordinates are “120, 120”, and its signal level is relatively “50”.
The subject of “No. 4” is the fourth subject having a start position coordinate of “300, 300” and an end position coordinate of “325, 315”, and its signal level is relatively “60”.

As described above, the light source analysis processing unit 14 of the present embodiment includes the position storage unit 144 that is a memory that can store information on the positions of a plurality of subjects, and the position storage unit 144 has coordinates of an image area that indicates the subject position. It has subject detection information such as value and subject number.
Source analyzing unit 14, when the detection position coordinates stored in the position storage section 144 is present, the detection position of the image of the test knowledge range designated by the coordinates are stored in the acquisition position storage section 145 It has a function of analyzing (detecting) including a larger size capture range.
The subject position determination unit 17 stores the detection information in the position storage unit 144.

The light source analysis processing unit 14 is, for example, two modes that are designated by a control system (not shown) or automatically switched according to the presence or absence of detection information in the position storage unit 144, specifically, an all-pixel detection mode and a subject position detection mode. It is possible to operate with.
In the all-pixel detection mode, the light source analysis processing unit 14 does not specify a detection range in the initial state where position information or the like is not stored in the position storage unit 144. Perform analysis processing.
Based on the detection result in the all-pixel detection mode, the subject position determination unit 17 described later detects information regarding the positions of a plurality of subjects, and the detection information is stored in the position storage unit 144.
For example, when this detection information is stored in the position storage unit 144, the light source analysis processing unit 14 switches from the all-pixel detection mode to the subject position detection mode.
The subject position detection mode, the light source analyzing unit 14, based on the detection position coordinates or the like stored in the position storage unit 144, analyzes the image of the test knowledge range designated by the coordinates (detection).
Since the light source analysis processing unit 14 is provided, the detection system 10 can detect only the position of the light source or the subject irradiated with the light source, and does not detect the position excluded as the subject. Is also effective and can be detected with high accuracy.

In the initial state described above, the filter processing unit 15 performs noise reduction on the entire image by filter processing and outputs the noise to the determination unit 16.
The filter processing unit 15 performs noise reduction on the n detected images in the light source analysis processing unit 14 or eliminates a portion that has not been detected by the filter processing, and outputs the result to the determination unit 16.

In the above-described initial state, the determination unit 16 is output from the arithmetic processing unit 143 and based on the square sum value (A ² + B ² ) of the entire image that has been reduced in noise by the filter processing unit 15. For example, it is determined whether the subject imaged is stationary or moving. Details of the operation of the determination unit 16 will be described later.
Based on the square sum value (A ² + B ² ) of the detected image that is output from the arithmetic processing unit 143 and from which the filter processing unit 15 has eliminated noise and has not been detected, the determination unit 16 For example, it is determined whether the subject imaged at 12 is stationary or moving. Details of the operation of the determination unit 16 will be described later.

  The subject position determination unit 17 stores the position search processing unit 171 that searches for the position of the subject from all the images from the detection state obtained by the determination unit 16 and the position search result of the position search processing unit 171. A search information storage unit 172 is included.

FIGS. 3A and 3B are diagrams conceptually showing states before and after the position of a subject in the subject position determination unit according to the present embodiment.
As shown in FIG. 3A, the subjects OBJ1 and OBJ2 before the position determination are searched, and as shown in FIG. The position is determined, and the range becomes the analysis target of the light source analysis processing unit 14 and the detection target of the determination unit 16.
Detection information such as the range after the position determination is stored in the position storage unit 144 of the light source analysis processing unit 14. In the present embodiment, as described above, there are a subject position detection mode in which only the detection of the range is possible and an all image detection mode for all images.
For example, in an application where it is necessary to detect only a subject at a specific position, the operation is performed in the subject position detection mode in the range stored in the position storage unit 144.
In that case, as described above, in order to set a specific position, the subject position determination unit 17 first searches the subject target position and stores it in the position storage unit 144, and thereafter only the detection of the stored position is performed. Applicable for use.

The subject position determination unit 17 determines the detection range from the size of the subject.
In the subject position determination unit 17, the detection range is determined using the first mode when the (x, y) width of the detection range DTRG is fixed to one and the (x, y) width according to the size of the subject. Second, a table value detection range DTRG having a size exceeding (or below) the current (x, y) width is selected from a plurality of tables prepared in advance with a (x, y) width. There are two modes:
The (x, y) width table of the detection range DTRG is stored in the position search information storage unit 172, for example.
The subject position determination unit 17 can select either the first mode or the second mode, and has a configuration having these functions.
This detection range is determined from a table in which the size of the subject is prepared in advance and the detection range is determined, so that the size and position of the subject can be detected at high speed.

  FIG. 4 is a diagram illustrating an example of (x, y) width table information of the detection range referred to for position determination in the subject position determination unit 17 according to the present embodiment.

In FIG. 4, “No” indicates the number of (x, y) width information of the subject detection range. “X, y” indicates the size information of the object detection range DTRG, and sBlK indicates the number of search blocks (number of pixels).
In the example of the (x, y) width table TBL2, the detection range DTRG of “No. 1” is “10 × 10” and the number of blocks is “2”.
The detection range DTRG of “No. 2” is “30 × 30” and the number of blocks is “4”.
The detection range DTRG of “No. 3” is “100 × 100” and the number of blocks is “8”.
The detection range DTRG of “No. 4” is “300 × 300” and the number of blocks is “12”.
The position search processing unit 171 of the subject position determination unit 17 refers to the (x, y) width table TBL2, and searches, for example, in ascending order from the smaller “No” or in descending order from the larger “No”. The detection range most suitable for the target subject is determined.

The subject position determination unit 17 is configured to be able to finely adjust the detection range DTRG from the size of the subject.
For fine adjustment of the detection range, from the state where the detection range DTRG (x, y) is determined, the fine adjustment value table prepared in advance or the fine adjustment formula [x1 = x * sP1, y1 = y * sP2] x, y) The configuration has a function of finely adjusting the width.
Here, “sP1” and “sP2” are adjustment parameters and take values of, for example, 1 <sP1, sP2 <2.
With this function, even if a small shake or movement occurs in a subject that can be detected within the normal detection range, it can be detected without departing from the subject range.
This is particularly effective when the subject is a light source.

Next, the detection position determination process in the subject position determination unit 17 according to the present embodiment will be specifically described with reference to FIGS.
5 and 6 are flowcharts for specifically explaining the detection position determination process in the subject position determination unit 17 according to the present embodiment, and FIG. 5 is a flowchart in the case of performing the position determination process. FIG. 6 is a flowchart when the position determination process is not performed.
FIG. 7 is a diagram for explaining a method of searching for a subject of one light source or reflected light.
FIG. 8 is a diagram for explaining detection range fine adjustment processing.
The following processing is performed mainly by the position search processing unit 171. Here, the processing will be described as processing by the subject position determination unit 17.

The subject position determination unit 17 determines whether or not to perform position determination processing (ST1).
When the subject position determination unit 17 determines that the position determination process is to be performed in step ST1, the subject position determination unit 17 performs detection (search) with reference to the (x, y) width table TBL2 stored in the position search information storage unit 172. The (x, y) width of the range DTRG is determined (ST2).
Then, for example, the subject position determination unit 17 clears the storage area for information on the detected position in the position storage unit 144 of the light source analysis processing unit 14 (ST3).
Next, the subject position determination unit 17 investigates the number of pixels of the target frequency with the (x, y) width of the detection range DTRG (ST4).

The process in step ST4 will be described with reference to FIGS.
The subject position determination unit 17 refers to the (x, y) width table TBL2 and searches for a subject of one light source or reflected light by a method as shown in FIGS.
First, as shown in FIG. 7A, the minimum unit of search is one block BLK.
FIG. 7A shows a 2 × 2 case as an example of a vertical × horizontal search pixel spix.
The detection intensity of one block is a value obtained by dividing the sum of all pixels by 2 × 2.
Here, the search is started.

As shown in FIG. 7B, one horizontal line LLN search process is performed.
In this case, the search block sblk = 2, and the 2 × 2 block is searched for the horizontal line (x direction) LLN.
When the search block sblk is 2, since it is vertical (y) 2 and horizontal (x) 2, the number of detected blocks is counted.
If the block BLK is detected by the blocks indicated by reference numerals 2, 3, and 4, the count value is 3.
In the case of a table in which the number of search blocks sblk is large, the subject position determination unit 17 repeats the process with reference to the next number sblk in the detection range table until the count value becomes 1.
Information on the detected horizontal line LLN is stored in the position search information storage unit 172.
The subject position determination unit 17 searches the entire process for every 2 × 2 blocks in the above process.
Next, one vertical line search process is performed.

As shown in FIG. 7C, one vertical line VLN search process is performed.
The subject position determination unit 17 searches the detection line of the horizontal line LLN stored in the position search information storage unit 172 with a block width indicated by the number of search blocks sblk in the vertical direction.
The search method is the same as the search of the horizontal line LLN. In the case of a table in which the number of search blocks sblk is large, the process is repeated with reference to the next number sblk in the detection range table until the count value becomes 1.
The detected vertical line VLN is stored in the position search information storage unit 172.
The subject position determination unit 17 searches the entire process for every 2 × 2 blocks in the above process.
Next, a search process for the detection range DTRG is performed.

As shown in FIG. 7D, the detection process of the detection range DTRG is performed.
A continuous vertical and horizontal block detected by the horizontal line LLN and the vertical line VLN from the above processing is defined as one range.
The total average value of the detection intensities of the blocks in the range is obtained.
Next, the detection range DTRG is determined.
Also in this case, continuous vertical and horizontal blocks that can be detected by the horizontal line LLN and the vertical line VLN from the above processing are set as one range.
The total average value of the detection intensities of the blocks in the range is obtained.
Then, the subject position determination unit 17 determines the detection range [(x, y) − (xe, ye)] DTRG.
Then, fine adjustment is performed within the determination range with the magnification of the fine adjustment parameters sP1 and sP2.
The above processing can be performed simultaneously with a plurality of light sources.

The subject position determination unit 17 performs fine adjustment processing of the detection range DTRG by a method as shown in FIGS.
The subject position determination unit 17 finely adjusts the detection intensity in each determination range, investigates the detection intensity in four directions or eight directions for each detection range, and finally determines the detection intensity in a place with the highest detection intensity.
In this example, the detection intensity on one block is investigated in FIG. 8 (B), the detection intensity on the right of one block is investigated in FIG. 8 (C), and the detection intensity below one block is examined in FIG. 8 (D). Then, in FIG. 8E, the detection intensity on the left of one block is investigated.
Then, in FIG. 8F, the final detection range DTRG is determined at the place where the detection intensity is the highest.
In other words, the place where the subject OBJ is located at substantially the center of the detection range DTRG is determined as the final detection range DTRG.

After examining the number of blocks of the target frequency in step ST4, the subject position determination unit 17 determines whether or not the number of blocks of the target frequency is greater than 0 (ST5).
If it is determined in step ST5 that the number of blocks of the target frequency is greater than 0, the subject position determination unit 17 stores the detected position information obtained by searching the position storage unit 144 of the light source analysis processing unit 14 (ST6).
Then, the number “No” in the table TBL1 of the position storage unit 114 is incremented by 1 (ST7).
The subject position determination unit 17 designates the next detection range DTRG (x, y) (ST8).
If the subject position determination unit 17 determines in step ST5 that the number of blocks of the target frequency is 0, the process proceeds to step ST8 without performing steps ST6 and ST7.
The subject position determination unit 17 repeats the processes of steps ST5 to ST8 until the detection of the target image is completed (ST9).
In step ST8, when the detection of the target image is completed, areas with numbers consecutive in the (x, y) direction are integrated from the detected position information in the position storage unit 144 (ST10), and the number of detected positions is acquired (ST11). .

If the subject position determination unit 17 determines not to perform the position determination process in step ST1, the subject position determination unit 17 proceeds to the process from step ST12 in FIG.
The subject position determination unit 17 acquires the designated number from the position storage unit 144 (ST12).
Here, the subject position determination unit 17 determines whether or not there is a designated number (ST13).
If it is determined in step ST13 that there is a designated number, the subject position determining unit 17 acquires (x, y) width information of the detection range FTRG in the position storage unit 144 from the designated number (ST14), and the subject in the detection range DTRG is acquired. A state is detected (searched) (ST15).
Then, the number “No” in the table TBL1 of the position storage unit 114 is incremented by 1 (ST16).
The subject position determination unit 17 repeats the processes of steps ST14 to ST16 until the detection of the target image is completed (ST17).
If the subject position determination unit 17 determines that there is no designated number in step ST13, the subject position determination unit 17 detects (searches) the subject state of all pixels without performing the processing of steps ST14 to ST17.

Sabotage determination unit 18, positional deviation detection image pickup device according to the state of the position of the detection range (mischief), subject no response detected by the state of the object detection information (sabotage prediction), subject continuous response detection (anomaly motion acts prediction ) To notify the number of reactions within a certain time, detection time measurement, detection position, and detection information.
Sabotage determination unit 18, for example on the basis of the acquisition position information acquisition position storage section 145 of the position search information location retrieval information stored in the storage unit 172 and the light source analyzing unit 14 of the object position determination unit 17, a position deviation detection Etc.

[Position detection]
The disturbing action determination unit 18 is configured such that the position of the subject is detected in the detection range Mn or the detection range Mn stored in the position storage unit 144 and the position search information storage unit 172 and the capture range MHn stored in the capture position storage unit 145. The detection signal cannot be acquired by deviating from the capture range MHn, and it is determined that an abnormality has been detected.

9A to 9C are diagrams for explaining the positional deviation determination according to the present embodiment.
9A to 9C, the inner frame of the double frames is the detection range Mn, and the outer frame is the capture range MHn. In the drawing, OBJ indicates a subject.

FIG. 9A shows a case where the subject OBJ is within the detection range Mn, and the disturbing action determination unit 18 determines that there is no positional deviation and is normal (first normal state) NR1.
FIG. 9B shows a case where a part of the subject OBJ is in the detection range and the remaining part of the subject OBJ is outside the detection range Mn and within the capture range MHn. It is determined that it is normal (second normal state) NR2 because the positional deviation is within the allowable range.
FIG. 9C shows a case where the subject OBJ is outside the detection range Mn and the capture range MHn, and the disturbing action determination unit 18 determines that a positional deviation PG has occurred.

FIG. 10 is a diagram showing the non-occurrence and occurrence of a positional shift using the detection range and the capture range in the present embodiment as waveforms.
In FIG. 10, the horizontal axis indicates time t, and the vertical axis indicates relative detection intensity.
In FIG. 10, corresponding to FIGS. 9A, 9B, and 9C, the first normal state NR1, the second normal state NR2, and the subject OBJ are not detected from the left side in the drawing, and the positional deviation PG is not detected. The waveform when it occurs is shown continuously.

Unlike the case where it can be determined that the blocking is completely detected as in the case where the positional deviation PG has occurred, in the case of the second normal state NR2, the determination can be performed as follows.
In this case, if there is a subject in the vicinity of the signal intensity in the detection range Mn and the signal intensity in the capture range MHn, it is possible to determine whether the state is a cutoff state or a position shift state based on the difference value VDF.

Further, in order to distinguish the positional deviation and the blocking, it can be determined as shown in FIGS.
FIG. 11 is a diagram illustrating detection waveforms corresponding to the state transition of the positional deviation.
FIG. 12 is a diagram illustrating a detection waveform corresponding to a state transition at the time of occurrence of blocking by the shielding object.
11 and 12, the signal waveform takes a high level when it is within the detection range Mn and the capture range MHn, and takes a low level when out of the range.

FIG. 11 shows a first state STT1 in which the subject OBJ is within the detection range Mn, a second state STT2 in which the subject OBJ is outside the detection range Mn and within the capture range MHn, and the subject OBJ is outside the detection range Mn and within the capture range MHn. The third state STT3 in FIG. 5 and the fourth state STT4 in which the subject is within the detection range Mn are shown.
In this case, in the second state STT2 and the third state STT3, the signal level of the detection range Mn is low, but the signal level of the capture range MHn is all high from the first state STT1 to the fourth state STT4, and is in a normal state. Is determined.

FIG. 12 shows a first state STT11 in which the subject OBJ is in the detection range Mn, a second state STT12 in which the detection range Mn and the capture range MHn are shielded by the shield MSK, and a third state in which the subject OBJ is in the detection range Mn. An example is shown in which the state STT13 and the subject OBJ transition to the fourth state STT14 in the detection range Mn.
In this case, in the second state to the fourth state STT12 to STT14, the signal level of the detection range Mn is low, but the signal level of the capture range MHn is high in the first state STT11 and the third state STT13. It is determined that the state STT12 and the fourth state STT14 are at the low level, and the interruption has occurred.

[Subject no response detection]
In the detection state of FIG. 10, when the state of no subject detection continues for the time tAlm, it is determined that there is no response and the continuous interruption detection.

[Continuous abnormal response detection]
The disturbing action determination unit 18 determines that the position of the subject OBJ is the detection range Mn with respect to the detection range Mn stored in the position storage unit 144 and the position search information storage unit 145 and the capture range MHn stored in the capture position storage unit 145. And detection abnormality due to fine movement by deviating from the capturing range MHn, or abnormal detection near the threshold Th is determined as abnormal detection.

FIGS. 13A to 13C are diagrams for explaining continuous abnormality response determination according to the present embodiment.
13A to 13C, the inner frame of the double frames is the detection range Mn, and the outer frame is the capture range MHn. In the drawing, OBJ indicates a subject.

FIG. 13A shows a case where the subject OBJ is within the detection range Mn, and the disturbance act determination unit 18 determines that there is no positional deviation and is normal (first normal state) NR1.
FIG. 13B shows a case where a part of the subject OBJ is in the detection range and the remaining part of the subject OBJ is outside the detection range Mn and within the capture range MHn. At this time, the subject OBJ is finely moved at the boundary between the detection range Mn and the capture range MHn, and the disturbance act determination unit 18 is normal (second normal state) NR2 because it is a positional deviation within the allowable range. Determined.
FIG. 13C shows a case where the subject OBJ is outside the detection range Mn and the capture range MHn. At this time, the subject OBJ is finely moved at the boundary between the capture range MHn and the outside of the capture range MHn, and the disturbance act determination unit 18 determines that a positional deviation PG of continuous abnormality (continuous cutting) has occurred. .

FIG. 14 is a diagram showing non-occurrence and occurrence of positional deviation using the detection range and the capture range, and continuous abnormal response detection in waveforms in this embodiment.
In FIG. 14, the horizontal axis indicates time t, and the vertical axis indicates relative detection intensity.
In FIG. 14, corresponding to FIGS. 13A, 13B, and 13C, the first normal state NR1, the second normal state NR2, and the subject OBJ are not detected from the left side in the figure, and the positional deviation PG is not detected. The waveform when it occurs is shown continuously.

<1>: The interfering action determination unit 18 detects the signal intensity variation range Dkw (t) at time tA in the signal intensity Dk (T) within the detection range Mn, so that the distance between the subject OBJ and the imaging device Variation of the signal intensity of the subject OBJ can be detected. Further, the disturbing action determination unit 18 can also detect a continuous variation in the time tA.

In this case, the fluctuation range Dkw (t) of the signal intensity Dk (T) at time t is given by the following equation.
[Equation 1]
Dkw (t) = Dk (t−1) −Dk (t) (1)

  The fluctuation range DkWA of the signal intensity Dk (t) at time (section) tA is given by the following equation.

Here, assuming that the threshold value of the fluctuation range Dkw (t) of the signal intensity Dk (t) is DkwTh, whether or not the intensity fluctuation detection determination is greater than the threshold value DkWTh (Dkw) It is determined by (t)> DkwTh). In this case, when the fluctuation range Dkw (t) of the signal intensity at time t is larger than the threshold value DkwTh, it is detected and determined as an intensity fluctuation.
Further, the continuous intensity fluctuation detection determination is made based on whether or not the fluctuation width DkWA of the signal intensity Dk (t) at time (section) tA is larger than the threshold DkWTh (DkWA> DkWTh). In this case, when the fluctuation width DkWA of the signal intensity Dk (t) at time (section) tA is larger than the threshold value DkWTh, it is detected and determined as a continuous intensity fluctuation.

<2>: The disturbing action determination unit 18 can determine detection due to positional deviation based on a change in the difference value VDF of the signal intensity caused by the detection range Mn and the capture range MHn, and when a difference occurs at the time tA of positional deviation detection. It can be determined that there is a continuous position shift.

In this case, if the signal intensity of the detection range Mn is Du (t) and the signal intensity of the capture range MHn is Dh (t), the time difference value Ds (t) is given by the following equation.
[Equation 3]
Ds (t) = Dh (t) −Du (t) (3)

  The difference value DstA of time (section) tA is given by the following equation.

Here, assuming that the threshold value of the difference amount is DstATh, the positional deviation detection determination is made based on whether or not the time difference value Ds (t) is larger than the threshold value DstATh (Ds (t)> DstATh). In this case, when the time difference value Ds (t) is larger than the threshold value DstATh, a position shift is detected and determined.
Further, the positional deviation fine motion detection determination is made based on whether or not the difference value DstA of the time (section) tA is larger than the threshold value DstATh (DstA> DstATh). In this case, when the time difference value DstA is greater than the threshold value DstATh, it is detected and determined as a slight positional shift.

<3>: If the signal intensity within the detection range Mn within the detection range Mn within the time thD on the threshold Th is continuously changed, the disturbing action determination unit 18 can detect continuous disconnection.

Also in this case, similarly to <1> above, the fluctuation range Dkw (t) of the signal intensity Dk (T) at time t is given by the following equation (similar to the above equation 1).
[Equation 5]
Dkw (t) = Dk (t−1) −Dk (t) (5)

  The fluctuation width DkWthD of the signal intensity Dk (t) at time (section) thD is given by the following equation.

  Here, assuming that the margin count of the threshold Th is k, the continuous cut detection determination is based on the value (Th + k) where the fluctuation width DkWthD of the signal intensity Dk (t) of the time (section) thD is added to the threshold Th. It is determined by whether or not it is small (DkWthD << Th + k)). In this case, when the fluctuation range DkWthD of the signal intensity Dk (t) of the time (section) thD is smaller than the value (Th + k) obtained by adding the coefficient k to the threshold Th, the continuous cutting is detected and determined.

  FIG. 15 is a diagram showing, as a table, the non-occurrence and occurrence of positional deviation using the detection range and the capture range and the detection state of continuous abnormal response detection in the present embodiment.

In FIG. 15, detection targets in the detection range Mn, in the capture range MHn, and outside the capture range MHn are shown in association with each other.
Examples of detection targets include “normal detection”, “intensity fluctuation detection”, “continuous intensity fluctuation detection”, “continuous cut detection”, “position shift detection”, and “position shift fine movement detection”.

  When within the detection range Mn, “normal detection”, “intensity fluctuation detection” by the process <1>, “continuous intensity fluctuation detection,“ continuous cutting detection ”by the process <3>, and the process <2> above “Position shift detection” and “Position shift fine movement detection” are possible.

  When within the capture range MHn, “position shift detection” and “position shift fine movement detection” by the process <2> are possible.

  When the position is outside the capture range MHn, “position shift detection” and “position shift fine movement detection” by the process <2> are possible.

Next, other configurations and functions of the detection system according to the present embodiment will be described in detail.
First, the configuration of the CCD of the imaging device 12 according to the present embodiment will be described.

  FIG. 16 is a diagram illustrating an example for explaining the structure of the CCD according to the present embodiment.

The CCD 20 in FIG. 16 is an interline transfer, and includes a photodiode PD21, a vertical transfer CCD22, a horizontal transfer CCD23, and an amplifier 24.
The photodiodes PD21 are arranged in a matrix. The photodiodes PD21 arranged in the vertical line direction are connected to a vertical transfer CCD 22 for transferring charges for each column. The end of each vertical transfer CCD 22 is connected to a horizontal transfer CCD 23 that transfers charges to the amplifier. An amplifier 24 is connected to the output side of the horizontal transfer CCD 23.

The video scanning method is interlaced, and one screen is interlaced scanning, and is composed of an odd field and an even field.
First, light enters the photodiode PD21, and charges are accumulated in the photodiode PD21 during the charge accumulation time. During this time, the photodiode PD21 and the vertical transfer CCD 22 are disconnected.
When the charge accumulation time ends, the photodiode PD21 and the vertical transfer CCD 22 are brought into conduction, and the accumulated charges are transferred to the vertical transfer CCD 22. Immediately after this, the photodiode PD21 and the vertical transfer CCD 22 are disconnected, and the next charge accumulation is started in the photodiode PD21. The charges transferred to the vertical transfer CCD 22 are transferred to the horizontal transfer CCD 23 for each horizontal line and input to the amplifier 24.
The frequency until the charge is transferred from the vertical transfer CCD 22 to the horizontal transfer CCD 23 for each horizontal line is the horizontal scanning frequency of the CCD 20 of 15.734 KHz. When all the charges of the vertical transfer CCD 22 are transferred to the horizontal transfer CCD 23, the vertical transfer CCD 22 and the photodiode PD 21 are again brought into conduction, and the charges of the photodiode PD 21 are transferred to the vertical transfer CCD 22. In the case of a field accumulation CCD, charges are accumulated in the photodiode PD21 by photoelectric conversion, and the transfer frequency until this charge is transferred from the photodiode PD21 to the vertical transfer CCD 22 is 59.94 Hz.

  FIG. 17 is a diagram for explaining a time series of the CCD 20 of FIG.

As shown in FIG. 17, the required time until charge is accumulated in the photodiode PD21 by photoelectric conversion is ΔT1, and the required time until the charge is transferred from the photodiode PD21 to the vertical transfer CCD 22 is ΔT2.
As can be seen from FIG. 17, the light energy incident on the CCD 20 is sampled at the charge accumulation period ΔT = ΔT1 + ΔT2 = (1 / 59.94) seconds while being integrated during the charge accumulation time ΔT1.

As shown in FIG. 1, a luminance signal is extracted by a luminance signal extraction unit 13 from a captured image signal captured by the imaging device 12, and the luminance signal is input to the light source analysis processing unit 14.
Here, a method for reading out pixels from the CCD 20 (see FIG. 16) according to the present embodiment will be described.

FIG. 18 is a diagram showing an arrangement example of pixels of a single-plate complementary color filter type CCD.
FIG. 19 is a diagram illustrating an example of a combination of color signals in the odd field OFD and the even field EFD.

The color filter of the pixel is composed of Ye (yellow), Cy (cyan), Mg (magenta), and G (green), and is arranged as shown in FIG. Pixels are read by adding the upper and lower pixels. The combination to be added is shifted by one column between the odd field OFD and the even field EFD. Specifically, in the n-th line of the odd field OFD, (C11 + C21), (C12 + C22), (C13 + C23), (C14 + C24), (C15 + C25),... In the n-line of the even field EFD, (C21 + C31), (C22 + C32), (C23 + C33), (C24 + C34), (C25 + C35),...
Accordingly, color signals are output in the odd field OFD and the even field EFD as shown in FIG.
In any case, the same color pattern of a combination of Ye, Cy, Mg, and G is repeated in a two-pixel cycle.
In other words, the color signal appears superimposed on a frequency of two pixel periods or more. Therefore, if this color signal is passed through a low-pass filter having a cutoff frequency of two pixel periods, the color signal is lost and only a luminance signal is obtained.
Therefore, luminance information is sampled at a cycle of two pixels.

A projection area REG indicated by a circle in FIG. 18 shows a state in which an image of a subject by a light source is projected. It is assumed that the pixels C35, C36, C45, C46, C55, and C56 are completely in the projection region REG and are uniformly irradiated with light.
In the odd field OFD, the luminance information is read by a combination of C35, C36, C45, and C46 of the horizontal line (n + 1), and in the even field EFD, by a combination of C45, C46, C55, and C56 of the (n + 1) line of the horizontal line. Is done.

  As described above, the luminance signal among the signals from the imaging device 12 is output to the light source analysis processing unit 14. This luminance signal is input to the first calculation unit 141 and the second calculation unit 142 and subjected to predetermined processing.

  Next, a luminance signal processing method performed by the first calculation unit 141 and the second calculation unit 142 will be described with reference to FIG.

  FIG. 20 is a timing chart for explaining the signal processing method of the luminance signal of the light source analysis processing unit according to the present embodiment.

FIG. 20A is a diagram showing interline scanning of the imaging device 12, and shows a state of either the even field EFD or the odd field OFD. FIGS. 20B to 20E respectively show waveforms W1, W2, W3, and W4 that represent temporal changes in the luminance signal level processed by the light source analysis processing unit 14, and FIG. 20F changes at a constant cycle. It is a figure which shows the waveform W5 of the sine wave to do.
The odd field OFD and even field EFD scan one frame. That is, as shown in FIG. 20A, one frame is scanned by A and B and C and D.
In the following description, f represents frequency, t represents time, θ represents phase difference, and ω satisfies (ω = 2πf). Note that π is the circumference ratio.

A waveform W1 and a waveform W2 illustrated in FIGS. 20B and 20C are waveforms for deriving a function of a waveform indicated by a waveform W3 and a waveform W4 described later.
A waveform W3 illustrated in FIG. 20D is a time development distribution of a function for obtaining the level difference AC when the luminance level difference between the luminance signal of the A field and the luminance signal of the C field is defined as the level difference AC. Indicates.
A waveform W4 illustrated in FIG. 20E is a function time for obtaining the level difference BD when the difference in luminance level between the luminance signal in the B field and the luminance signal in the D field is the level difference BD. The development distribution is shown.
Waveform W3 is derived from waveform W1 and waveform W2, and is obtained by adding waveform W1 and waveform W2 and dividing by two. Waveform W4 is derived from waveform W1 and waveform W2, and is obtained by subtracting waveform W1 from waveform W2 and dividing it by 2.

At this time, a time average SAC that is a first time average of the level difference AC is calculated by the first calculation unit 141 shown in FIG. In addition, a time average SBD that is a second time average of the level difference BD is calculated by the second calculation unit 142.
Specifically, the time average SAC is calculated from the luminance level difference AC between the A field and the C field by a combination of C35, C36, C45, and C46.
Similarly, the time average SBD is calculated from the level difference BD between the B field and the D field by a combination of C45, C46, C55, and C56.

The calculation method of the time average is described.
The time average SAC of the level difference AC between the A field and the C field is calculated by multiplying the waveform W1 by the waveform W3 shown in FIG.
Similarly, the time average SBD of the level difference BD between the B field and the D field is shown in FIG. 20 (E) in a waveform representing the time change of the light irradiated to the pixel by the combination of C45, C46, C55, and C56. The time average SBD is calculated by multiplying the waveform W4 shown.

First, a method for calculating the time average SAC will be specifically described.
The waveform W3 is expressed by a mathematical expression. First, the waveform W1 and the waveform W2 can be expressed by the following Fourier series.

  Here, it is assumed that the waveforms W1 and W2 have the same period f2. From the expressions (7) and (8), the waveform W3 can be expressed as the expression (10).

  By the way, the waveform W5 having the period f1 shown in FIG. 20F can be expressed by a sine wave as shown in the equation (11).

  When the waveform W3 represented by the equation (10) is multiplied by the sine wave W5 represented by the equation (11), the equation (13) is obtained.

Next, the time average of the expression (13) from time 0 to time T is taken. Of the terms shown on the right side of equation (13), the term including time t is an AC signal, and therefore the time average is zero.
Therefore, only when (ω ₁ − (2n−1) ω ₂ = 0), the constant cos θ ₁ and the constant sin θ ₁ remain, and the time average SAC is expressed by the equation (14).

  In this way, the time average SAC is obtained by the first calculation unit 141. Similarly, the time average SBD is obtained by the second calculation unit 142, and is expressed by equation (15).

Now, the sum of squares (S _AC ² + S _BD ² ) of the time average SAC and SBD expressed by the equations (14) and (15) is expressed by the equation (16).

From this equation (16), when the frequency component (f ₁ = (2n−1) f ₂ ) is included in the light incident on the CCD 20 (see FIG. 16), it is expressed by the equation (16). Waveform components are detected.

Next, frequency components included in the light source 11 will be considered.
FIG. 21 is a diagram illustrating a waveform W6 for frequency components included in the light source 11. The light source 11 emits light at the luminance level L1 at the frequency f3 for the time τ.

This waveform W6 is developed into a Fourier series. The waveform W6 is a periodic function of a period (T ₃ = 1 / f ₃ ). When (ω ₃ = 2πf ₃ ), the waveform W6 is represented by a general expression of Fourier series as shown in Expression (17).

The coefficients a ₀ , a _n , and b _{n in the} equation (17) are obtained from the waveform W6 as in the equations (18) to (20).

  Therefore, the Fourier series of the waveform W6 is expressed by equation (21).

Therefore, when the blinking period of the light source 11 is set to four times the field period, that is, when (f ₃ = f ₂ ), the frequency coincides with the odd term from the expressions (13) and (21), and the time average SAC And the sum of squares of SBDs is as shown in equation (22).

  When the duty ratio of lighting of the light source 11 is D, it is expressed by equation (23).

    Therefore, the square sum SACBD of the time average SAC and SBD expressed by the equation (22) is expressed by the equation (24) when the equation (23) is used.

  By the way, the term (25) on the right side of the following expression (24) converges.

  The term (25) on the right side of the equation (24) takes values as shown in Table 1 with respect to the duty ratio D. Table 1 is shown below.

  Based on Table 1, when the duty ratio D is taken on the horizontal axis and the square sum SACBD of the time average SAC and SBD is taken on the vertical axis, the relationship between the duty ratio D and the square sum SACBD is as shown in FIG. become.

From FIG. 22, it can be seen that the square sum SACBD becomes maximum at the duty ratio D = 0.5.
Therefore, the sum of squares SACBD expressed by the equation (24) is expressed by the following equation.

[Equation 21]
S _AC ² + S _BD ² = 0.08333L ₁ ² (26)

  As shown in equation (26), the arithmetic processing unit 133 detects the luminance of the light source 11 (see FIG. 1), and outputs the detection result (square sum SACBD) to the determination unit 16 via the filter processing unit 15. To do. This detection result is used for determining the state of the subject by the determination unit 16.

The light source 11 according to the present embodiment does not depend on a specific light source. Therefore, it is considered whether the luminance can be detected for other light sources.
Incandescent light bulbs and fluorescent lamps that are widely used as light sources are 100 Hz in regions where the power supply frequency is 50 Hz and 120 Hz in regions where 60 Hz. The field frequency of an NTSC television is 59.94 Hz, and the field frequency of a monitor used for a personal computer is 60 Hz or more so as not to flicker.

  The field frequency of the NTSC television is 59.94 Hz, and if the light source is caused to emit light at a quarter period, the frequency f2 of the luminance level difference is as follows.

[Equation 22]
f ₂ = 59.94 / 4 = 14.985 Hz (27)

  From the equations (13) and (21), a signal component is detected when the odd multiple of the frequency f2 matches the integer multiple of the frequency f3 of the light source 11.

  Table 2 is a table of values indicating the relationship between the light emission frequency of different light sources and the frequency in the difference in luminance signal level.

F1 in Table 2 is the frequency of the sine wave of equation (11), f2 is the frequency shown in equation (27), and f3 is the frequency of the light source 11, the frequency of illumination in the 50 Hz region, and the illumination in the 60 Hz region, respectively. Frequency, NTSC television field frequency, and monitor field frequency used in personal computers.
According to Table 2, up to m = 30, f3 is 100, 120, 59.94 Hz, and there is nothing that satisfies (n × f ₃ = (2m−1) × f ₂ ).

For example, regarding a monitor of a personal computer, if the field frequency is 60 Hz or more, there is a monitor that is scanned at 74.925 Hz as a possibility that the largest output is detected in the signal processing output of the detection system 10. When
That is, when f ₃ = 74.925 Hz, as shown in Table 2, (5 × f ₂ ), (15 × f ₂ ), (25 × f ₂ ), and (1 × f ₃ ), ( 3 × f ₃ ), (5 × f ₃ ). The signal level detected at this time is expressed by the following equation.

したがって、（２８）式で示される信号レベルは光源１１の１／２５のレベルであり、図１に図示していない信号処理で別に除去できる。

Therefore, the signal level expressed by the equation (28) is 1/25 of that of the light source 11 and can be removed separately by signal processing not shown in FIG.

  As described above, the detection system 10 detects the state of the light source or the subject irradiated on the light source without depending on the light emission frequency of the light source.

  Hereinafter, a series of operations of the detection system according to the present embodiment will be described with reference to FIG.

  FIG. 23 is a flowchart for explaining an outline of a series of operations of the detection system according to the present embodiment.

In the present embodiment, first, the luminance of the light source 11 within the charge accumulation time of the imaging device 12 is changed by 4n times the field period of the imaging device 12 (ST21).
Next, a luminance signal is acquired by the luminance signal extraction unit 13 every n fields in the field unit from the imaging device 12 (ST22), and this luminance signal is output to the first calculation unit 141 and the second calculation unit 142.
The first arithmetic unit 141 obtains the time average SAC of the level difference AC of the luminance signal level in the projection area REG of the mth and (m + 2) th fields. Further, the second arithmetic unit 142 obtains the time average SBD of the level difference BD of the luminance signal level in the projection area REG of the (m + 1) th and (m + 3) th fields (ST23).
These time averages SAC and SBD are output to the arithmetic processing unit 143.
Subsequently, the arithmetic processing unit 133 obtains the square sum SACBD of the time average SAC and SBD (ST24), and outputs the result to the determination unit 16 via the filter processing unit 15.
The determination unit 16 determines the state of the subject according to the value of the square sum SACBD (ST25). The determination result is supplied to the subject position determination unit 17.
The subject position determination unit 17 determines whether or not to determine the subject position (ST26).
Here, when determining the subject position, the subject position determination unit 17 detects the state of the subject and acquires information about the detection position (ST27), and the detected position information is stored in the position storage unit 144 of the light source analysis processing unit 24. Remembered.
When the subject position is not determined, the subject position determination unit 17 acquires the specified position (specified number) from the position information stored in the position storage unit 144 (ST28), and detects the state of the subject at the specified position. (ST29).
Then, the disturbance act determination unit 18 detects the position shift (mischief) of the imaging device based on the state of the detection range , the subject non-response detection (prediction of the disturbing act) based on the state of the subject detection information, and the subject continuous response detection (abnormal moving object). An action prediction process is performed (ST30). After the detection process, the number of reactions within a certain time, detection time measurement, detection position, and detection information are notified to a control system (not shown).
Sabotage determination unit 18, for example on the basis of the acquisition position information acquisition position storage section 145 of the position search information location retrieval information stored in the storage unit 172 and the light source analyzing unit 14 of the object position determination unit 17, a position deviation detection Etc.

In addition, when performing steps ST23 and ST24 in the light source analysis processing unit 14, based on the detection position coordinates of the detection range Mn stored in the position storage unit 144 and the capture range MHn stored in the capture position storage unit 145. Te, analyzing one or more images of test knowledge range designated by the coordinates (detection) is performed.
By this processing of the light source analysis processing unit 14, the present detection system 10 can detect only the position of the light source or the subject irradiated with the light source, and does not detect the position excluded as the subject. It is effective and can be detected with high accuracy.

In the arithmetic processing unit 143 according to the present embodiment, the square sum SACBD of the time average SAC and SBD is obtained, and the sum of the time average SAC and SBD (S _AC + S _BD ) is determined by the determination unit 16. It can also be used.

  As described above, according to the present embodiment, the background noise of the captured image can be removed, and the state of the subject irradiated with the light source or the light source can be detected.

According to the detection system according to the present embodiment, in the detection of the light source or the state of the subject irradiated to the light source, detection when there are a plurality of subjects can be performed for each subject, and tracking and capturing of the subject can be performed. By doing so, there is no need to detect the position of an unnecessary object that is not a subject, and the effect of detecting the state of the subject at high speed and with high accuracy can be obtained.
Moreover, since it does not depend on the light source, it does not depend on the specifications of the imaging device, and for example, a generally available camera can be used as the imaging device of the present detection system.

Furthermore, according to the detection system according to the present embodiment, in detecting the state of the light source or the subject irradiated on the light source, it is possible to infer a disturbing action from the detection state by performing detection of the subject with high accuracy. Thus, it is possible to detect a disturbing action or a positional deviation with high accuracy.
In the detection target range, it is possible to determine a case where the block of the subject continues for a certain period of time with the block of the subject. If it is determined that there is no response continuously, it means that there is a positional deviation of the imaging device or an object that blocks the subject, and an obstructive action due to spraying or covering can be detected.
It is possible to detect a case where the position of the imaging device is shifted and notify the state. In addition, the detection accuracy of misalignment and blocking is further improved.
In addition, it is possible to detect a change in the fluctuation range of the signal intensity, thereby detecting an abnormal fluctuation value due to wind and rain, dense fog, snow, artificial mischief, and the like.
It is possible to detect fine movements due to misalignment, thereby detecting artificial mischiefs, earthquakes, shaking in installed buildings, and the like.
Continuous blockage detection can be performed, and it is possible to detect the influence of obstacle detection of natural phenomena such as places with heavy traffic, crowds, wind and rain, and trees.

Furthermore, this detection system uses a plurality of light sources and can transmit signals to the signal processing unit in parallel.
Alternatively, it is possible to provide a plurality of light source colors and perform wavelength multiplexing transmission of signals.
It is also possible to process the signal by appropriately blinking the light source.

[Second Embodiment]
Next, a second embodiment according to the present invention will be described.

FIG. 24 is a diagram illustrating a configuration example of a detection system according to the second embodiment of the present invention.

The detection system 10A according to the second embodiment is different from the detection system 10 according to the first embodiment as follows.
The detection system 10A according to the second embodiment includes a recording device 31 that records the subject OBJ captured by the imaging device 12 on the storage medium 30, and also reproduces the subject image stored in the storage medium 30. A device 32 is included.
Thereby, the disturbance action determination can be repeatedly determined by repeatedly reproducing the video imaged by the imaging device 12.

[Third Embodiment]
Next, a third embodiment according to the present invention will be described.
FIG. 25 is a diagram illustrating a configuration example of a detection system according to the third embodiment of the present invention.

The difference between the detection system 10B according to the third embodiment and the detection system 10A according to the second embodiment is that the light source analysis processing unit 14B employs a configuration corresponding to the phase problem of the subject light source during reproduction. It is in.
In the light source analysis processing unit 14B, a difference determination unit 146 and a determination value memory 147 are added.
The difference determination unit 146 monitors the calculation result A of the first calculation unit 141 and the calculation result of the second calculation unit 142, and the value of the calculation result A or B converges to 0 or a value close to 0 as much as possible. When it is determined that the correction has been performed, the arithmetic processing unit 143 is instructed to correct this value.
The difference determination unit 146 performs determination processing by applying the detection range Mn and the capture range MHn stored in the position storage unit 144.
The determination value memory 147 stores data referred to by difference determination.
With this configuration, it is possible to solve the problems associated with the phase shift of the subject light source during reproduction.

[Fourth Embodiment]
Next, a fourth embodiment according to the present invention will be described.

In the fourth embodiment, the field accumulation / interlace imaging device 12 according to the first embodiment is replaced with a frame accumulation / interlace imaging device. At the same time, the luminance change period of the light source 11 according to the first embodiment is changed from 4n times the field period to 4n times the frame period. Accompanying this change in the change period, the luminance signal acquisition period is changed from every n fields to every 2n fields.
As described above, by changing the luminance change cycle of the light source 11 and the luminance signal acquisition cycle, the same effects as those of the first embodiment can be obtained.
Therefore, according to the present embodiment, the background noise of the captured image can be removed, the state of the light source or the subject irradiated with the light source can be detected, and even when there are a plurality of subjects, detection can be performed for each subject.

[Fifth Embodiment]
Next, a fifth embodiment according to the present invention will be described.

In the fifth embodiment, the field accumulation / interlace scanning imaging device 12 according to the first embodiment is replaced with a frame accumulation / non-interlace scanning imaging device.
As described above, even when a non-interlaced scanning imaging apparatus is used, the same effect as that of the first embodiment can be obtained.
Therefore, according to the present embodiment, the background noise of the captured image can be removed, the state of the light source or the subject irradiated with the light source can be detected, and even when there are a plurality of subjects, detection can be performed for each subject.

Note that the method described above in detail can be formed as a program according to the above-described procedure and executed by a computer such as a CPU.
Further, such a program can be configured to be accessed by a recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a floppy (registered trademark) disk, or the like, and to execute the program by a computer in which the recording medium is set.

  DESCRIPTION OF SYMBOLS 10, 10A, 10B ... Detection system, 11 ... Light source, 12 ... Imaging device, 13 ... Luminance signal extraction part, 14 ... Light source analysis process part, 141 ... 1st calculation Part (A), 142 ... 2nd operation part (B), 143 ... Operation processing part, 144 ... Position storage part, 145 ... Capture position storage part, 15 ... Filter processing part, 16 ... determination unit, 17 ... subject position determination unit, 171 ... position search processing unit, 172 ... position search information storage unit, 18 ... obstruction act determination unit.

Claims (13)

A light source;
An imaging device for imaging the light source or a subject illuminated by the light source;
An analysis processing unit that performs analysis processing on a signal acquired from the imaging device;
A determination unit that detects and determines the state of the light source or subject based on the analysis result of the analysis processing unit;
A position storage unit capable of storing information on positions including detection ranges in images of a plurality of subjects;
From the detection information obtained by the determination unit, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is acquired, A subject position determination unit capable of storing detection information as information on the position in the position storage unit;
Positional deviation detection image pickup device according to the state of the position of the detection range obtained by the object position determination unit, an object no response detection, and have a, and sabotage judging unit to perform at least one of the detection determining of the subject continuous response detection And
The analysis processing unit
The signal is acquired from the imaging device every predetermined scanning plane period, a time average of the signal level difference is obtained from a signal level difference of the signal between a plurality of different scanning planes, and further, based on the value of the time average Perform an analysis process to perform a predetermined calculation, and output the calculation result of the analysis process to the determination unit,
Detection system when the detection information is stored in the position storage unit for selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information. A capture position storage unit that stores the position of the capture range corresponding to the detection range,
The analysis processing unit
The analysis process is selectively performed on images in the detection range and the capture range,
The disturbing action determination unit
The detection system according to claim 1, wherein when the position of the subject deviates from the detection range and the capture range and a detection signal cannot be acquired, it is determined that a positional shift has been detected. The disturbing action determination unit
When the subject is located at a position corresponding to the midpoint between the signal strength within the detection range and the signal strength within the capture range, the signal is blocked by the difference between the signal strength within the detection range and the signal strength within the capture range. The detection system according to claim 2, wherein a state or a misalignment state is determined. The disturbing action determination unit
The detection system according to claim 2 or 3, wherein in a detection state, if a state in which a detection signal of a subject cannot be acquired continues for a certain period of time, it is determined that no interruption has been detected as a non-response. The disturbing action determination unit
When the subject is located at a position corresponding to the midpoint between the signal intensity within the detection range and the signal intensity within the capture range, a change in the difference value between the signal intensity within the detection range and the signal intensity within the capture range To determine misalignment,
The detection system according to claim 4, wherein when the difference for a certain time occurs, it is determined that a continuous position shift is detected. The disturbing action determination unit
In the signal intensity within the above detection range, the fluctuation of the signal intensity of the subject between the subject and the imaging device is detected by detecting the fluctuation range of the signal intensity for a certain time, and the continuous fluctuation is detected when the fluctuation continues for a certain time. The detection system according to any one of claims 2 to 5. The disturbing action determination unit
The detection system according to any one of claims 2 to 6, wherein it is determined that continuous cutting has been detected when there is a change in the signal intensity that repeatedly increases and decreases with respect to the threshold value within a certain time. The analysis processing unit
When the detection information is not stored in the position storage unit or in the all-image detection mode, the analysis processing is performed on the entire image,
When the detection information to the position storage section is stored, when the object position detection mode, any one of claims 1 to 7, which performs the analysis process on the image of the test knowledge range specified in the above detection information The detection system according to one. The subject position determination unit
In the determination of the detection range, the size of the subject is measured, and a table whose size exceeds the current coordinate (x, y) width or below the coordinate (x, y) width is prepared beforehand. The detection system according to any one of claims 1 to 8, wherein a detection range of values is selected. The subject position determination unit
After searching all the lines in the x and y directions for every predetermined number of search blocks, the detection range search process is performed, and the continuous vertical and horizontal blocks detected by the line search are regarded as one range. The detection system according to claim 9, wherein a detection range is determined by obtaining a total average value of detection intensities of blocks. The subject position determination unit
The detection system according to claim 9 or 10, wherein the determined detection range can be finely adjusted. An imaging step of imaging with an imaging device a light source or a subject illuminated by the light source;
A signal is acquired from the imaging device every predetermined scanning plane period, a time average of the signal level difference is obtained from a signal level difference of the signal between a plurality of different scanning planes, and a predetermined average is obtained based on the time average value. An analysis process step for performing an analysis process for performing an operation;
A determination step of detecting and determining the state of the light source or subject based on the analysis result of the analysis processing step;
From the detection information obtained in the determination step, the position and size of the subject in all images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is acquired, A subject position determination step for storing detection information in the position storage unit as information on the position;
An obstruction act determination step for performing at least one detection determination among the position shift detection of the imaging device, the subject non-response detection, and the subject continuous response detection according to the position state of the detection range obtained by the subject position determination step. And
In the analysis processing step,
Signal processing method of the detection system when the detection information is stored in the position storage unit, including the step of selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information. An imaging process in which an imaging device captures an image of a light source or a subject illuminated by the light source;
A signal is acquired every predetermined scanning plane period, a time average of the signal level difference is obtained from a signal level difference of the signal between a plurality of different scanning planes, and a predetermined calculation is performed based on the time average value. Analysis processing,
A determination process for detecting and determining the state of the light source or subject based on the analysis result of the analysis process;
From the detection information obtained in the determination process, the position and size of the subject in all the images are searched to determine the position of the subject including the detection range, and the detection information including the detection range related to the subject is acquired. Subject position determination processing for storing detection information in the position storage unit as information relating to the position;
A jamming action determination process that performs at least one detection determination among a position shift detection of the imaging device, a subject non-response detection, and a subject continuous response detection according to the position state of the detection range obtained by the subject position determination process. And
In the above analysis process,
A computer signal processing of the detection system including processing when the detection information is stored in the position storage unit for selectively performing the analysis processing for the image of the test knowledge range specified in the above detection information The program to be executed.

Images