JP5406787B2 - Detection system, signal processing method thereof, and program - Google Patents

Description

  The present invention relates to a detection system that detects the state of a subject using an imaging device or the like, a signal processing method thereof, and a program, for example.

For example, an imaging apparatus used in a nighttime crime prevention system shown in Patent Document 1 has been proposed.
This imaging apparatus has a signal processing unit, and the light source has a frequency higher than 100 Hz or 120 Hz (when a commercial power source of 50 Hz or 60 Hz is used as the power source, the brightness of the light source fluctuates at 100 Hz or 120 Hz that is twice that). Modulate with high frequency. Further, the signal processing unit has a detection unit for detecting the high frequency modulation signal.
However, this imaging apparatus needs to have a frame rate higher than its modulation frequency.

By the way, a standard called NTSC (National Television Standards Committee) system or PAL (Phase Alternating Line) system is adopted for imaging apparatuses which are widely spread in general.
Since an image pickup apparatus adopting such a standard has a low frame rate, for example, it cannot be used as an image pickup apparatus for a crime prevention system as shown in Patent Document 1.

Therefore, a detection system that can detect a light source or the state of a subject irradiated with the light source and can be applied to a crime prevention system adopting a standard such as NTSC has been proposed (see Patent Document 2).
This detection system detects the state of the subject of the frequency component of the light source.

Japanese Patent No. 3019309 JP 2008-141251 A

By the way, in the above detection system, when data transmission is performed using isochronous transfer of the USB interface, for example, in transmission of the video signal of the luminance signal extraction unit from the imaging device, data loss or transmission / reception failure may occur. .
At that time, in the state detection of the light source or the object irradiated to the light source, the processing in the calculation unit is not accurately performed, and there is a possibility that the determination unit makes an erroneous determination.

  The present invention is capable of detecting the state of a light source or a subject irradiated with the light source, as well as a detection system capable of preventing erroneous determination in a determination unit when a problem occurs in video data transfer and a signal thereof It is to provide a processing method and a program.

A detection system according to a first aspect of the present invention includes an imaging unit including a light source, an imaging device that images the light source or an object irradiated by the light source, and a first interface that transmits a signal acquired from the imaging element. When receives the signal transmitted from the imaging unit, and a signal processing portion for detecting a state of an object by performing a signal processing on the received signal, the signal processing unit, transmitted from the imaging unit A second interface for receiving the received signal, a reception buffer unit for storing the signal received by the second interface and outputting it as data, and the state of the subject from the signal output from the reception buffer unit A signal extraction unit that extracts a signal used for signal processing for detection; and a current frame and a previous frame when the signal extraction unit receives a signal from the reception buffer unit A time difference acquisition unit that calculates a time difference from a lemme, and a time difference determination unit that obtains a difference value between the time difference acquired by the time difference acquisition unit and a preset threshold and determines whether the time difference is normal or abnormal And acquiring the signal from the signal extraction unit for each predetermined scanning plane period, obtaining a time average of the signal level difference from the signal level difference of the signal between a plurality of different scanning planes, and further calculating a value of the time average An arithmetic unit for obtaining a square sum of time averages, an imaging target extraction determination unit for determining whether an imaging target is detected based on a calculation result of the calculation unit, and the light source based on a calculation result of the calculation unit Alternatively, in response to the determination results of the determination unit that determines the detection of the subject irradiated with the light source and the time difference determination unit and the imaging target extraction determination unit, the time difference is abnormal and the imaging target extraction determination unit If detecting the subject, including a signal transfer error determination section for updating the threshold value of the time difference determination unit to a threshold where the time difference determination unit has determined.

In the signal processing method of the detection system according to the second aspect of the present invention, a light source or a subject illuminated by the light source is imaged by an imaging device, and a signal acquired from the imaging device is transmitted from an imaging unit that is transmitted by the first interface. A signal processing step for detecting a state of a subject by performing signal processing on the received signal, and the signal processing step receives the signal transmitted in the imaging step by the second interface A reception step for storing the signal received by the second interface and outputting it when it is determined as data, and performing signal processing for detecting the state of the subject from the signal output by the reception buffer step. A signal extraction step for extracting a signal to be used, and the signal extraction step from the reception buffer step. The time difference acquisition step for calculating the time difference between the current frame and the previous frame at the time of receiving the signal and the difference value between the time difference acquired in the time difference acquisition step and a preset threshold value are obtained, and whether the time difference is normal A time difference determination step for determining whether it is abnormal, acquiring the signal for each predetermined scanning plane period, obtaining a time average of the signal level difference from the signal level difference of the signal between a plurality of different scanning planes, and further A calculation step for obtaining a square sum of time averages based on a time average value, an imaging target extraction determination step for determining whether or not an imaging target is detected based on a calculation result of the calculation step, and A determination step for determining detection of the light source or the object irradiated to the light source based on a calculation result; the time difference determination step; In response to the determination result of step, the above-mentioned time difference is abnormal, if detecting the imaging object in the imaging target extraction determination step, the signal transfer of updating the threshold value of the time difference determining step the threshold where the time difference determining step is determined An error determination step.

According to a third aspect of the present invention, a light source or a subject irradiated with the light source is imaged by an imaging device, and a signal transmitted from an imaging unit that transmits a signal acquired from the imaging device through a first interface is received. Signal processing for performing signal processing on the received signal to detect the state of the subject, and the signal processing includes receiving processing for receiving the signal transmitted in the imaging step by the second interface, and the second processing described above. A reception buffer process for storing a signal received by the interface and outputting it when it is determined as data; and a signal extraction process for extracting a signal used for signal processing for detecting the state of the subject from the signal output by the reception buffer process; The time difference for calculating the time difference between the current frame and the previous frame when the signal from the reception buffer process is received in the signal extraction process Obtaining a difference value between the acquisition process, the time difference acquired in the time difference acquisition process and a preset threshold, and determining whether the time difference is normal or abnormal; and for each predetermined scanning plane cycle Arithmetic processing for acquiring the above signal, obtaining a time average of the signal level difference from the signal level difference of the signal between a plurality of different scanning planes, and further obtaining a sum of squares of the time average based on the value of the time average And an imaging target extraction determination process for determining whether or not an imaging target is detected based on the calculation result of the calculation process, and a determination for determining detection of the light source or a subject irradiated to the light source based on the calculation result of the calculation process In response to the determination result of the process and the time difference determination process and the imaging target extraction determination process, when the time difference is abnormal and the imaging target is detected in the imaging target extraction determination process, The threshold between difference determination process is a program for executing the signal transfer error determination process of updating the threshold value the time difference determination processing is determined, the signal processing of the detection system including a computer.

  According to the present invention, it is possible not only to detect the state of the light source or the object irradiated to the light source, but also to prevent erroneous determination in the determination unit when a problem occurs in video data transfer.

It is a figure which shows the structural example of the detection system which concerns on embodiment of this invention. It is a figure which shows the result of having performed the setting of a threshold value, and the determination of the state of data transfer using the determination result in a frame time difference determination part and an imaging target extraction determination part in the video signal transfer error determination part which concerns on this embodiment. It is a figure which shows an example for demonstrating the structure of CCD which concerns on this embodiment. It is a figure for demonstrating the time series of CCD of FIG. It is a figure which shows the example of an arrangement | sequence of the pixel of a single plate complementary color filter type CCD. It is a figure which shows an example of the combination of the color signal in odd field OFD and even field EFD. It is a timing chart for demonstrating the signal processing method of the luminance signal in the 1st calculating part 135 and the 2nd calculating part 136 which concern on this embodiment. It is a figure which shows the waveform W6 about the frequency component contained in a light source. It is a figure which shows an example of the relationship between duty ratio D and square sum SACBD. It is a flowchart figure for demonstrating a series of operation | movement outline | summary of the detection system which concerns on this embodiment. It is a figure which shows the other structural example of the detection system which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example of a detection system according to an embodiment of the present invention.

  The detection system 10 includes a light source 11, an imaging unit 12, and a signal processing unit 13.

  The light source 11 illuminates the subject with a predetermined luminance. The luminance of the light source 11 is variable, and the luminance within the charge accumulation time of the imaging element included in the imaging device 12 changes at a period 4n times the field period of the imaging device 12. Here, n = 1, 2, 3,.

The imaging unit 12 includes an imaging device 121 and a first USB interface (I / F) 122.
The imaging device 121 used in the detection system 10 according to the present embodiment employs an imaging device having the following specifications.
As an example of the imaging device constituting the imaging apparatus 121, a single-plate complementary color filter and a field storage type interline transfer CCD (Charge Coupled Device) image sensor (hereinafter simply referred to as a CCD) are used.
As an example, the television system of the imaging device 12 employs the NTSC system, the scanning system employs interlace, the scanning frequency is 15.734 KHz, and the vertical frequency is 59.94 Hz.
The imaging device 121 having such a configuration images the subject illuminated by the light source 11, converts the analog signal into a digital signal, and outputs the digital signal to the signal processing unit 13 through the USB I / F 122.
Although a CCD image sensor is illustrated here, a CMOS image sensor is also applicable.

In the imaging unit 12, the imaging apparatus 121 performs AD conversion on the output signal from the imaging element, and transmits this digital signal to the outside using, for example, isochronous transfer of the USB I / F 122.
In isochronous transfer, a minimum data transfer amount per certain time is guaranteed, and image data is transferred at a prescribed frame rate in a video signal. However, if the transfer fails, the data is not retransmitted.

The signal processing unit 13 includes a second USB I / F 131 connected to the imaging unit 12, a reception data buffer unit 132, a luminance signal extraction unit 133, a frame time difference acquisition unit 134, a first calculation unit (A) 135, a second A calculation unit (B) 136 and a calculation processing unit (time average square sum calculation processing unit) 137 are included.
The first calculation unit 135, the second calculation unit 136, and the calculation processing unit 137 form a calculation unit.
Furthermore, the signal processing unit 13 includes a frame time difference determination unit 138, an imaging target extraction determination unit 139, a video signal transfer error determination unit 140, a storage unit 141, and a determination unit 142.

The USB I / F 131 of the signal processing unit 13 receives the signal output from the imaging unit 12 and stores the information in the reception data buffer unit 132.
When the data for one frame is stored and determined as data, the reception data buffer unit 132 outputs the stored information to the luminance signal extraction unit 133.

The luminance signal extraction unit 133 extracts a luminance signal from the input reception signal and outputs the luminance signal to the frame time difference acquisition unit 134.
The luminance signal extracted by the luminance signal extraction unit 133 is adjusted to a signal level optimized for calculation.
The signal level needs to be a signal level that does not overflow in the output values of the first calculation unit 135, the second calculation unit 136, and the calculation processing unit 137. Therefore, the luminance signal extraction unit 133 includes a circuit that adjusts the luminance signal level.
The adjustment value of the luminance signal level may have a table capable of mode switching when there are several modes. The mode may be a video signal standard such as NTSC or PAL, or a frequency mode of the imaging apparatus.

The frame time difference acquisition unit 134 calculates a time difference between the current frame and the previous frame when the luminance signal extraction unit 133 receives data from the reception data buffer unit 132.
In the frame time difference acquisition unit 134, since a frame rate is defined in the video signal, a constant value is normally calculated.
On the other hand, if a transmission / reception failure has occurred in the data transfer between the USB I / F 122 of the imaging unit 12 and the USB I / F 131 of the signal processing unit 13, there is a problem in storing data in the reception data buffer unit 132. As a result, the timing for determining the data for one frame is shifted, so that this time difference value takes an abnormal value.
The frame time difference acquisition unit 134 outputs the acquired time difference data to the first calculation unit 135 and the second calculation unit 136 together with the luminance signal.

The first calculation unit 135 calculates the time average of the level difference of the luminance signal in the m-th and (m + 2) -th fields in the same region of the image sensor for the input luminance signal.
The output result A of the first calculation unit 135 is output to the calculation processing unit 137.
The second calculation unit 136 obtains a time average of the level difference of the luminance signal in the (m + 1) th and (m + 3) th fields in the same region of the input luminance signal.
The output result B of the second calculation unit 136 is output to the calculation processing unit 137.
Details of operations of the first calculation unit 135 and the second calculation unit 136 will be described later.

The output result A and the output result B output from the first calculation unit 135 and the second calculation unit 136, respectively, are input to the calculation processing unit 137. The arithmetic processing unit 137 obtains a detected value of the frequency component of the subject.
The arithmetic processing unit 137 obtains a square sum value (A ² + B ² ) as a detection value of the output result, and outputs it to the frame time difference determination unit 138 and the imaging target extraction determination unit 139.

The frame time difference determination unit 138 uses the time difference calculated by the frame time difference acquisition unit 134 and determines that the frame time difference is normal when the value is equal to or less than a preset threshold value. The frame time difference determination unit 138 obtains a difference value between the time difference calculated by the frame time difference acquisition unit 134 and the threshold value.
The frame time difference determination unit 138 determines that an abnormality has occurred in the frame time difference when the time difference value calculated by the frame time difference acquisition unit 134 exceeds the threshold value.

Here, in determining the state of data transfer, the determination can be made by using the above-described frame time difference.
However, in reality, this time difference varies depending on the device or system to be used, and it is necessary to set a threshold value suitable for each environment.
Therefore, in the present embodiment, in addition to the above-described determination of the frame time difference, the following imaging target extraction determination unit 139 performs determination, and using these two determination results, optimal threshold value setting and video signal data transfer are performed. The state is determined.

The imaging target extraction determination unit 139 determines whether the imaging target is detected based on the calculation result in the calculation processing unit 137.
The imaging target extraction determination unit 139 determines that the imaging target is extracted if the calculation value of the arithmetic processing unit 137 is within the allowable range, and determines that the imaging target is not extracted if the calculation value is outside the range.
If an abnormality occurs in data transfer at this time, sampling is performed at a timing different from the specified frame rate in the video signal in the state detection of the imaging target, so that the arithmetic processing unit 137 deviates from the allowable range. An abnormal value is calculated, and it is determined that the imaging target is not extracted.

  The determination result of the frame time difference determination unit 138 and the determination result of the imaging target extraction determination unit 139 are supplied to the video signal transfer error determination unit 140.

  The video signal transfer error determination unit 140 uses the determination results in the frame time difference determination unit 138 and the imaging target extraction determination unit 139 to set a threshold and determine the data transfer state, and the results are as shown in FIG. .

  That is, FIG. 2 shows the video signal transfer error determination unit 140 according to the present embodiment, which uses the determination results in the frame time difference determination unit 138 and the imaging target extraction determination unit 139 to set a threshold and determine the data transfer state. It is a figure which shows the result.

As shown in FIG. 2, when the frame time difference is normal and the imaging target extraction is normal, the transfer of the video signal is normal and the threshold value is not updated.
When the frame time difference is abnormal and the imaging target extraction is normal, the transfer of the video signal is normal. Therefore, the threshold value of the frame time difference determination unit 138 is updated to the difference value calculated by the frame time difference determination unit 138.
When the frame time difference is normal and the imaging target extraction is abnormal, the transfer of the video signal is normal and the threshold value is not updated.
If the frame time difference is abnormal and the imaging target extraction is abnormal, the video signal transfer is abnormal and the threshold value is not updated.

When the transfer of the video signal is normally performed based on the determination result of FIG. 2, the information stored in the storage unit 141 is updated to the calculation result of the calculation processing unit 137.
If a video signal transfer error has occurred, the information in the storage unit 141 is not updated.
As a result, when the determination unit 142 determines the detection of the light source or the subject irradiated with the light source, the determination based on the information in the storage unit 141 prevents the erroneous detection of the state due to the abnormal data transfer of the video signal. Can do.
Further, in determining the transfer state, an optimum threshold can be automatically determined regardless of the device or system configuration.

Based on the value of the sum of squares of the whole image (A ² + B ² ) that is the calculation result of the calculation processing unit 137, the determination unit 142 determines whether the subject imaged by the imaging device 121 is stationary or moving, for example. Determine. Details of the operation of the determination unit 142 will be described later.

Next, other configurations and functions of the detection system according to the present embodiment will be described in detail.
First, the configuration of the CCD of the imaging device 12 according to the present embodiment will be described.

  FIG. 3 is a view showing an example for explaining the structure of the CCD according to the present embodiment.

The CCD 20 in FIG. 3 is an interline transfer, and includes a photodiode PD21, a vertical transfer CCD22, a horizontal transfer CCD23, and an amplifier 24.
The photodiodes PD21 are arranged in a matrix. The photodiodes PD21 arranged in the vertical line direction are connected to a vertical transfer CCD 22 for transferring charges for each column. The end of each vertical transfer CCD 22 is connected to a horizontal transfer CCD 23 that transfers charges to the amplifier. An amplifier 24 is connected to the output side of the horizontal transfer CCD 23.

The video scanning method is interlaced, and one screen is interlaced scanning, and is composed of an odd field and an even field.
First, light enters the photodiode PD21, and charges are accumulated in the photodiode PD21 during the charge accumulation time. During this time, the photodiode PD21 and the vertical transfer CCD 22 are disconnected.
When the charge accumulation time ends, the photodiode PD21 and the vertical transfer CCD 22 are brought into conduction, and the accumulated charge is transferred to the vertical transfer CCD 22. Immediately after this, the photodiode PD21 and the vertical transfer CCD 22 are disconnected, and the next charge accumulation is started in the photodiode PD21. The charges transferred to the vertical transfer CCD 22 are transferred to the horizontal transfer CCD 23 for each horizontal line and input to the amplifier 24.
The frequency until the charge is transferred from the vertical transfer CCD 22 to the horizontal transfer CCD 23 for each horizontal line is the horizontal scanning frequency of the CCD 20 of 15.734 KHz. When all the charges of the vertical transfer CCD 22 are transferred to the horizontal transfer CCD 23, the vertical transfer CCD 22 and the photodiode PD 21 are again brought into conduction, and the charges of the photodiode PD 21 are transferred to the vertical transfer CCD 22. In the case of a field accumulation CCD, charges are accumulated in the photodiode PD21 by photoelectric conversion, and the transfer frequency until this charge is transferred from the photodiode PD21 to the vertical transfer CCD 22 is 59.94 Hz.

  FIG. 4 is a diagram for explaining the time series of the CCD 20 of FIG.

As shown in FIG. 4, the required time until charge is accumulated in the photodiode PD21 by photoelectric conversion is ΔT1, and the required time until the charge is transferred from the photodiode PD21 to the vertical transfer CCD 22 is ΔT2.
As can be seen from FIG. 4, the light energy incident on the CCD 20 is sampled at the charge accumulation period ΔT = ΔT1 + ΔT2 = (1 / 59.94) seconds while being integrated during the charge accumulation time ΔT1.

As shown in FIG. 1, a luminance signal is extracted by a luminance signal extraction unit 133 from a signal of a captured image captured by the imaging device 121, and the luminance signal is output from the first calculation unit 135 and the second calculation unit. 136 is input.
Here, a method for reading out pixels from the CCD 20 (see FIG. 3) according to the present embodiment will be described.

FIG. 5 is a diagram showing an arrangement example of pixels of a single-plate complementary color filter type CCD.
FIG. 6 is a diagram illustrating an example of a combination of color signals in the odd field OFD and the even field EFD.

The color filter of the pixel is composed of Ye (yellow), Cy (cyan), Mg (magenta), and G (green), and has an arrangement as shown in FIG. Pixels are read by adding the upper and lower pixels. The combination to be added is shifted by one column between the odd field OFD and the even field EFD. Specifically, in the n-th line of the odd field OFD, (C11 + C21), (C12 + C22), (C13 + C23), (C14 + C24), (C15 + C25),... In the n-line of the even field EFD, (C21 + C31), (C22 + C32), (C23 + C33), (C24 + C34), (C25 + C35),...
Therefore, color signals are output in the odd field OFD and the even field EFD as shown in FIG.
In any case, the same color pattern of a combination of Ye, Cy, Mg, and G is repeated in a two-pixel cycle.
In other words, the color signal appears superimposed on a frequency of two pixel periods or more. Therefore, if this color signal is passed through a low-pass filter having a cutoff frequency of two pixel periods, the color signal is lost and only a luminance signal is obtained.
Therefore, luminance information is sampled at a cycle of two pixels.

A projection region REG illustrated by a circle in FIG. 5 shows a state in which an image of a subject by a light source is projected. It is assumed that the pixels C35, C36, C45, C46, C55, and C56 are completely in the projection region REG and are uniformly irradiated with light.
In the odd field OFD, the luminance information is read by a combination of C35, C36, C45, and C46 of the horizontal line (n + 1), and in the even field EFD, by a combination of C45, C46, C55, and C56 of the (n + 1) line of the horizontal line. Is done.

  As described above, the luminance signal among the signals from the imaging device 121 is input to the first calculation unit 135 and the second calculation unit 136. The luminance signal is input to the first calculation unit 135 and the second calculation unit 136, and a predetermined process is performed.

  Next, a luminance signal processing method performed by the first calculation unit 135 and the second calculation unit 136 will be described with reference to FIG.

  FIG. 7 is a timing chart for explaining the signal processing method of the luminance signal in the first calculation unit 135 and the second calculation unit 136 according to the present embodiment.

FIG. 7A is a diagram showing interline scanning of the imaging apparatus 121, and shows a state of either the even field EFD or the odd field OFD. FIGS. 7B to 7E respectively show waveforms W1, W2, W3, and W4 that represent temporal changes in the luminance signal level processed by the arithmetic unit, and FIG. 7F shows a sine wave that changes at a constant period. It is a figure which shows the waveform W5.
The odd field OFD and even field EFD scan one frame. That is, as shown in FIG. 7A, one frame is scanned by A and B and C and D.
In the following description, f represents frequency, t represents time, θ represents phase difference, and ω satisfies (ω = 2πf). Note that π is the circumference ratio.

Waveforms W1 and W2 illustrated in FIGS. 7B and 7C are waveforms for deriving functions of waveforms indicated by a waveform W3 and a waveform W4 described later.
A waveform W3 illustrated in FIG. 7D shows a time development distribution of a function for obtaining the level difference AC when the luminance level difference between the luminance signal of the A field and the luminance signal of the C field is the level difference AC. Indicates.
A waveform W4 shown in FIG. 7E is a function time for obtaining the level difference BD when the difference in luminance level between the luminance signal of the B field and the luminance signal of the D field is the level difference BD. The development distribution is shown.
Waveform W3 is derived from waveform W1 and waveform W2, and is obtained by adding waveform W1 and waveform W2 and dividing by two. Waveform W4 is derived from waveform W1 and waveform W2, and is obtained by subtracting waveform W1 from waveform W2 and dividing it by 2.

At this time, a time average SAC that is a first time average of the level difference AC is calculated by the first calculation unit 135 shown in FIG. In addition, a time average SBD that is a second time average of the level difference BD is calculated by the second calculation unit 136.
Specifically, the time average SAC is calculated from the luminance level difference AC between the A field and the C field by a combination of C35, C36, C45, and C46.
Similarly, the time average SBD is calculated from the level difference BD between the B field and the D field by a combination of C45, C46, C55, and C56.

The calculation method of the time average is described.
The time average SAC of the level difference AC between the A field and the C field is calculated by multiplying the waveform W1 by the waveform W3 shown in FIG. 7D.
Similarly, the time average SBD of the level difference BD between the B field and the D field is shown in FIG. 7E as a waveform representing the time change of light irradiated to the pixel by the combination of C45, C46, C55, and C56. The time average SBD is calculated by multiplying the waveform W4 shown.

First, a method for calculating the time average SAC will be specifically described.
The waveform W3 is expressed by a mathematical expression. First, the waveform W1 and the waveform W2 can be expressed by the following Fourier series.

  Here, it is assumed that the waveforms W1 and W2 have the same period f2. From equation (1) and equation (2), waveform W3 can be expressed as equation (4).

  By the way, the waveform W5 having the period f1 shown in FIG. 7F can be represented by a sine wave as shown in the equation (5).

  When the waveform W3 represented by the equation (4) is multiplied by the sine wave W5 represented by the equation (5), the equation (7) is obtained.

Next, the time average of the equation (7) from time 0 to time T is taken. Of the terms shown on the right side of the equation (7), the term including time t is an AC signal, so the time average is zero.
Therefore, only when (ω ₁ − (2n−1) ω ₂ = 0), the constant cos θ ₁ and the constant sin θ ₁ remain, and the time average SAC is expressed by the equation (8).

  In this way, the time average SAC is obtained by the first calculation unit 135. Similarly, the time average SBD is obtained by the second calculation unit 136 and is expressed by equation (9).

Now, the sum of squares (S _AC ² + S _BD ² ) of the time average SAC and SBD expressed by the equations (8) and (9) is expressed by the equation (10).

From this equation (10), when the frequency component (f ₁ = (2n−1) f ₂ ) is included in the light incident on the CCD 20 (see FIG. 9), it is expressed by the equation (10). Waveform components are detected.

Next, frequency components included in the light source 11 will be considered.
FIG. 8 is a diagram illustrating a waveform W <b> 6 for the frequency component included in the light source 11. The light source 11 emits light at the luminance level L1 at the frequency f3 for the time τ.

This waveform W6 is developed into a Fourier series. The waveform W6 is a periodic function of a period (T ₃ = 1 / f ₃ ). When (ω ₃ = 2πf ₃ ), the waveform W6 is represented by a general formula of Fourier series as shown in Expression (11).

The coefficients a ₀ , a _n , and b _{n in the} equation (11) are obtained from the waveform W6 as in the equations (12) to (14).

  Therefore, the Fourier series of the waveform W6 is expressed by equation (15).

Therefore, when the blinking period of the light source 11 is set to four times the field period, that is, when (f ₃ = f ₂ ), the frequency coincides with the odd term from the expressions (7) and (15), and the time average SAC And the sum of the squares of SBD is given by equation (16).

  When the duty ratio of lighting of the light source 11 is D, it is expressed by equation (17).

    Therefore, the square sum SACBD of the time average SAC and SBD expressed by the equation (16) is expressed by the equation (18) when the equation (17) is used.

  By the way, the term (19) on the right side of the following equation (18) converges.

  The term (19) on the right side of the equation (18) takes a value as shown in Table 1 with respect to the duty ratio D. Table 1 is shown below.

  Based on Table 1, when the duty ratio D is taken on the horizontal axis and the square sum SACBD of the time average SAC and SBD is taken on the vertical axis, the relationship between the duty ratio D and the square sum SACBD is as shown in FIG. become.

From FIG. 9, it can be seen that the square sum SACBD is maximized at the duty ratio D = 0.5.
Therefore, the sum of squares SACBD expressed by the equation (18) is expressed by the following equation.

[Equation 15]
S _AC ² + S _BD ² = 0.08333L ₁ ² (20)

  As shown in the equation (20), the arithmetic processing unit 137 detects the luminance of the light source 11 (see FIG. 1), and the detection result (square sum SACBD) is used by the determination unit 142 to determine the state of the subject. It is done.

The light source 11 according to the present embodiment does not depend on a specific light source. Therefore, it is considered whether the luminance can be detected for other light sources.
Incandescent light bulbs and fluorescent lamps that are widely used as light sources are 100 Hz in regions where the power supply frequency is 50 Hz and 120 Hz in regions where 60 Hz. The field frequency of an NTSC television is 59.94 Hz, and the field frequency of a monitor used for a personal computer is 60 Hz or more so as not to flicker.

  The field frequency of the NTSC television is 59.94 Hz, and if the light source is caused to emit light at a quarter period, the frequency f2 of the luminance level difference is as follows.

[Equation 16]
f ₂ = 59.94 / 4 = 14.985 Hz (21)

  From the equations (7) and (15), the signal component is detected when the odd multiple of the frequency f2 matches the integer multiple of the frequency f3 of the light source 11.

  Table 2 is a table of values indicating the relationship between the light emission frequency of different light sources and the frequency in the difference in luminance signal level.

F1 in Table 2 is the frequency of the sine wave of Formula (5), f2 is the frequency shown in Formula (21), and f3 is the frequency of the light source 11, the frequency of illumination in the 50 Hz region, and the illumination in the 60 Hz region, respectively. Frequency, NTSC television field frequency, and monitor field frequency used in personal computers.
According to Table 2, up to m = 30, f3 is 100, 120, 59.94 Hz, and there is nothing that satisfies (n × f ₃ = (2m−1) × f ₂ ).

For example, regarding a monitor of a personal computer, if the field frequency is 60 Hz or more, there is a monitor that is scanned at 74.925 Hz as a possibility that the largest output is detected in the signal processing output of the detection system 10. When
That is, when f ₃ = 74.925 Hz, as shown in Table 2, (5 × f ₂ ), (15 × f ₂ ), (25 × f ₂ ), and (1 × f ₃ ), ( 3 × f ₃ ), (5 × f ₃ ). The signal level detected at this time is expressed by the following equation.

したがって、（２２）式で示される信号レベルは光源１１の１／２５のレベルであり、図１に図示していない信号処理で別に除去できる。

Therefore, the signal level expressed by the equation (22) is 1/25 of that of the light source 11 and can be removed separately by signal processing not shown in FIG.

  As described above, the detection system 10 detects the state of the light source or the subject irradiated on the light source without depending on the light emission frequency of the light source.

  Hereinafter, a series of operations of the detection system according to the present embodiment will be described with reference to FIG.

  FIG. 10 is a flowchart for explaining a series of operation outlines of the detection system according to the present embodiment.

For example, the luminance of the light source 11 within the charge accumulation time of the imaging device 121 is changed by 4n times the field period of the imaging device 121.
In the imaging unit 12, the output signal from the imaging element is AD converted in the imaging device 121 (ST 1), and this digital signal is transmitted to the external signal processing unit 13 using isochronous transfer of the USB I / F 122.
In isochronous transfer, a minimum data transfer amount per certain time is guaranteed, and image data is transferred at a prescribed frame rate in a video signal. However, if the transfer fails, the data is not retransmitted.
In the signal processing unit 13, the digital signal output from the imaging unit 12 by the USB I / F 131 is received (ST2), and the information is stored in the reception data buffer unit 132 (ST3).
When data for one frame is stored and determined as data, the information is output to the luminance signal extraction unit 113 (ST4).

The luminance signal extraction unit 133 extracts a luminance signal from the received data, and the luminance signal is adjusted to a signal level optimized for calculation (ST5).
Since the signal level needs to be a signal level that does not overflow in the first calculation unit 135, the second calculation unit 136, and the calculation processing unit 137, the luminance signal extraction unit 131 adjusts the luminance signal level, It is output to the frame time difference acquisition unit 134.
The frame time difference acquisition unit 134 calculates a time difference between the current frame and the previous frame when the luminance signal extraction unit 133 receives data from the reception data buffer unit 132 (ST6).
Since a frame rate is defined for a video signal, a constant value is normally calculated. However, a transmission / reception failure has occurred for some reason in data transfer between the imaging unit 12 and the USB I / F of the signal processing unit 13. In this case, the reception data buffer unit 132 has a problem in storing data, and the timing for determining the data for one frame is shifted. For this reason, this time difference value may take an abnormal value.
Next, the luminance signal is input to the first calculation unit 135 and the second calculation unit 136.
The first arithmetic unit 135 obtains the time average SAC of the level difference AC of the luminance signal level in the projection area REG of the mth and (m + 2) th fields. Further, the second arithmetic unit 136 obtains the time average SBD of the level difference BD of the luminance signal level in the projection area REG of the (m + 1) th and (m + 3) th fields.
These time averages SAC and SBD are output to the arithmetic processing unit 137 (ST7).
Next, the arithmetic processing unit 137 obtains the square sum SACBD of the time average SAC and SBD (ST8), and the result is output to the frame time difference determination unit 138 and the imaging target extraction determination unit 139.

The frame time difference determination unit 138 uses the time difference calculated by the frame time difference acquisition unit 134 and determines that the frame time difference is normal when the value is equal to or less than the threshold value. On the other hand, if the threshold value is exceeded, it is determined that an abnormality has occurred in the frame time difference (ST9).
Here, in determining the state of data transfer, the determination can be made by using the above-described frame time difference.
However, in reality, this time difference varies depending on the device or system to be used, and it is necessary to set a threshold value suitable for each environment. Therefore, in addition to the above-described frame time difference determination, the following imaging target extraction determination unit 139 performs determination, and using these two determination results, the optimum threshold value setting and the state determination in the video signal data transfer can be performed. Done.
The imaging target extraction determination unit 139 determines whether the imaging target is detected from the calculation result of the calculation processing unit 137 (ST10).
If the calculation value of the arithmetic processing unit 137 is within the allowable range, it is determined that the imaging target is extracted, and if it is out of the range, it is determined that the imaging target is not extracted.
If an abnormality occurs in data transfer at this time, sampling is performed at a timing different from the specified frame rate in the video signal in the state detection of the imaging target, so that the arithmetic processing unit 137 deviates from the allowable range. An abnormal value is calculated, and it is determined that the imaging target is not extracted.
Next, the video signal transfer error determination unit 140 uses the determination results of the frame time difference determination unit 138 and the imaging target extraction determination unit 139 to set a threshold and determine the data transfer state.
If the transfer of the video signal is abnormal according to the determination result and the imaging target is normally extracted, the threshold value of the frame time difference determination unit 138 is updated to the difference value calculated by the frame time difference determination unit 138 (ST11).
If the video signal is normally transferred based on the determination result, the information stored in the storage unit 141 is updated to the calculation result of the calculation processing unit 137 (ST12). When a video signal transfer error has occurred, the information in the storage unit is not updated.
As a result, when the determination unit determines the detection of the light source or the subject irradiated with the light source, the determination based on the information in the storage unit 141 can prevent erroneous detection of a state due to abnormal data transfer of the video signal. it can.
Further, in determining the transfer state, an optimum threshold can be automatically determined regardless of the device or system configuration.
Then, in the determination unit 142, based on the value of the sum of squares of the entire image (A ² + B ² ) that is the calculation result of the calculation processing unit 137, the subject imaged by the imaging device 121 is, for example, still or moving. Is determined (ST13).

  Through the above processing, in detecting the state of the light source or the object irradiated to the light source, it is possible to detect a stable subject by detecting an abnormality in video signal data transfer with high accuracy and performing correction processing.

  As described above, according to the present embodiment, the background noise of the captured image can be removed, and the state of the subject irradiated with the light source or the light source can be detected.

According to the detection system according to the present embodiment, in the video signal transfer error determination, the video signal state determination process and the threshold value setting process are separated, and an arbitrary processing time or processing frequency threshold value setting process is performed, thereby improving reliability. Determination with a high threshold value and subject detection with a low system load.
Moreover, since it does not depend on the light source, it does not depend on the specifications of the imaging device, and for example, a generally available camera can be used as the imaging device of the present detection system.

Furthermore, this detection system uses a plurality of light sources and can transmit signals to the signal processing unit in parallel.
Alternatively, it is possible to provide a plurality of light source colors and perform wavelength multiplexing transmission of signals.
It is also possible to process the signal by appropriately blinking the light source.

In the configuration of FIG. 1, a storage device may be connected to the frame time difference determination unit 138 so that the storage contents of this storage device are not erased when the system is terminated.
In this case, when the detection system 10 in FIG. 1 is once terminated and activated again, a threshold value before termination is stored using, for example, a non-volatile memory, and the threshold value is read out at the time of rebooting and set. In this way, the update information of the threshold value at the previous operation can be inherited.

[Second Embodiment]
Next, a second embodiment according to the present invention will be described.
FIG. 11 is a diagram illustrating another configuration example of the detection system according to the embodiment of the present invention.

This detection system 10A is different from the detection system 10 of FIG. 1 in the configuration after the arithmetic processing unit 137 of the signal processing unit 13A.
In the signal processing unit 13A of the detection system 10A, a frame time difference determination unit 138A is disposed after the arithmetic processing unit 137, and the first processing unit 143 and the second processing unit 144 are connected in parallel to the output thereof. Yes.
In the first processing unit 143, an imaging target extraction determination unit 139, a first video signal transfer error determination unit 140-1, and a threshold storage unit 145 are cascade-connected to the output of the frame time difference determination unit 138A.
In the second processing unit 144, a second video signal transfer error determination unit 140-2, a storage unit 141, and a determination unit 142 are cascade-connected to the output of the frame time difference determination unit 138A.
The first transfer error determination unit 140-2 is given the threshold value of the threshold value storage unit 145.

In this configuration, the frame time difference determination unit 138A performs mode determination in addition to the processing of the frame time difference determination unit 138 in FIG. 1. If the mode is set to a threshold value, the first processing unit 143 determines the mode. The subsequent processing is transferred to the second processing unit 144.
The mode can be set arbitrarily.
The first video signal transfer error determination unit 140-1 performs only the threshold setting process in FIG. 2, and the second video signal transfer error determination unit 140-2 performs only the data transfer state determination process in FIG.
According to this configuration, in the video signal transfer error determination, the video signal state determination process and the threshold value setting process are separated, and an arbitrary processing time or processing frequency threshold value setting process is performed. In addition, it is possible to perform subject detection with a small system load.

[Third Embodiment]
Next, a third embodiment according to the present invention will be described.

In the third embodiment, the field accumulation / interlace imaging device 121 according to the first embodiment is replaced with a frame accumulation / interlace imaging device. At the same time, the luminance change period of the light source 11 according to the first embodiment is changed from 4n times the field period to 4n times the frame period. Accompanying this change in the change period, the luminance signal acquisition period is changed from every n fields to every 2n fields.
As described above, by changing the luminance change cycle of the light source 11 and the luminance signal acquisition cycle, the same effects as those of the first embodiment can be obtained.
Therefore, according to the present embodiment, the background noise of the captured image can be removed, the state of the light source or the subject irradiated with the light source can be detected, and even when there are a plurality of subjects, detection can be performed for each subject.

[Fourth Embodiment]
Next, a fourth embodiment according to the present invention will be described.

In the fourth embodiment, the field storage and interlace scanning imaging device 121 according to the first embodiment is replaced with a frame storage and non-interlace scanning imaging device.
As described above, even when a non-interlaced scanning imaging apparatus is used, the same effect as that of the first embodiment can be obtained.
Therefore, according to the present embodiment, the background noise of the captured image can be removed, the state of the light source or the subject irradiated with the light source can be detected, and even when there are a plurality of subjects, detection can be performed for each subject.

Note that the method described above in detail can be formed as a program according to the above-described procedure and executed by a computer such as a CPU.
Further, such a program can be configured to be accessed by a recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a floppy (registered trademark) disk, or the like, and to execute the program by a computer in which the recording medium is set.

  DESCRIPTION OF SYMBOLS 10,10A ... Detection system, 11 ... Light source, 12 ... Imaging part, 121 ... Imaging device, 122 ... USB interface (I / F), 13 ... Signal processing part, 131 ... USB I / F, 132 ... Reception data buffer section 132, 133 ... Luminance signal extraction section, 134 ... Frame time difference acquisition section, 135 ... First calculation section (A), 136 .. second calculation unit (B), 137... Calculation processing unit (time average square sum calculation processing unit), 138, 138A... Frame time difference determination unit, 139. DESCRIPTION OF SYMBOLS 1140 ... Video signal transfer error determination part, 141 ... Memory | storage part, 142 ... Determination part, 143 ... 1st process part, 144 ... 2nd process part.

Claims (7)

A light source;
An imaging unit including an imaging device that images the light source or the object irradiated by the light source, and a first interface that transmits a signal acquired from the imaging element;
A signal processing unit that receives a signal transmitted from the imaging unit and performs signal processing on the received signal to detect the state of the subject,
The signal processor is
A second interface for receiving a signal transmitted from the imaging unit;
A reception buffer unit that stores a signal received by the second interface and outputs the signal when it is determined as data;
A signal extraction unit that extracts a signal used for signal processing for detecting the state of the subject from a signal output from the reception buffer unit;
A time difference acquisition unit that calculates a time difference between the current frame and the previous frame when the signal extraction unit receives a signal from the reception buffer unit;
Obtaining a difference value between the time difference acquired by the time difference acquisition unit and a preset threshold, and determining whether the time difference is normal or abnormal; and
The signal is acquired from the signal extraction unit for each predetermined scanning plane period, a time average of the signal level difference is obtained from the signal level difference of the signal between a plurality of different scanning planes, and further, based on the time average value. An arithmetic unit for calculating a sum of squares of time average in
An imaging target extraction determination unit that determines whether an imaging target is detected based on a calculation result of the calculation unit;
A determination unit that determines detection of the light source or a subject irradiated to the light source based on a calculation result of the calculation unit;
In response to the determination results of the time difference determination unit and the imaging target extraction determination unit, when the time difference is abnormal and the imaging target extraction determination unit detects the imaging target, the time difference determination unit determines the threshold value of the time difference determination unit. A signal transfer error determination unit that updates the threshold value obtained by the unit. As information for determining the detection of the light source or the subject irradiated with the light source, a storage unit that stores the calculation result of the calculation unit,
The signal transfer error determination unit is
The detection system according to claim 1, wherein when the signal transfer is normally performed, the information stored in the storage unit is updated to a calculation result of the calculation unit. The signal transfer error determination unit is
The detection system according to claim 2, wherein the storage unit is not updated when a transfer error occurs. The imaging target extraction determination unit
The determination unit determines that the imaging target is extracted if the calculation value of the calculation unit is within the allowable range, and determines that the imaging target is not extracted if the calculation value is out of the range. Detection system. Upper Symbol time between difference determination unit in the subsequent stage of the operation portion is disposed, the first processing unit and second processing unit are connected in parallel with the output of those said time difference determining unit,
The first processing unit includes:
The output of the upper Symbol time between difference determination unit, the imaging target extraction judgment unit, transfer error determination section first signal, the threshold storage unit are cascaded,
The second processing unit includes:
The output on SL at between difference determination unit, a second signal transfer error determination unit, the storage unit, the determination unit is cascaded,
The first signal transfer error determination unit is given a threshold value of the threshold value storage unit,
In the threshold setting mode, the first processing unit performs processing,
The detection system according to claim 2 or 3 , wherein in the subject detection mode, processing is performed by the second processing unit. The image sensor captures an image of a light source or a subject illuminated by the light source, receives a signal transmitted from an imaging unit that transmits a signal acquired from the imaging device through a first interface, and performs signal processing on the received signal. A signal processing step for detecting the state of the subject
The signal processing step includes
A reception step of receiving the signal transmitted in the imaging step by the second interface;
A reception buffer step for storing a signal received by the second interface and outputting the signal when it is confirmed as data;
A signal extraction step of extracting a signal used for signal processing for detecting the state of the subject from the signal output in the reception buffer step;
A time difference obtaining step for calculating a time difference between the current frame and the previous frame at the time of receiving the signal from the reception buffer step in the signal extraction step;
Obtaining a difference value between the time difference acquired in the time difference acquisition step and a preset threshold, and determining whether the time difference is normal or abnormal; and
The signal is acquired for each predetermined scanning plane period, a time average of the signal level difference is obtained from the signal level difference of the signal between a plurality of different scanning planes, and the time average of 2 is obtained based on the time average value. A calculation step for obtaining a sum of multipliers;
An imaging target extraction determination step for determining whether an imaging target is detected based on the calculation result of the calculation step;
A determination step of determining detection of the light source or a subject irradiated to the light source based on a calculation result of the calculation step;
In response to the determination results of the time difference determination step and the imaging target extraction determination step, when the time difference is abnormal and the imaging target is detected in the imaging target extraction determination step, the threshold of the time difference determination step is determined as the time difference determination A signal transfer error determination step for updating the threshold to the threshold obtained by the step . The image sensor captures an image of a light source or a subject illuminated by the light source, receives a signal transmitted from an imaging unit that transmits a signal acquired from the imaging device through a first interface, and performs signal processing on the received signal. Signal processing to detect the state of the subject
The above signal processing
A reception process for receiving the signal transmitted in the imaging step by the second interface;
A reception buffer process for storing a signal received by the second interface and outputting it when it is determined as data;
A signal extraction process for extracting a signal used for signal processing for detecting the state of the subject from the signal output in the reception buffer process;
A time difference acquisition process for calculating a time difference between the current frame and the previous frame at the time of receiving a signal from the reception buffer process in the signal extraction process;
A time difference determination process for determining a difference value between the time difference acquired in the time difference acquisition process and a preset threshold value, and determining whether the time difference is normal or abnormal;
The signal is acquired for each predetermined scanning plane period, a time average of the signal level difference is obtained from the signal level difference of the signal between a plurality of different scanning planes, and the time average of 2 is obtained based on the time average value. An arithmetic process for calculating a sum of multipliers;
An imaging target extraction determination process for determining whether an imaging target is detected based on a calculation result of the calculation process;
A determination process for determining detection of the light source or a subject irradiated to the light source based on a calculation result of the calculation process;
In response to the determination result of the time difference determination process and the imaging target extraction determination process, when the time difference is abnormal and the imaging target is detected by the imaging target extraction determination process, the threshold of the time difference determination process is determined as the time difference determination A program for causing a computer to execute signal processing of a detection system, including signal transfer error determination processing for updating to a threshold obtained by processing .

Images