JP2001022943A - Device and method for image processing and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To know the position of a light emitting device at all times. SOLUTION: A video camera 101 executes normal imaging and panning imaging alternately. An image processing part 102 calculates the position of the device 11 on the basis of an image imaged by the normal imaging and the position of the device 11 and a flickering pattern, on the basis of an image imaged by the panning imaging. The part 102 stores the position of the device 11 and the flickering pattern, and corrects the value of the stored position of the device 11 on the basis of the position of the device 11 calculated from the image imaged by the normal imaging and the value of the stored position of the light emitting device, on the basis of the position of the device calculated from the image imaged by the panning imaging and the flickering pattern.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and a recording medium, and more particularly to an image processing apparatus and method for constantly outputting the position and blinking pattern of one or more light emitting devices, and a recording medium.

[0002]

2. Description of the Related Art FIG. 1 shows a conventional image processing system utilizing panning imaging. The position and state of the light emitting device 11 such as a robot (for example, forward state, backward state)
Is attached to the object to be detected, in a predetermined pattern that is specific to the object or corresponding to the state of the object,
This is a light emitting device that changes the luminance of a light spot over time.

A video camera 21 is provided with a panning drive device 2
The light-emitting device 1 is driven by the
The light spot emitted from 1 is imaged. Video camera 21
Is a change occurring over time of the luminance of the light spot of the light emitting device 11 obtained by the panning imaging (hereinafter, this is referred to as
The image data is simply output to the image processing device 23. The filter 21A includes the light emitting device 11
An optical filter for suppressing light other than the light spot emitted from the video camera 21 from entering the video camera 21.
Is provided before the lens unit (not shown).

[0004] The image processing device 23 performs predetermined processing on the image data supplied from the video camera 21, detects the position and blinking pattern of the light emitting device 11, and outputs the detection result to the display device 24. The display device 24 displays data supplied from the image processing device 23.

FIG. 2 shows the internal structure of the light emitting device 11. The light emitting diode blink control circuit 31 controls the blink pattern of the light emitting diode 32. The light emitting diode 32 has one end connected to the light emitting diode blinking control circuit 31 and the other end grounded, and emits light by a voltage applied by the light emitting diode blinking control circuit 31.

FIG. 3 shows the internal structure of the image processing apparatus 23 shown in FIG. The image processing device 23 includes an image capture unit 41 and an image processing unit 42. The image capture unit 41 performs predetermined processing on image data of a blinking pattern supplied from the video camera 21 to generate image data that allows the image processing unit 42 to execute predetermined image processing. Output to

[0007] The image processing section 42 includes an image capturing section 41.
Image data of a blinking pattern represented as a change in luminance corresponding to the passage of time supplied from
That is, the image data is converted into image data of a spatial pattern represented by the respective coordinates of the X axis and the Y axis (written in the frame memory), and the position of the light emitting device 11 and the blinking pattern are detected from the image data.

[0008] Here, the panning imaging performed by the video camera 21 will be described with reference to FIGS. Floating imaging means a CCD (charge-coup) built in the video camera 21.
This is an imaging method in which an output of a photodiode constituting each pixel is transferred to the vertical transfer CCD a plurality of times during the same field in an imaging device.

As shown in FIG. 4A, the light emitting diode 32 of the light emitting device 11 has 12 light emitting diodes during one field period.
During the HD (Horizontal Drive: horizontal scanning cycle), the light is turned off, and during the next 12 HD, the blinking pattern 1 is turned on by 2
In this case, the video camera 21 shoots the image, and as shown in FIG. 4B, the output of the light emitting diode constituting each pixel is transferred to the vertical transfer CCD once every 2 HD as shown in FIG. When the image is transferred, a plurality of outputs (lighting or extinguishing) of the light emitting diode 32 appear in the resulting image as shown in FIG. 4C. The light emitting diode 32 has a size of three lines on the imaging screen.

FIG. 4 output from the video camera 21.
When an image as shown in (C) is supplied to the image capture unit 41 of the image processing device 23, the image capture unit 41
Performs a capture process on the image to generate an image as shown in FIG. As described above, since the light emitting diodes 32 of the light emitting device 11 are represented by three lines, the image of each output partially overlaps the images of the previous and next outputs, and the luminance of the pixels of the captured image is gradually increased. Is changing.

In the line L8, the lighted image of the light emitting diode 32 is superimposed three times, so that the brightness D becomes the brightest. In the line L7, the lighted image is turned off twice and the light is turned off. Since the image in the state of being present is displayed once, the luminance C becomes lower than the luminance D, and in the line L6,
Since the image in the light-on state is displayed once and the image in the light-off state is displayed twice, the luminance B is lower than the luminance C, and the image in the light-off state is displayed three times on the line L5. Therefore, the luminance A is lower than the luminance B.

Further, when the image shown in FIG. 5 output from the image capture unit 41 is supplied to the image processing unit 42, the image processing unit 42 performs a binarization process on the image, and An image as shown in FIG. In this case, the brightness represented by the brightness A and the brightness B of the image shown in FIG.
Luminance L, and luminance represented by luminance C and luminance D,
The brightness is set to H. The pixel value of the pixel having the luminance H is “
The pixel value of the pixel having the luminance L is encoded to “0”.

As described above, the temporal change of the luminance twice during one field period is converted into two spatial changes by performing the panning imaging.

Next, the processing procedure of the image processing unit 42 of the image processing apparatus 23 for detecting the position of a luminescent spot from the image data converted into the spatial pattern obtained as described above will be described.

FIG. 7A shows a bright spot Li1 corresponding to the light emitting diode 32 of the light emitting device 11-1 and a bright spot Li2 corresponding to the light emitting diode 32 of the light emitting device 11-2 arranged as shown in FIG. , And an example in which image data of a blinking pattern of the bright spot Li3 corresponding to the light emitting diode 32 of the light emitting device 11-3 is converted into a spatial pattern. When determining the positions of the bright points Li1, Li2, and Li3, the image processing unit 42 of the image processing device 23 converts the pixel value “1” of the bright point arranged on the scanning line into a value as shown in FIG.
The sum is added in the direction in which the panning image is taken (in this case, the y-axis direction), a peak is determined from the addition, and the coordinate on the x-axis where the peak is located is set as the X coordinate of the bright spot on the display image. .
In the case of this display example, the X coordinates on the display image of the bright points Li1, Li2, and Li3 are X1, X2, and X3, respectively.

On the other hand, the y coordinate is the y point of the first luminescent spot.
The coordinate on the axis is the Y coordinate of the bright spot on the display image. In the case of this display example, since the panning imaging direction is the negative direction, the bright point 1, the bright point 2, and the bright point 3 are the points at which the light starts to be emitted first. Y1, Y2, and Y3, which are the coordinates of the bright point 3 on the y-axis, are bright points Li1, Li2, and Li3.
Is the Y coordinate.

As described above, the image processing unit 42
As shown in the figure, the X and Y coordinates of the bright points Li1, Li2, and Li3 on the display image, that is, (X1, Y1), (X
(2, Y2), (X3, Y3).

Next, the processing of the image processing section 42 of the image processing device 23 when detecting the blinking pattern of the bright spot will be described with reference to FIG.

FIG. 9A shows a spatial pattern of the luminescent spot Li1 shown in FIG. When detecting the state of the bright point Li1, the image processing unit 42 of the image processing device 23
Selects a bright point within a predetermined variation range (X1−ΔX <X <X1 + ΔX) with respect to the x coordinate X1, and X
The addition is performed in the axial direction, and the peak is obtained as shown in FIG. Next, the image processing unit 42 performs a binarization process on the obtained peak with a predetermined threshold to generate bit pattern information as shown in FIG. 9C. From the bit pattern (10101010) thus obtained, the bright spot L
The state of i1 is detected. In this case, the bright point Li1 is blinking in a blinking pattern such that 8-bit information is extracted.

As described above, the light-emitting device 11 emits a luminescent spot in a predetermined pattern, and the light-emitting device 11 detects the position and the light-emitting pattern of the light-emitting device 11 at the same time by flowing the image and processing the image.

[0021]

However, FIG.
As shown in the figure, the light emitting diode 32 of the light emitting device 11-1
When the bright point Li1 corresponding to the light emitting device 11-2 and the bright point Li2 corresponding to the light emitting diode 32 of the light emitting device 11-2 are located on a predetermined straight line that coincides with the direction of the video shooting of the video camera 21, the video camera 21 is controlled to flow. When the blinking patterns overlap on the captured image, the image processing system cannot detect each blinking pattern, and the light emitting device 11-1.
In addition, there is a problem that the light emitting device 11-2 cannot be specified.

The present invention has been made in view of such a situation, and has as its object to be able to always specify a plurality of light emitting devices 11 and always know the positions of the light emitting devices 11. .

[0023]

According to the first aspect of the present invention, there is provided an image processing apparatus comprising: an imaging control unit configured to control imaging of an imaging device so that normal imaging and floating imaging are alternately performed; A first calculating unit that calculates the position of the light emitting device based on the image captured by the image capturing device, and calculates the position of the light emitting device and the blinking pattern of the light emitting device based on the image captured by the image capturing device flowing The second calculating means, the storing means for storing the blinking pattern calculated by the second calculating means together with the position of the corresponding light emitting device, and the storing means based on the position of the light emitting device calculated by the first calculating means. First correcting means for correcting the position of the light emitting device stored in the means, and the position of the light emitting device stored in the storing means based on the position and blinking pattern of the light emitting device calculated by the second calculating means Correction means for correcting Characterized in that it comprises a.

[0024] The imaging control means can insert an identification signal into at least one of the predetermined lines of the normally taken image and the flow-taken image.

The imaging control means can control the imaging so that the imaging device performs normal imaging for a predetermined period before the panning imaging in the panning imaging period.

The imaging control means can control the imaging so that the imaging device performs normal imaging for a predetermined period after the panning imaging in the panning imaging period.

The image pickup control means can control the image pickup so that the image pickup apparatus performs the normal image pickup for a predetermined period during the period in which the shot image is output.

[0028] The second calculating means may calculate the blinking pattern of the light emitting device based on the position of the edge of the image shot and shot.

According to a seventh aspect of the present invention, in the image processing method, an imaging control step of controlling imaging of the imaging apparatus so that normal imaging and panning imaging are alternately performed, and an image captured by the imaging apparatus in normal imaging. The first to calculate the position of the light emitting device based on
Calculation step, a second calculation step of calculating the position of the light-emitting device and a blinking pattern of the light-emitting device based on the image captured by the imaging device in the panning imaging, and a blink calculated by the processing of the second calculation step. Based on the storage step of storing the pattern together with the position of the corresponding light-emitting device and the position of the light-emitting device calculated in the processing of the first calculation step, the position of the light-emitting device stored in the processing of the storage step is corrected. A first correction step, a second correction step of correcting the position of the light emitting device stored in the processing of the storage step based on the position and the blinking pattern of the light emitting device calculated in the processing of the second calculation step; It is characterized by including.

According to another aspect of the present invention, there is provided a program for recording on a recording medium, wherein an imaging control step of controlling imaging of the imaging device so that normal imaging and panning imaging are alternately performed; A first calculating step of calculating the position of the light emitting device based on the image, and a second calculating step of calculating the position of the light emitting device and a blinking pattern of the light emitting device based on the image taken by the image capturing device by panning A storage step of storing the blinking pattern calculated in the processing of the second calculation step together with the position of the corresponding light emitting device; and a storage step based on the position of the light emitting device calculated in the processing of the first calculation step. A first correction step of correcting the position of the light emitting device stored in the processing of step (b), and a storage step based on the position and blinking pattern of the light emitting device calculated in the processing of the second calculation step. Characterized in that it comprises a second correction step for correcting the position of the light-emitting device storing the processing.

An image processing apparatus according to claim 1, wherein
In the image processing method according to the above, and the recording medium according to the eighth aspect, the imaging of the imaging device is controlled so that the normal imaging and the flow imaging are alternately performed, and the imaging device performs the normal imaging. The position of the light emitting device is calculated based on the image,
The position of the light-emitting device and the blinking pattern of the light-emitting device are calculated based on the image captured by the imaging device by the panning imaging, and the blinking pattern calculated from the image taken by the panning-photographing is stored together with the position of the corresponding light-emitting device. The stored position of the light emitting device is corrected based on the position of the light emitting device calculated from the image captured in the imaging of the image, and stored based on the position and the blinking pattern of the light emitting device calculated from the image captured by the flow shooting. The position of the light emitting device is corrected.

[0032]

FIG. 11 is a diagram showing an embodiment of an image processing system according to the present invention. Light emitting device 11
Is a light emitting device that is attached to an object such as a robot and changes the luminance of a light spot with time in a predetermined pattern that is unique to the object or that corresponds to the state of the object.

The video camera 101 captures an image of a light spot emitted from the light emitting device 11 in a panning shot and a normal shooting. The video camera 101 outputs to the image processing device 23 the image data of the blinking pattern of the light spot of the light emitting device 11 obtained by the panning imaging and the image data of the light spot of the light emitting device 11 obtained by the normal imaging. Filter 101A
Is an optical filter that suppresses light other than the light spot emitted from the light emitting device 11 from entering the video camera 101, and is provided in front of a lens unit (not shown) of the video camera 101.

The image processing device 102 comprises an image capture unit 111 and an image processing unit 112, performs predetermined processing on the image data supplied from the video camera 101, detects the position of the light emitting device 11 and the blinking pattern,
The position of the light emitting device 11 and data specifying the light emitting device 11 are output to the display device 24.

The image capture unit 111 performs a predetermined process on the image data of the blinking pattern and the image data obtained by normal imaging, which are supplied from the video camera 101, and the image processing unit 112 executes the predetermined image processing. It generates image data that can be generated and outputs the generated image data to the image processing unit 112.

The image processing unit 112 includes the image capturing unit 1
The image data of the blinking pattern (image data of the image taken by the panning) represented as the change in the luminance corresponding to the lapse of time supplied from the image data 11 is converted into a two-dimensional space, that is, X
The image data is converted into image data of a spatial pattern represented by each coordinate of the axis and the Y axis (written in the frame memory), and from the image data, the position and the blinking pattern of the light emitting device 11 are detected and obtained by ordinary imaging. The position of the light emitting device 11 is detected from the obtained image data. The image processing unit 112 determines the position of the light emitting device 11 and the blinking pattern,
Output to the display device.

The display device 24 displays data supplied from the image processing device 102.

FIG. 12 is a diagram showing the configuration of the video camera 101. When the video camera 101 performs normal imaging, the synchronization signal generation circuit 131 generates a normal imaging field signal supplied to the CCD imaging circuit 136, outputs the signal to the switch 135, and outputs a horizontal synchronization signal synchronized with the normal imaging field signal. Then, a vertical synchronization signal is generated and output to the flowing imaging field drive signal generation circuit 132, the index generation circuit 133, and the drive signal switching pulse generation circuit 134.

The normal imaging field signal output by the synchronization signal generation circuit 131 is used as a flowing imaging output field signal during a flowing imaging image output period described later with reference to FIG.

Flow imaging field drive signal generation circuit 13
Reference numeral 2 denotes a floating imaging generation field signal synchronized with the horizontal synchronizing signal and the vertical synchronizing signal based on the horizontal synchronizing signal and the vertical synchronizing signal input from the synchronizing signal generating circuit 131. (Supplied to the circuit 136), and outputs the same to the switch 135.

The index generation circuit 133 includes a horizontal synchronization signal and a vertical synchronization signal input from the synchronization signal generation circuit 131, and a drive signal switching pulse generation circuit 13
4 based on the normal imaging field pulse (turned on when the CCD imaging circuit 136 performs the normal imaging) input from the camera signal processing circuit 137 to the index (whether the image processing apparatus 102 performs the normal imaging or not). A signal for determining whether or not the image signal has been captured, and an index signal for setting the image signal to a normally captured image.
37.

FIG. 13 is a diagram showing a configuration of the index generation circuit 133. The 8-bit counter 151 inputs the horizontal synchronization signal input from the synchronization signal generation circuit 131 as a clock, and compares the 8-bit count value with the comparison circuit 1.
52. 8 output from the 8-bit counter 151
The bit count value indicates which line from the top of the predetermined image the CCD imaging circuit 136 is currently outputting image data. For example, the CCD imaging circuit 136
When the image data output by the... Is composed of 240 lines, the count value output by the 8-bit counter 151 has a value from 0 to 239, for example.

The 8-bit counter 151 inputs the vertical synchronizing signal input from the synchronizing signal generation circuit 131 as a reset signal, and when the vertical synchronizing signal takes a predetermined value (when an image is switched), the 8-bit counter 151 counts. Reset the value.

The comparison circuit 152 includes an 8-bit counter 15
The count value supplied from 1 is compared with the fixed value “238”, and a signal indicating the result of the comparison is output to the JK flip-flop 153. That is, the 8-bit counter 15
When the count value from 1 is equal to the fixed value “238”,
The comparison circuit 152 sets “1” to the JK flip-flop 15
3 is output to the set input.

When “1” is input to the set input, the JK flip-flop 153 outputs “1” to the logical product circuit 15.
4 is output. The JK flip-flop 153 inputs a vertical synchronization signal to a reset input. So, for example,
The image data output by the CCD imaging circuit 136 is 240
When configured with lines, JK flip-flop 1
53 outputs “1” when the count value output from the 8-bit counter 151 is either 238 or 239.

The logical product circuit 154 calculates the logical product of the output signal of the JK flip-flop 153 and the normal imaging field pulse, and supplies it to the camera signal processing circuit 137 as an index signal.

That is, for example, the CCD image pickup circuit 136
However, when outputting the image data of 238 lines or 239 lines (the two lines from the bottom of the image) of the normal captured image, the index generation circuit 133 sends the index signal of “1” to the camera signal processing circuit 137. Supply.

Drive signal switching pulse generation circuit 134
Generates a signal for the CCD imaging circuit 136 to switch between an operation of performing normal imaging, an operation of performing panning imaging, and an operation of outputting an image taken by panning, which will be described later. The drive signal switching pulse generation circuit 134
A switching pulse for controlling a switch 135 that switches a field signal supplied to the CD imaging circuit 136 (the CCD imaging circuit 136 outputs a normal imaging operation, a panning imaging operation, or a panning imaging image) One of the signals to switch each of the actions)
Is supplied to the switch 135, and a normal imaging field pulse which is turned ON when the CCD imaging circuit 136 is performing normal imaging is supplied to the index generation circuit 133.

FIG. 14 is a diagram illustrating the configuration of the drive signal switching pulse generation circuit 134. The two-bit counter 171 receives the vertical synchronization signal input from the synchronization signal generation circuit 131 to the drive signal switching pulse generation circuit 134 as a clock input, counts it with two bits, and counts the upper one bit of the count value (C1 in the figure). Are supplied to the AND circuit 172, the AND circuit 173, and the AND circuit 175, respectively.

The upper one bit of the count value is returned to the 2-bit counter 171 as a reset input. That is, the 2-bit counter 171 cyclically outputs the values “00”, “01”, and “10” in response to the input of the vertical synchronization signal.

The AND circuit 172 has a 2-bit counter 1
Both the upper one bit and the lower one bit of the count value input from 71 are inverted (determining a negative value), and the logical product of the inverted values is determined and output as a normal imaging field pulse. The normal imaging field pulse is output to the index generation circuit 133 as an output of the drive signal switching pulse generation circuit 134.

The AND circuit 173 includes a 2-bit counter 1
The upper one bit of the count value input from 71 is inverted, the logical product of the upper one bit of the inverted count value and the lower one bit of the count value is obtained, and the result is output to the inverting circuit 174 as a flowing imaging generation field pulse. . The inverting circuit 174 inverts the flowing imaging generation field pulse and outputs it as a switching pulse. The switching pulse is supplied to the switch 135 as an output of the drive signal switching pulse generation circuit 134.

The AND circuit 175 is a 2-bit counter 1
The lower 1 bit of the count value input from 71 is inverted, and the logical product of the lower 1 bit of the inverted count value and the upper 1 bit of the count value is obtained and output as a floating imaging output field pulse.

Thus, the normal imaging field pulse,
The panning imaging generation field pulse and the panning imaging output field pulse, when any one signal is ON,
This is an exclusive signal that turns off other signals.

Returning to FIG. 12, when the switch pulse is ON based on the switch pulse supplied from the drive signal switch pulse generating circuit 134,
The normal imaging field signal supplied from the synchronization signal generation circuit 131 is supplied as a CCD drive signal to the CCD imaging circuit 136, and when the switching pulse is OFF, the panning imaging supplied from the panning imaging field drive signal generation circuit 132 The generated field signal is supplied to the CCD imaging circuit 136 as a CCD drive signal.

The CCD image pickup circuit 136 actually picks up an image of the light emitting device 11 by normal image pickup or flow image pickup based on the normal image pickup field signal supplied from the switch 135 or the flow image pickup generation field signal. (Alternately taken in a predetermined order), and the normally taken image and the flow-taken image are taken in the camera signal processing circuit 13 in a predetermined order.
7

FIG. 15 is a timing chart for explaining the timing of image pickup by the CCD image pickup circuit 136. FIG.
(A) is a signal indicating a field index for identifying a field of an interlaced image. FIG.
FIG. 5B is a diagram showing a CCD drive signal. FIG.
As shown in (C), when the normal imaging field pulse is ON, the drive signal switching pulse generation circuit 134 turns the flowing imaging generation field pulse OFF. Since the switching pulse generated by inverting the flowing imaging generation field pulse is turned on as shown in FIG. 15F, the switch 135 switches the normal imaging field signal supplied from the synchronization signal generation circuit 131 to the CCD imaging. Circuit 136
Output to Hereinafter, a period in which the normal imaging field signal is supplied to the CCD imaging circuit 136 is referred to as a normal imaging period. Therefore, during the normal imaging period, the normal imaging field signal is supplied to the CCD imaging circuit 136 as a CCD drive signal, so that the CCD imaging circuit 136 captures an image of the light emitting device 11 by normal imaging.

Normal imaging field signal is CCD imaging circuit 1
In the field next to the predetermined field supplied to 36, as shown in FIG. 15D, the drive signal switching pulse generation circuit 134 turns on the flowing imaging generation field pulse, so that the switching pulse is turned off. . The switch 135 outputs a panning imaging generation field signal supplied from the panning imaging field drive signal generation circuit 132. Hereinafter, a period in which the flowing imaging generation field signal is supplied to the CCD imaging circuit 136 is referred to as a flowing imaging period. Therefore, during the floating imaging period, the CCD imaging circuit 13
6 is supplied with the imaging field signal, so that CC
The D imaging circuit 136 captures an image of the light emitting device 11 by panning imaging.

In the field next to the predetermined field in which the flowing image pickup field signal is supplied to the CCD image pickup circuit 136, as shown in FIG. Is turned ON, the panning imaging generation field pulse is turned OFF. Therefore, the switch 135 outputs the normal imaging field signal supplied from the synchronization signal generation circuit 131 to the CCD imaging circuit 136. Hereinafter, after the flowing imaging period, the CCD imaging circuit 136 outputs the normal imaging field signal.
Is referred to as a flowing captured image output period.
Therefore, in the flowing captured image output period, the flowing imaging output field signal (the same signal as the normal imaging field signal) is supplied to the CCD imaging circuit 136, so that the CCD imaging circuit 136 converts the image that has been flowed and captured in the flowing imaging period. Output.

As described above, the switch 135 switches the signal supplied to the CCD imaging circuit 136 to the normal imaging field signal, the flowing imaging generation field signal, and the flowing imaging signal in accordance with the switching pulse supplied from the driving signal switching pulse generating circuit 134. Since the output field signal is sequentially switched to the output field signal (the same signal as the normal imaging field signal), FIG.
As shown in FIG. 6, the CCD imaging circuit 136 performs an operation of normal imaging and outputting a normal image, an operation of flowing imaging and generating an image of flowing imaging (at this time, an image output from the CCD imaging circuit 136). Are discarded because they are not used), and the operation of outputting the flow-taken image is sequentially switched and repeated.

The bright spot Li1 corresponding to the light emitting diode 32 of the light emitting device 11-1 and the light emitting device 11
Even if the bright point Li2 corresponding to the -2 light emitting diode 32 overlaps, as shown in FIG. 17, the bright point Li1 and the bright point Li2 in a normal image obtained by normal imaging.
And do not overlap.

The camera signal processing circuit 137 determines that the index signal supplied from the index generation circuit 133 is
When N, a predetermined pixel value predetermined as an image signal (for example, a pixel value in which the R signal, the G signal, and the B signal are both "255") is output. Therefore, predetermined pixel values are set to the pixel values of the lines 238 and 239 of the normally captured image output from the camera signal processing circuit 137, as shown in FIG. It should be noted that the camera signal processing circuit 137 is connected to the line 0 of the normally captured image.
237 and the images taken by panning are output as they are.

Therefore, the image processing unit 112 of the image processing apparatus 102 determines whether the supplied image lines 238 and 239
By confirming the pixel value of the pixel set in, it is possible to easily determine whether or not the image is a normally captured image.

As described with reference to FIG. 15, the CCD imaging circuit 136 is supplied with a normal imaging field signal, a floating imaging generation field signal, and a floating imaging output field signal sequentially. 1
The image output by the camera 36 is sequentially switched to a normally captured image and a panned image (CCD during the panning imaging period).
The image output from the imaging circuit 136 is discarded because it is not used.)

Accordingly, the image processing unit 112 determines that the field number f corresponding to the image determined to be a normally captured image
_If 3 is set as _normal and the field number corresponding to the sequentially input image is incremented by 1 according to the switching of the field index value, the image processing unit 112 determines the field number of the image to be determined. The type of the image can be determined based on the remainder obtained by dividing f by 3.

That is, if the value obtained by dividing the field number f by 3 is 0, the image processing unit 112 determines that the image is a normally captured image, and divides the field number f by 3. If the value of the remainder is 1, it is determined that the image is output during the panning imaging period, and if the value of the remainder obtained by dividing the field number f by 3 is 2, it is determined that the image is a panning image. I do.

Further, the image processing unit 112 can identify the image that has been shot and taken without using the index of the normally taken image. In the image taken in the panning direction, the pixel values of the pixels arranged in the direction in which the panning image is taken have a high correlation with each other.
Calculates a correlation value of pixel values in the direction of panning imaging based on the image supplied from the image capture unit 111, and when a correlation value equal to or greater than a predetermined threshold is obtained, the image is a panned image May be determined.

The image processing unit 112 sets the field number f corresponding to the image determined to be the panning image.
_{If the flowshoot} is set to 3, and the field number corresponding to the sequentially input image is incremented by 1 according to the switching of the field index, the image processing unit 112 sets the field number f of the image to be determined to The type of the image can be determined based on the remainder value divided by 3.

If the image capture unit 111 generates the field number f and determines the type of image corresponding to the field number f in advance and captures it, the image processing unit 112
Can determine the type of image without setting and detecting an index or calculating a correlation value.
That is, for example, when capturing the normally captured image, the image capturing unit 111 sets the field numbers to 0, 3, 6, 9,..., And captures the image output during the flowing imaging period. (You may discard them without outputting them), and set the field numbers to 1, 4, 7,
Are set to 10,..., And the field numbers are 2, 5, 8, 1
1,... May be set.

Flow imaging field drive signal generation circuit 13
More specifically, as shown in FIG. 19, the flow image generation 2 is performed during a predetermined A period after a reset pulse (a signal for resetting the photoelectrically converted charge stored in the CCD imaging circuit 136) is input. CCD imaging circuit 1 as field signal
36 outputs a field signal for normal imaging,
Thereafter, a field signal for performing the panning imaging is output (the panning imaging period in the figure).

Then, as shown in FIG. 20, the CCD image pickup circuit 136 precedes the image of the bright spot Li picked up during the panning pick-up period (in the same horizontal direction and on the line located above the blinking pattern). ), And outputs an image of the bright spot Li normally captured during the period A.

The image of the bright spot Li normally taken in the period A is
Since the signal level of the image is higher than that of the image of the bright spot Li captured in the flowing imaging period, the image processing apparatus 102 can easily identify the image corresponding to the bright spot Li to be processed, and The load of processing for specifying an image can be reduced. Further, the coordinates of the luminescent spot Li in the flowing image can be reliably obtained.

Conversely, as shown in FIG. 21, after the reset pulse is input, the floating imaging field drive signal generating circuit 132 outputs a field signal for performing floating imaging, and in the subsequent A period, the floating imaging generation field A field signal for causing the CCD imaging circuit 136 to perform normal imaging may be output as a signal.

In this case, the CCD image pickup circuit 136 has the configuration shown in FIG.
As shown in (1), after the image of the bright point Li captured in the panning imaging period, the image of the bright point Li normally captured in the period A is output.

Further, it is also possible to make the CCD image pickup circuit 136 pick up an image of the bright spot Li which is normally picked up during the flowing picked-up image output period. As shown in FIG. 23, the photodetector of the CCD imaging circuit 136 corresponds to the bright spot Li during the period from the start of the field of the flowing captured image output period until the CCD imaging circuit 136 actually transfers the electric charge in the vertical direction. Before the CCD imaging circuit 136 starts the transfer in the vertical direction, the electric charge corresponding to the bright point Li of the photodetector is read out to the V register (vertical CCD). Thereafter, the CCD image pickup circuit 136 transfers the electric charge on the V register in the vertical direction, and outputs an image.

By the above operation, the CCD image pickup circuit 136
As shown in FIG. 24, the image of the bright point Li normally captured in the period A can be output after the image of the bright point Li captured in the panning imaging period.

FIGS. 25 and 26 show the difference between the coordinates of the bright spot Li on the normally captured image and the coordinates of the bright spot Li (corresponding to the same light emitting device 11) on the flow-taken image. FIG. As shown in FIG. 25, if the coordinates of the bright spot Li on the normally captured image are (x, y), as shown in FIG. Since a shift occurs by Δy (corresponding to the same number of lines as the number of times the output of the photodiode is transferred to the vertical transfer CCD during the field of the panning image pickup), the bright spot Li on the panning image is shot. The coordinates are
(X, y + Δy).

Next, the bright spot Li executed by the image processing unit 112
FIG. 2 shows the process of calculating the coordinates and the blinking pattern of FIG.
7 and FIG. 28. First, the image processing unit 112 selects a bright point within a predetermined variation range (X1−ΔX <X <X1 + ΔX) with respect to the x coordinate X1, and adds the selected bright point in the X-axis direction to obtain the result shown in FIG. As shown in (1), the integrated value I of the pixel value corresponding to the direction of panning imaging (for example, the direction of the vertical axis of the image) is obtained.

Next, the image processing unit 112 performs the processing shown in FIG.
As shown in (1), an edge detection filter for detecting an edge in the direction of panning imaging is applied to the integrated value I of the pixel values.
The edge detection filter has, for example, E (n) = I (n) -I with respect to (1, -1), that is, the coordinate value n in the direction of panning imaging (corresponding to the horizontal axis of the graph in the figure). (N-1)
Is a filter that outputs.

Then, the image processing unit 112
As shown in (C), the absolute value of the output value E of the edge detection filter is obtained, and the output value E of the edge detection filter that is smaller than a predetermined threshold is discarded.

The image processing unit 112 generates a rising edge (FIG. 28) based on the output value E of the remaining edge detection filter.
A section from an output value E having a positive value in (A) to a falling edge (output value E having a negative value in FIG. 28A) is set to bit 1, and a section from the falling edge to the rising edge is set. The section up to the edge is set to bit 0.

As described above, the image processing unit 112 calculates the blinking pattern based on the edge of the image of the bright spot Li, thereby reducing errors in the blinking pattern due to images other than the light emitting device 11. it can.

FIG. 29 shows the position corresponding to the bright point Li calculated in the past by the image processing unit 112, and the ID (Idetificatio).
n) Data (data corresponding to the blinking pattern of bright spot Li)
FIG. 6 is a diagram for explaining a list A that stores. List A is
It is composed of elements a (1) to a (n) corresponding to the light emitting devices 11-1 to 11-n to be imaged, respectively. Each element a (i) is the I of the bright spot Li (i).
D, the coordinates (position on the image) of the bright point Li (i), and the distance between the coordinates of the bright point Li (i) in the image in the normal imaging and the coordinates of the bright point Li (i) in the image taken in the following manner. Consists of

The image processing unit 112 calculates the coordinates corresponding to the bright points Li (1) to Li (n) when an image obtained by normal imaging is input, and calculates the element a (1) of the list A. To the coordinates of the element a (n).

The image processing unit 112 calculates coordinates and IDs (data corresponding to a blinking pattern) corresponding to the bright points Li (1) to Li (n) when an image obtained by panning is input. And elements a (1) to a of the list A
Set to the coordinates and ID of (n). When the images of the luminescent spots Li overlap in the image obtained by the panning imaging (when the blinking pattern cannot be calculated), the image processing unit 112 is used.
Calculates the blinking pattern only for the bright spots Li for which the blinking pattern can be calculated, and calculates the element a of the list A.
(1) to ID of element a (n) are set.

As described above, even when the images of the bright spots Li overlap in the image taken by the moving image capturing, the image processing unit 112 always keeps the bright spots Li (1) corresponding to the light emitting devices 11-1 to 11-n. ) To accurate coordinates and ID of the bright point Li (n) can be output.

Lists a, b, and A 'which are temporarily used by the image processing unit 112 to calculate the position corresponding to the bright point Li and the ID data have the same configuration.

Next, a process of calculating the position and ID of the bright spot Li corresponding to the light emitting device 11 of the image processing device 102 will be described with reference to the flowchart of FIG. In step S11, the image processing unit 112 initializes the list A.

In step S12, the image capture section 111 captures an image supplied from the video camera 101. In step S13, the image processing unit 112 performs processing based on the index and the field number embedded in the image,
It is determined whether or not the image supplied from the image capture unit 11 is a normally captured image. If it is determined that the image supplied from the image capture unit 111 is a normally captured image, the process proceeds to step S14 to perform image processing. The unit 112 executes the processing of the normally captured image, and proceeds to step S17. Step S
Details of the processing of the 14 normal captured images will be described later with reference to the flowchart in FIG.

If it is determined in step S13 that the image supplied from the image capture unit 111 is not a normally captured image, the process proceeds to step S15, where the image processing unit 112 determines the index and field number embedded in the image. It is determined whether or not the image supplied from the image capture unit 111 is an image that has been shot and captured by the processing based on. If it is determined in step S15 that the image supplied from the image capture unit 111 is an image shot and taken, the process proceeds to step S16.
The image processing unit 112 executes processing of the image of the panning imaging,
Proceed to step S17. The details of the processing of the flow-captured image in step S16 will be described later with reference to the flowcharts in FIGS.

If it is determined in step S15 that the image supplied from the image capture unit 111 is not an image that has been shot and taken, the image is output when a field signal for generating a shot image is supplied to the CCD image pickup circuit 136. Since the image is output by the circuit 136 (the image is output during the floating imaging period), step S16 is skipped, and the process proceeds to step S17.

In step S17, the image processing unit 11
Step 2 determines whether or not to end the processing. If it is determined that the processing is not to be ended, the process returns to step S12, and the processing is repeated from image capture.

If it is determined in step S17 that the processing is to be ended, the procedure ends.

As described above, the image processing apparatus 102 executes a predetermined process according to the type of the input image.

Next, the processing of the normally picked-up image corresponding to step S14 in FIG. 30 will be described with reference to the flowchart in FIG. In step S31,
The image processing unit 112 clears the list a and the list A ′, which are lists temporarily used for processing.
In step S32, the image processing unit 112 performs a process of calculating the position of the bright spot Li on the normally captured image. For details of the process of calculating the position of the bright spot Li, see FIG.
This will be described later with reference to the flowchart of FIG.

In step S33, the image processing unit 11
No. 2 stores the position of the bright spot Li calculated in step S32 in the list a. In this step, only the position information is stored in each element of the list a, and "-1" indicating indefinite is stored in the ID and distance of each element of the list a.

For example, in a (1), the information of the LED 1c of the normally picked-up image 2 shown in FIG.
Is stored in the element a (1), the information of the LED 2c is stored in the element a (2) of the list a, and the information of the LED 3c is stored in the element a (3) of the list a. More specifically, "-1" indicating indefinite is stored in the ID of the element a (1) of the list a, and the coordinates of the element a (1) of the list a are LED.
(X1c, y1c) which is the coordinate of 1c is stored, and "-1" indicating indefinite is stored in the distance of the element a (1) of the list a. Similarly, “−1” is stored in the ID of the element a (2) of the list a, and (x2c, y2c) that is the coordinate of the LED 1c is stored in the coordinates of the element a (2) of the list a. , The distance of element a (2) of list a is "-
1 "is stored in the ID of the element a (3) of the list a, and" -1 "is stored in the coordinate of the element a (3) of the list a, which is the coordinates of the LED 1c (x3c, y3c). ) Is stored, and the distance of the element a (3) of the list a is “−”.
1 "is stored.

In step S34, the image processing unit 11
2 finds the element of list A having the closest distance for each element of list a, and writes its ID and distance in the distance of the element of list a.

For example, the LED 1c of the normally captured image 2 shown in FIG.
LED2c of image 2 which is closest to D1b and is normally taken
Is closest to the LED 2b of the image 1 shot and taken, and the LED 3c of the image 2 normally picked up is closest to the LED 3b of the image 1 shot and taken. The distance of the element a (1) is
The distance of the element a (2) is calculated by Expression (1), and is calculated by Expression (2).
And the distance of the element a (3) is calculated by equation (3). The ID of element a (1) is set to 1 and element a (1)
The ID of (2) is set to 2 and the ID of element a (3)
Is set to 3.

[0100]

(Equation 1)

(Equation 2)

(Equation 3) In step S35, the image processing unit 112 determines whether the distance of the element a (1) of the list a is smaller than a predetermined threshold, and determines that the distance of the element a (1) of the list a is smaller than the predetermined threshold. If it is determined to be smaller, step S36
To move the light emitting device 11 or the video camera 101
, A plurality of elements of the list a may correspond to one element of the list A.
It is determined whether or not the list a includes another element having the same ID as the element a (1).

If it is determined in step S36 that there is another element having the same ID as the element a (1) in the list a, the process proceeds to step S37, where the image processing unit 1
12 is an element a (1) having the smallest distance among elements a (1) and other elements having the same ID as the ID of the element a (1).
Select (j). In step S38, the image processing unit 112 selects the element a (j) selected in step S38.
Is added to the list A ′.

In step S39, the image processing unit 11
2 deletes the element having the same ID as that of the element a (j) from the list A. In step S40, the image processing unit 112 deletes the element a (j) from the list a, and proceeds to step S44.

In step S36, when it is determined that there is no other element including the same ID as the element a (1) in the list a, the process proceeds to step S41, and the image processing unit 112 proceeds to step S41. 1) is added to list A '.

In step S42, the image processing unit 11
2 deletes the element having the same ID as the ID of the element a (1) from the list A. In step S40, the image processing unit 112 deletes the element a (1) from the list a, and proceeds to step S44.

If it is determined in step S35 that the distance of the element a (1) in the list a is larger than a predetermined threshold, the procedure proceeds to step S43, assuming that the element a (1) is erroneous.

In step S44, the image processing unit 11
2 determines whether list a is empty or not,
If it is determined that is not empty, the process returns to step S34 and repeats the process because all IDs for the bright spot Li have not been calculated.

If it is determined in step S44 that the list a has become empty, the process proceeds to step S45, where the image processing unit 112 writes the contents of the list A 'into the list A, and the process ends.

As described above, the image processing apparatus 102 can specify the ID of the light emitting device 11 with respect to the bright spot Li from the normally captured image.

Next, the process of calculating the position of the bright spot of the image processing unit 112 corresponding to step S32 in FIG.
This will be described with reference to the flowchart in FIG.

In step S61, the image processing unit 11
2 is to determine whether each pixel of the normally captured image is equal to or greater than a predetermined threshold, and when it is determined to be equal to or greater than the threshold, 1 is set to that pixel, and it is determined that the value is less than the threshold. In this case, the pixel is set to 0, and the normally captured image is binarized.

At step S62, the image processing unit 11
2 applies a two-dimensional Sobel operator to each pixel in the x direction and the y direction, and determines whether the sum of absolute values of the output of the Sobel operator in the x direction and the output of the Sobel operator in the y direction is equal to or greater than a predetermined threshold value If it is determined that the sum of the absolute values of the output of the Sobel operator in the x direction and the output of the Sobel operator in the y direction is equal to or greater than a predetermined threshold, the pixel value of the pixel is set to 1, and the Sobel operator X
If it is determined that the sum of the absolute values of the output in the direction and the output in the y direction of the Sobel operator is less than a predetermined threshold, the pixel value of the pixel is set to 0, and an edge is extracted.

At step S63, the image processing unit 11
2 assigns the same number to connected pixels having a pixel value of 1,
Labeling.

At step S64, the image processing unit 11
2 finds the coordinates and radius of the center of the circle using the Houth transformation for each of the labeled regions. Alternatively, the image processing unit 112 may use the average value of the coordinates of the pixels included in each area as the center coordinates, and use the average value of the distance from the obtained center to each pixel as the radius.

As described above, the image processing section 112 can obtain the position of the bright spot Li corresponding to the light emitting device 11 from the normally captured image.

Next, a description will be given, with reference to the flow charts of FIGS. 33 and 34, of the processing of the flow picked-up image corresponding to step S16 of FIG. In step S81, the image processing section 112 clears the lists a, b, and A 'which are temporary lists.
In step S82, the image processing unit 112 obtains the position of the luminescent spot Li corresponding to the light emitting device 11 on the image shot and shot.

At step S83, the image processing unit 11
In step 2, the image processing unit 112 obtains the ID of the bright spot Li on the image taken by the panning, that is, the data corresponding to the blinking pattern.

In step S84, the image processing unit 11
No. 2 stores in the list a the data of the bright spot Li for which both the position and the ID have been obtained. In this step, the distance between the elements of the list a is "-1" indicating indefinite.
Is stored. In step S85, the image processing unit 1
Numeral 12 stores the data of the bright spot Li for which only the position is obtained in the list b. In this step, "-1" indicating indefinite is stored in the distance and ID of each element of the list b.

In step S86, the image processing unit 11
2 adds the head element a (1) of the list a to the list A ′. In step S87, the image processing unit 112
Deletes, from the list A, an element having the same ID as the element a (1) among the elements of the list A. Step S88
In, the image processing unit 112 deletes the element a (1) from the list a.

At step S89, the image processing unit 11
2 determines whether or not the list a is empty, and if it is determined that the list a is not empty, step S
Returning to 86, the process is repeated.

In step S89, when it is determined that the list a has become empty, the position and the I
Since the data of the bright point Li from which both of D and D can be obtained is deleted, the process is performed on the bright point Li from which only the position can be obtained, so that the process proceeds to step S90, and the image processing unit 112 proceeds to step S90. For each element of list b, the element of list A having the closest distance is obtained, and its ID and distance are written in the distance of the element of list b.

In step S91, the image processing unit 11
2 determines whether or not the distance of the element b (1) of the list b is smaller than a predetermined threshold, and when it is determined that the distance of the element b (1) of the list b is smaller than the predetermined threshold, Proceeding to step S92, the movement of the light emitting device 11 or the movement of the video camera 101 may cause one element of the list A to correspond to a plurality of elements of the list b. , It is determined whether there is another element with the same ID as the ID of the element b (1).

If it is determined in step S92 that there is another element having the same ID as the element b (1) in the list b, the process proceeds to step S93, where the image processing unit 1
12 is an element b (1) and an element b (1) having the minimum distance among other elements having the same ID as the ID of the element b (1).
Select (j). In step S94, the image processing unit 112 selects the element b (j) selected in step S93.
Is added to the list A ′.

At step S95, the image processing unit 11
2 deletes the element having the same ID as the element b (j) from the list A. In step S96, the image processing unit 112 deletes the element b (j) from the list b, and proceeds to step S100.

If it is determined in step S92 that there is no other element including the same ID as that of the element b (1) in the list b, the process proceeds to step S97, and the image processing unit 112 proceeds to step S97. 1) is added to list A '.

In step S98, the image processing unit 11
2 deletes the element having the same ID as the ID of the element b (1) from the list A. In step S99, the image processing unit 112 deletes the element b (1) from the list b, and proceeds to step S100.

If it is determined in step S91 that the distance of the element b (1) of the list b is larger than the predetermined threshold, the procedure proceeds to step S99, assuming that the element b (1) is erroneous.

At step S100, the image processing unit 1
12, it is determined whether or not the list b is empty. If it is determined that the list b is empty, the process proceeds to step S101. Are overlapped and only the data corresponding to the bright spot Li for which both the ID and the position could not be obtained, the content of the list A 'is added to the list A, and the process is terminated.

If it is determined in step S100 that list b is not empty, the process proceeds to step S102, where image processing section 112 determines whether list A is empty and list A is empty. If it is determined that the association has not been completed, the association has not been completed, so the flow returns to step S90 and the processing is repeated.

If it is determined in step S102 that list A has become empty, no further correspondence can be made, and the process ends.

As described above, the image processing apparatus 102 obtains the position and the ID of the bright point Li corresponding to the light emitting device 11 even if the blinking patterns indicating the ID of the bright point Li overlap in the shot image. be able to. Therefore, the image processing apparatus 102 can always know the position of the light emitting device 11 corresponding to the bright spot Li.

In the present specification, the system is
It is assumed that the entire device including a plurality of devices is represented.

Further, as a providing medium for providing a user with a computer program for performing the processing as described above,
In addition to recording media such as magnetic disks, CD-ROMs, and solid-state memories, communication media such as networks and satellites can be used.

[0133]

According to the image processing apparatus according to the first aspect, the image processing method according to the seventh aspect, and the recording medium according to the eighth aspect, normal imaging and flowing imaging are alternately performed. The imaging of the imaging device is controlled as described above, and the position of the light emitting device is calculated based on the image captured by the imaging device in the normal imaging,
The position of the light-emitting device and the blinking pattern of the light-emitting device are calculated based on the image captured by the imaging device by the panning imaging. The stored position of the light emitting device is corrected based on the position of the light emitting device calculated from the image captured in the imaging of the image, and stored based on the position of the light emitting device and the blinking pattern calculated from the image captured by the flow. Since the positions of the light emitting devices that are present are corrected, a plurality of light emitting devices 11 can always be specified, and the light emitting devices 1
1 can always be known.

[Brief description of the drawings]

FIG. 1 is a diagram showing a conventional image processing system using panning imaging.

FIG. 2 is a diagram showing an internal configuration of a light emitting device 11;

FIG. 3 is a diagram showing an internal configuration of an image processing apparatus 23.

FIG. 4 is a diagram illustrating panning imaging.

FIG. 5 is a diagram illustrating panning imaging.

FIG. 6 is a diagram illustrating panning imaging.

FIG. 7 is a diagram for explaining processing of the image processing unit with respect to an image taken by panning.

FIG. 8 is a diagram illustrating the X coordinate and the Y coordinate of a bright point Li1, Li2, and Li3 on a display image.

FIG. 9 is a diagram for explaining processing of the image processing unit with respect to an image taken by panning.

FIG. 10 is a diagram illustrating an image in which light emission patterns overlap.

FIG. 11 is a diagram showing an embodiment of an image processing system according to the present invention.

FIG. 12 is a diagram showing a configuration of a video camera 101.

FIG. 13 is a diagram showing a configuration of an index generation circuit 133.

14 is a diagram illustrating a configuration of a drive signal switching pulse generation circuit 134. FIG.

FIG. 15 is a timing chart illustrating the timing of imaging by the CCD imaging circuit 136.

FIG. 16 is a diagram illustrating the order of images output by the video camera 101.

FIG. 17 is a diagram illustrating an image that is normally captured when light emission patterns overlap in an image that has been shot and shot.

FIG. 18 is a diagram illustrating an index.

FIG. 19 is a diagram illustrating a normal imaging period provided in a panning imaging period.

FIG. 20 is a diagram illustrating an image taken by panning.

FIG. 21 is a diagram illustrating a normal imaging period provided in a panning imaging period.

FIG. 22 is a diagram illustrating an image taken by panning.

FIG. 23 is a diagram illustrating a normal imaging period provided in a panned image output period.

FIG. 24 is a diagram illustrating an image taken by panning.

FIG. 25 is a diagram illustrating a difference between the coordinates of the bright point Li on the normally captured image and the coordinates of the bright point Li on the flow-captured image.

FIG. 26 is a diagram illustrating the difference between the coordinates of a bright point Li on a normally captured image and the coordinates of a bright point Li on a flow-captured image.

FIG. 27 is a diagram illustrating a process of calculating coordinates and a blinking pattern of a bright point Li.

FIG. 28 is a diagram illustrating a process of calculating the coordinates of a bright point Li and a blinking pattern.

FIG. 29 is a diagram illustrating a list.

FIG. 30 is a flowchart illustrating a process of calculating the position and ID of a bright spot Li of the image processing apparatus 102.

FIG. 31 is a flowchart illustrating processing of an image that is normally captured.

FIG. 32 is a flowchart illustrating a process of calculating the position of a bright spot by the image processing unit 112.

FIG. 33 is a flowchart illustrating processing of an image taken by panning.

FIG. 34 is a flowchart illustrating processing of an image taken by panning.

[Explanation of symbols]

101 video camera, 102 image processing device, 1
11 image capture unit, 112 image processing unit, 1
Reference Signs List 31 synchronization signal generation circuit, 132 moving imaging field drive signal generation circuit, 133 index generation circuit, 134 drive signal switching pulse generation circuit, 1
35 switch, 136 CCD imaging circuit, 137
Camera signal processing circuit, 151 8-bit counter,
152 comparison circuit, 153 JF flip-flop,
154 AND circuit, 1712 bit counter,
172 AND circuit, 173 AND circuit, 175
Inverting circuit, 175 AND circuit

 ──────────────────────────────────────────────────続 き Continuing on the front page F term (reference) 2F065 AA04 BB29 FF04 JJ03 JJ26 NN08 QQ01 QQ17 QQ24 QQ25 QQ27 QQ42 QQ45 5B057 AA05 BA02 CA02 CA08 CA12 CA16 CB02 CB06 CB12 CB16 DA07 5L096 AA03 HA13 AA03 AA03 AA13 AA03 AA03 AA03 AA03 AA03 AA03 HZ19 KZ32

Claims (8)

[Claims] An image processing apparatus for processing an image supplied from an image pickup apparatus for picking up one or more light emitting devices, wherein the image pickup of the image pickup apparatus is controlled so that normal image pickup and floating image pickup are alternately performed. Imaging control means, first calculation means for calculating the position of the light emitting device based on an image captured by the imaging device using normal imaging, and A second calculating unit that calculates a position of the light emitting device and a blinking pattern of the light emitting device; a storing unit that stores the blinking pattern calculated by the second calculating unit together with a corresponding position of the light emitting device; First correction means for correcting the position of the light emitting device stored in the storage means based on the position of the light emitting device calculated by the first calculation means, and calculation by the second calculation means Said An image processing apparatus comprising: a second correction unit configured to correct the position of the light emitting device stored in the storage unit based on the position of the light emitting device and the blinking pattern. 2. The image processing apparatus according to claim 1, wherein the imaging control unit inserts an identification signal into a predetermined line of at least one of a normally captured image and a flow captured image. . 3. The imaging apparatus according to claim 1, wherein the imaging control unit controls the imaging so that the imaging apparatus performs normal imaging for a predetermined period before the panning imaging in the panning imaging period. Image processing device. 4. The apparatus according to claim 1, wherein the imaging control unit controls the imaging so as to cause the imaging device to perform normal imaging for a predetermined period after the panning imaging in the panning imaging period. Image processing device. 5. The image pickup apparatus according to claim 1, wherein the image pickup control means controls the image pickup so that the image pickup apparatus performs a normal image pickup for a predetermined period during a period in which the flow picked-up image is output. Image processing device. 6. The image processing apparatus according to claim 1, wherein the second calculating unit calculates the blinking pattern of the light emitting device based on a position of an edge of the panned image. apparatus. 7. An image processing method for an image processing apparatus for processing an image supplied from an imaging apparatus for imaging one or more light emitting devices, wherein the normal imaging and the panning imaging are alternately performed. An imaging control step of controlling imaging; a first calculation step of calculating a position of the light emitting device based on an image captured by the imaging apparatus in normal imaging; and an image captured by the imaging apparatus in flowing imaging. A second calculating step of calculating a position of the light emitting device and a blinking pattern of the light emitting device, and the blinking pattern calculated in the process of the second calculating step, together with a corresponding position of the light emitting device. A storage step of storing, and a position of the light emitting device stored in the processing of the storage step is modified based on the position of the light emitting device calculated in the processing of the first calculation step. Correcting the position of the light emitting device stored in the processing of the storage step based on the position of the light emitting device and the blinking pattern calculated in the processing of the second calculating step. An image processing method comprising: a second correction step. 8. An image processing program for processing an image supplied from an image pickup device for picking up one or more light emitting devices, wherein the program for the image pickup device is arranged such that normal image pickup and floating image pickup are alternately performed. An imaging control step of controlling imaging; a first calculation step of calculating a position of the light emitting device based on an image captured by the imaging apparatus in normal imaging; and an image captured by the imaging apparatus in flowing imaging. A second calculating step of calculating a position of the light emitting device and a blinking pattern of the light emitting device, and the blinking pattern calculated in the process of the second calculating step, together with a corresponding position of the light emitting device. Based on the storing step of storing and the position of the light emitting device calculated in the processing of the first calculating step, the position of the light emitting device stored in the processing of the storing step is corrected. A first correcting step, and correcting the position of the light emitting device stored in the processing of the storing step based on the position of the light emitting device and the blinking pattern calculated in the processing of the second calculating step. A recording medium on which a program for causing a computer to execute a process characterized by including the following two correction steps is recorded.