JPS6058505B2 - Mark detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a mark detection device, and more particularly to a mark detection device that observes an object using an imaging device and automatically extracts marks placed on the object from video information. .

Visual information processing devices that use imaging devices as a means of inputting information have been applied in various industrial fields as a means of speeding up processing and saving labor, but in recent years there has been an increasing demand for automation using this type of visual devices. One of the fields in which this field has been growing is cargo handling, where cargo is loaded and unloaded onto freight cars, trucks, and ships.

Automation of cargo handling basically involves the use of gripping devices (
This system automatically controls the positioning of the vehicle (hereinafter referred to as a spretsuda) at the loading/unloading point.A moving imaging device captures the mark attached to the destination point, detects the position of the mark that appears in the image, and uses this positional information. The structure controls the movement of the spreader based on this. However, unlike other general industrial product manufacturing or handling fields, the cargo handling field faces special problems in that the objects to be imaged tend to get dirty, and since they are used outdoors, there are large variations in imaging conditions such as weather and background. Therefore, it is difficult to accurately detect marks from video information by simply capturing an image of the object.

The present invention has been made to solve the above problems, and is a mark detection method that can detect marks on an observation target from among other similar patterns with a small-scale device configuration even in an environment where imaging conditions are unstable. The purpose is to provide equipment.

For the above purpose, the present invention turns on and off the light incident on the imaging device from the mark attached to the observation target, and detects a signal portion having predetermined characteristics from the image when the light state is in one state. The mark is then detected based on the result of comparison with the previously stored position. Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is an overall configuration diagram when mark detection according to the present invention is applied to a cargo handling device.

In the figure, reference numeral 1 denotes a trolley mounted on a container crane so as to be able to move horizontally.This trolley 1 is equipped with a beam 2 that can be raised and lowered, and a spreader 3 for holding a container 4 provided at the lower end of the beam.

Furthermore, a position control system is installed on top of the trolley 1, which includes an illumination device 5, a television camera 6 closed for imaging, and a control device 7 that processes video data and outputs a position control signal based on the processing result. and is adapted to move together with the spreader 3. The lighting device 5 illuminates the loading/unloading surface 10 through the space 1a in the trolley 1. Furthermore, the television camera 6 images the surface 10 illuminated by the illumination device viewed through the space 1a. The illustrated television camera is arranged horizontally, and has a format in which a reflected image of a mirror 8 is captured through a specific wavelength light transmission filter 9. The mark Ms attached to the top surface of the spreader 3 is always within the field of view of the television camera 6, while the positioning mark M provided on the cargo loading/unloading surface 10
As the trolley moves slowly, it comes into the field of view of the television camera. The control device 7 includes a circuit 11 for blinking the illumination device 5, a circuit 12 for binarizing the video signal 6 from the television camera 6 in order to digitize information on the video screen for each pixel, and a circuit 12 for binarizing the video signal 6 from the television camera 6. a circuit 13 for processing the converted signal 12s and extracting a characteristic pattern on the video, a memory 4 for storing coordinate data of the characteristic pattern, and the television camera 6.
A synchronizing pulse generating circuit 15 provides horizontal and vertical synchronizing pulses 15x and 15Y to Data processing device to output, for example, microcomputer 1
Contains 6.

The other circuit elements 17 included in the control device 7 are a clock pulse generation circuit, and 18 is address data 1 indicating the X and Y coordinates of the scanning point on the image based on the clock pulse 17a.
Counter circuit that outputs 8x, 18Y, 19 is memory 1
4 shows an interface circuit for accessing 4. In this embodiment circuit, the feature extraction circuit 13 extracts feature points similar to the marks M3 and Mc from the video, and the feature extraction circuit 1
Address data 18x and 18Y indicating the X and Y coordinates are written in the memory 14 in response to the output signal 13 of No. 3.

The data processing device 16 controls the blinking circuit 11 using the on/off control signal 1, thereby turning on and off the illumination of the imaging field by the illumination device 5, and when the illumination is off, the coordinates input into the memory 14 are The mark position is detected by comparing the data with the data input into memory when the light is on. FIG. 2 conceptually shows the data processing flow in the above device.

M corresponds to whether the lighting is on or off, and when M=0, the lighting is off. Block 21 shows an image when the illumination is off, and this image is processed by the feature extraction circuit 13 to extract only a predetermined feature pattern. Block 22 shows the distribution state of each feature point after feature extraction, and the center coordinate data of each black circle is stored in memory 14. Data processing device 1
6 once reads these coordinate data into internal memory,
Set the value of M to 1 (block 23). This turns on the lights. When the illumination is on, as shown in block 24, a characteristic pattern Ms'' of the mark Ms appears in the image. Also, if the trolley has reached the vicinity of the target point, a characteristic pattern Mc'' of the mark Mc indicating the loading/unloading point appears. is also shown in this video. In this example, the marks M, , Mc are each in the form of three circles arranged at each vertex of an equilateral triangle, and when Ms<5Mc, the triangles are oriented in opposite directions.
Block 25 shows the feature point distribution of the video 24 after feature extraction. The data processing device 16 reads each feature point center coordinate data of the pattern 25 from the memo 1 river 4, compares it with each coordinate data of the pattern 22 that has already been read into the internal memory, and erases the noise pattern existing in the background. Then, the coordinate data of the discrepancy is obtained (block 26). The coordinate distribution of feature points obtained as a result of the above matching process is as shown in block 27. The data processing device 16 extracts the coordinate points forming the marks Ms and Mc from the positional relationship of the remaining coordinate points, calculates the center of the triangle, and detects the mark center position coordinates (block 28). As understood from the above description, in the present invention, the on/off control signal 11c from the data processing device is
The feature is that the mark blinks on the video in response to the on-screen image pattern, and the noise component included in the background is erased by comparing the on-screen image pattern with the off-screen image pattern.

Therefore, the mark used for position detection does not need to be a mark that can be clearly discerned by the human eye at all times under all weather conditions, as was the case with conventional devices. It is preferable that the mark is such that it reflects light in the direction of the camera position and does not reflect light irradiated from other directions in the direction of the camera when the illumination device is turned off. Mark materials having such properties include, for example, light regressive reflectors known under the trade names Scotchilite, Cat's Eye, Corner Cube, and the like. Furthermore, a light emitting element such as a light emitting diode which is blinked by a wireless receiver is used as the alignment mark MG instead of a reflector, and a transmitter that controls the blinking of the light emitting element is used instead of the illumination device 5 and the blinking circuit 11. may be applied. In contrast to the embodiments of the present invention as described above, there is a known pattern matching technique that directly matches video data instead of matching coordinate data.

However, considering the existence of a wide variety of noise patterns in the background of a mark, a video data processing format that extracts only the characteristic patterns that correspond to the mark and detects the true mark position by comparing the center coordinate data of each characteristic pattern. is easier to collate, requires less data memory capacity, and is advantageous in terms of processing speed. An example of the feature extraction circuit 13 suitable for such clear image data processing is shown in the third section.
This will be explained using figures.

This circuit is the pixel information 12s output from the binary circuit 12.
A circuit 3 that sequentially captures pixel information, arranges pixel information two-dimensionally, and reproduces a two-dimensional local pattern surface that is one part of the image surface.
0, a pattern reduction circuit 40, an isolated pattern determination circuit 45, and a picture element information selection circuit 50. This circuit compresses the image included in the two-dimensional local pattern surface and compresses the central picture of the isolated pattern. This method selectively performs elemental extraction. As the two-dimensional local pattern reproducing circuit 30, in principle, the one known in Japanese Patent Publication No. 51-1249 can be adopted, and shift registers 31 to 34 are used to output pixel information of the two-dimensional local pattern in parallel. and a shift register 3 for temporarily storing pixel information on the x scanning line.
5 to 37, and a pulse signal generation circuit 38 for supplying a shift pulse signal to each shift register.
The above shift registers are sequentially connected in cascade, and the picture element information 21S input to the shift register 31 is sequentially transferred to the shift registers 35, 32 by a shift pulse signal 38.
, 36, 33, 37, and 34. Now, assuming that we start scanning horizontally on the video screen from the upper left corner and reach the fourth pixel from the scanning point, each pixel information on the first scanning line has one pair. shift register 34
and 37 according to the scanning order from the right end to the left end, and similarly, the pixel information on each of the second and third scanning lines is stored in the set of shift registers 33 and 36 and the set of shift registers 32 and 35. , the first four picture element information on the fourth scanning line are stored in scanning order from the right end of the shift register 31. Therefore, the shift registers 31 to 34, each consisting of 4 bits, represent a local area consisting of 4×4 picture elements in the upper left corner of the video screen, as shown in FIG. If there is a shift, the shift registers 31 to 34 will contain picture element information of a local plane shifted by one picture element in the X direction.

In this embodiment, the pattern reduction circuit 40 selects Sll-Sl3, S2l-J, S3l of the local patterns.
The nine picture element information from S33 to S33 are input, and picture element information 40 converted according to the states of these picture elements is output.

For example, as shown in FIG. 5, the pattern reduction circuit 40 inputs 30 s of pixel information from the local pattern regeneration circuit, counts the number of pixel elements with a pattern "", and outputs a 4-bit binary code. The counting circuit 41 compares the counted value 41s with a preset value N, and when it is equal to or greater than the set value ゜"
It can be configured as a comparison circuit that outputs a signal 40 of ゜゜0゛ when the value is smaller than the 1-setting value. The pattern reduction circuit 40 also includes a memory device 33, which stores predetermined data in correspondence with addresses, as shown in FIG.
For example, it can be replaced by a read-only memory (ROM). In this case, 7 to 7 memory addresses are designated by the combination of states of the pixel information 30s input to the nine address lines 33a, among which the set value N
By storing data "1" in advance in an address specified by a binary number where the above number of bits becomes "1", it is possible to set the data when the number of picture elements equal to or greater than the set value is in the state "1". A binary signal 40 of state "゜1゛" is outputted.The above setting value N determines the degree of reduction of the pattern, and when the area of the local surface serving as the judgment area is S, , N
=S/2, the pattern is only slightly shifted in position, and the pattern reproduced with the output signal 40s is almost equal to the pattern reproduced from the input signal 12s. However, when N=S/2+ΔS is set, the output signal 40s is 0゛ when the center point of the judgment area is on the ring of the target pattern, and it becomes ``1゛'' only after entering the inside of the target pattern. Since it will be output, the output signal 40
The pattern reproduced by , has a shape in which the ring part of the original pattern is removed by a dimension proportional to ΔS.
In the above example circuit, the area S of the determination area is 9 pixels, so the set value N that satisfies N=S/2+ΔS is 5 to 5 pixels.
8, and a practical setting value N is 6 or 7.

l In FIG. 3, the isolated pattern determination circuit 45 detects the picture elements in the limbal part of the local surface, that is, Sll to Sl49S in FIG.
Osamu S24? This is a circuit, for example, a NOR circuit, which takes in the signal of the picture element corresponding to S3l9S349S4l''S44 and outputs a signal 45s of ゛゜1゛ when the state of these picture element information is ``0゛''.

The signal 45s is "1" if there is no object pattern in the local plane or if the picture elements S22 and S
This is when an isolated pattern exists in the area consisting of 23, S32, and S3. The selection circuit 50 includes a gate 51 that is opened when the output signal 45s from the isolated pattern determination circuit 45 is "°1", and the output signal 45 is applied via an inverter circuit 52 so that the signal 45 is "0". The signal 48S from the representative picture element located at the center of the local plane is input to the gate 51. The output signal 40s from the pattern reduction circuit 40 inputted to the gate 53 is selected according to the signal 45s, and is outputted to the subsequent stage circuit as the signal 50s.

Further, a changeover switch 46 is provided on the output side of the determination circuit 45, and by switching this switch to the "0" potential side, the signal 40 of the pattern reduction circuit 40 is sent to the subsequent circuit regardless of the isolated pattern determination result. Feature pattern extraction circuit 1 shown in Figure 1 earlier
3 is constructed by cascading the above-mentioned pixel information conversion circuits in a plurality of stages, for example, five stages. The first stage circuit has a circuit configuration in which the function of the isolated pattern determination circuit 45 is excluded by setting the changeover switch 46 to the ゜゜0゛ position.
When passing through this circuit, all isolated patterns having a number of picture elements of 5 or less can be erased. A circuit configuration in which the changeover switch 41 is set to the isolated pattern determination circuit side is applied to the second to fourth stage circuits.

In these circuit sections, even when the pattern is reduced to an isolated pattern with 4 or less picture elements, the isolated pattern determination circuit 45 functions, so the representative picture element S22 is saved and the isolated pattern of a single picture element disappears. doesn't happen. The circuit required at the final stage is a circuit for selecting an isolated pattern consisting of a local pattern reproducing circuit 30 and an isolated pattern determining circuit 45, and a circuit adapted to take out a signal from the gate 51 of the circuit shown in FIG. 3 is applied.

By passing the signal through this final stage circuit, a binary signal 13s is obtained in which all parts other than the isolated pattern with the number of picture elements of 4 or less are erased, and only one picture element representing this isolated pattern remains. FIG. 7 shows the progress of pattern conversion in the feature extraction circuit configured as described above.

In the figure, A, B, C, and D indicate object patterns of different sizes included in the video, and changes in these patterns occur at each step I to (2) corresponding to the plurality of pixel information conversion circuits. It is shown in As can be seen from the figure, pattern A in which the number of picture elements is less than the set value disappears in the first stage.
Further, patterns such as B and C, which are reduced to a single picture element size, are output without disappearing due to the function of the isolated pattern determination circuit.

On the other hand, the large pattern D that cannot be recognized as an isolated pattern at step ■ disappears at the final stage.
It does not appear in the output signal. Therefore, when the feature extraction circuit described above is used, isolated patterns smaller than a predetermined size are first erased from the video imaged by the television camera 6, and then the remaining patterns are sequentially compressed toward the center, and the center coordinate position is Converge on the picture element. In addition, large patterns that cannot be converged on the center picture element are erased in the final stage, so in the end, only isolated patterns of a predetermined size included in the original video are represented by the center picture element and are distinctive. By storing the coordinate data when this output signal 13 is .degree. 1.degree., pattern data to be verified can be obtained.
FIG. 8 shows the construction of a flashing circuit 11 suitable for the case where a seven-metal vapor discharge tube emitting single-spectrum light, such as a sodium lamp, is used as the light source of the illumination device 5. As is well known, discharge tubes such as sodium lamps are lit with alternating current to prevent electrode deterioration, so the amount of light emitted from the discharge tube pulsates at twice the frequency of the power supply. For this reason, if the pulsation of light emission and the scanning of the television camera are not synchronized, it is necessary to
The brightness of the image varies depending on the location on the screen, and the timing of the on/off control of the lamps does not match, resulting in the problem that objects cannot be stably recognized. The blinking circuit in FIG. 8 inputs the vertical synchronization signal 15Y of the television camera obtained from the synchronization pulse generation circuit 15'', first widens the pulse width by the waveform shaping circuit 61 to set the duty ratio to 50%, and then The signal 113 is input to a filter circuit 62 having a sine wave and converted into a sine wave, and further power-amplified by an amplifier circuit 63 to obtain a signal 113.

The waveform shaping circuit 61 generates an on/off control signal 11 synchronized with a vertical synchronization signal from the data processing device 16. is input, and in response to this, a signal output operation is performed. Therefore, when the vertical synchronization signal 15Y1 and the on/off control signal 11c are input with the waveforms shown in FIGS. 9A and 9B, the output signal 61S of the waveform shaping circuit 61 becomes as shown in FIG. 9C, and the blinking circuit output current 11, The waveform is shown in FIG. 9D. According to this circuit configuration, the on/off control signal 11. Since the waveform shaping circuit 61S does not output any output during the period when the angle is 0°, the amount of light from the discharge tube during this period can be reduced to almost zero as shown in FIG. 9E. Metal vapor discharge tubes such as sodium lamps are designed to keep the temperature inside the tube constant and the metal vapor pressure inside the tube constant during lighting in order to maintain discharge and light emission. It has been confirmed that when the off period in the control is set to 0.1 seconds or less, there is no change in vapor pressure, and when an on signal is applied, it is possible to easily restart the discharge of a steady amount of light. If the off-period is longer, it will take more time to reach the steady light amount, but the time delay is about 0.2 seconds for an off-period of 1 second, so there is no problem in implementing the present invention. One of the advantages of employing such a metal vapor discharge tube in the lighting device 5 is that the illumination light from this kind of discharge tube is single-spectrum light, and the reflected light of this wavelength is selectively transmitted through the filter 9 to the television. By inputting it to the camera 6, it is possible to prevent noise patterns from being input due to reflection of natural light. The case where the mark detection device of the present invention is applied to an automatic cargo handling device has been described above. However, according to the present invention, an image of an object having a mark is captured by an imaging device, and even if the imaging conditions are unstable, marks can be detected from video data. Position can be detected accurately with a small device configuration,
It can be widely applied not only to cargo handling and disassembly but also to other industrial fields. When applied to the cargo handling field, if a mark is provided on the side of the cargo handling equipment and the cargo handling point as shown in the example, it is possible to detect not only the position of the target point but also the deflection of cargo handling due to strong winds. This has the advantage of being able to unload or lift loads from high places with high precision.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing one embodiment of a mark detection device according to the present invention, FIG. 2 is a flow diagram for explaining the mark detection procedure, and FIG. 3 is a diagram showing a feature extraction circuit suitable for the device of the present invention.
FIG. 4 is a diagram illustrating a local aspect of a two-dimensional pattern in the feature extraction circuit, and FIGS. 5 and 6 are pattern reduction circuits indicated by the reference numeral 40 in FIG. 4, respectively. FIG. 7 is a diagram for explaining the conversion process of pixel information in the feature extraction circuit, and FIG. 8 is a diagram showing an example of the blinking circuit indicated by the reference numeral 11 in FIG. 1. The circuit block diagram shown in FIG. 9 is an explanatory diagram of signal waveforms in the blinking circuit. In the figure, 1 is a trolley that moves on a crane, 3 is a spreader that transports a container 4, 5 is a lighting device, 6 is an imaging device, and 7 is a control device.

Claims (1)

[Scope of Claims] 1. A mark detection device that photoelectrically converts an image of an observation target to which a mark is attached using an imaging device and detects a signal corresponding to the mark from among the obtained photoelectric conversion signals, means for turning on and off the light incident on the imaging device from the image pickup device; extraction means for detecting a signal portion having predetermined characteristics from the photoelectric conversion signal of the imaging device and determining the position at which the signal portion is detected; and the on-off means means for storing the position extracted by the extracting means when the on/off means is in at least one state, and the position extracted by the extracting means and the one stored in the storage means when the on/off means is in the other state; A mark detection device comprising means for comparing the position of the mark with the position of the mark and detecting a partial signal corresponding to the mark from a position where there is a difference. 2. The mark attached to the observation target is composed of a light reflecting plate having retroreflectivity in the direction of light incidence, and the on/off means is composed of a light irradiation device that irradiates on/off light from the imaging device side in the direction of the observation target. A mark detection device according to claim 1, characterized in that: 3. The mark detection device according to claim 2, wherein the light irradiation device includes a light source that emits light of a specific wavelength. 4. The mark detection device according to claim 3, wherein the imaging device receives the imaging light through a filter having a light transmission characteristic for the specific wavelength light. 5. Claim 3, wherein the light source is a metal vapor discharge tube, and the discharge tube is lit by a power source synchronized with a vertical synchronization signal for video scanning of the imaging device. The mark detection device according to item 1 or 4. 6. The mark detection device according to claim 1, wherein the mark attached to the observation target is made of a light emitting element, and the on/off means controls blinking of the light emitting element.