JPS6379490A - Processing method for picture signal of image - Google Patents

Abstract

PURPOSE:To obtain correct control information irrespective of day and night by lighting or putting out an illumination according to the brightness of the picture quality of an image sensor and inverting binarized picture data at the time of lighting. CONSTITUTION:A conduct sensor is constituted of a CCD(charge coupled device) 71 having the number of picture elements of nXm and an optical lens 72 and the output is applied to the A/D converter 81 of a picture signal processing part 8. The respective picture elements of the CCD71 stores a charge according to the intensity of light and irradiated on the surface and an irradiating time, the picture signal from the CCD71 is defined to be picture data of four bits according to the level in the A/D converter 81 and stored in a video memory 82. A prescribed operation is executed in an arithmetic control part 83 to output a level control signal according to the relative position of a airframe to string lines of the crop stalk to a conduct control part 9 and output a signal for lighting or putting out a projector 70 according to the brightness of the visual field for picking up an image of the conduct sensor 7.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a method for processing image signals from an image sensor when the image sensor is used as a sensor for controlling various types of agricultural machinery.

[Prior art]

Photodiode, COD (Charge Cou
pled-Device.

Image sensors using charge-coupled devices (charge-coupled devices) have the excellent property of being able to detect a large amount of information, such as the size and position of an object, without contact. Since good quality sensors are now available at relatively low prices, they are beginning to be widely used as control sensors in various industrial fields.

[Problem that the invention seeks to solve]

Such an image sensor can also be applied as a variety of control sensors in agricultural machines. For example, in a harvester, an image sensor is installed to image the culm in the conveyance path from the reaping section to the threshing section. The culm length of the shell culm is detected to control the handling depth, and an image sensor is installed to image the row of shell culms planted in front of the harvester in the direction of travel, and the relative position of the machine relative to the row of shell culms is detected. Image sensors can be used as various control sensors, such as steering control.

By the way, agricultural machinery is mainly operated outdoors due to its purpose, and is often operated at intervals of day and night during the busy farming season, so image sensors used as control sensors are At night, the object to be imaged is imaged under natural light, and at night under illumination. However, for example, when an image sensor is used as a sensor for steering control of a harvester as described above, the culm row, which is the object to be imaged by the sensor, is imaged darkly against the background under natural light, and under illumination. On the other hand, the image below is captured brightly against the background, so if the processing method of the image @ signal from the image sensor is the same regardless of day or night, there is a risk that the relative position of the aircraft with respect to the culm row will be incorrectly detected at night. Furthermore, at dusk or during rain, the amount of natural light is insufficient, making it difficult to distinguish between the culm row and the background from the image sensor imaging results under natural light, making it difficult to determine the relative position of the aircraft. There is a risk of false detection.

In either case, the problem is that the steering control cannot be performed correctly based on the detection results of the image sensor. The same applies when an image sensor is used as a control sensor for various types of control.

The present invention has been made in view of the above circumstances, and provides an image signal processing method for an image sensor that can obtain accurate control information regardless of day or night when an image sensor is used as a sensor for controlling agricultural machinery. The purpose is to provide.

[Means for Solving the Problems] The image signal processing method for an image sensor according to the present invention includes processing for an image signal obtained by imaging under natural light and an image signal obtained by imaging under illumination. This is an attempt to change the method, and the image signal of the image captured by the image sensor installed on agricultural machinery is
In an image signal processing method for an image sensor in which various controls are performed based on image data after digitization processing and binarization, lighting is turned on or off depending on the brightness of the image of the image sensor, while; The method is characterized in that when the lighting is on, the binarized image data is inverted.

〔Example〕

The present invention will be explained below based on drawings showing the male side of the invention. In this example, an image sensor whose image signal is processed according to the image signal processing method for an image sensor according to the present invention (hereinafter referred to as the method of the present invention) is used.
A case will be described in which the present invention is used as a steering sensor in a steering control device of lt&'31. Fig. 1 is an external perspective view of a harvester equipped with a steering sensor using an image sensor, Fig. 2 is a schematic diagram of reaping work shown together with a schematic plan view of the JN harvester, and Fig. 3 is a steering control FIG. 2 is a block diagram of the system.

In the figure, reference numeral 1 denotes a running roller, on which a threshing section 2, a driver's seat DS, etc. are mounted, and these constitute a main body 3. On the front side of the main body 3, a grass dividing plate 4
．． A reaping section 6 composed of a culm lifting device 5 and the like is attached so as to be movable up and down.

The rollers 1 that run are provided on the left and right sides of the main body 3, and are powered by a 11131 engine that drives the respective cranks (not shown).
It is designed to be driven through 7 channels.

These clutches have a clutch cylinder 10 shown in FIG.
j! , 10r are engaged and disengaged separately, and when the clutch cylinder 101 is actuated, the clutch of the roller 1 running on the right side is disengaged and the aircraft is steered to the left. The clutch of steering crawler 1 is F! Iim and the aircraft steers to the right.

Clutch cylinder 1. OI! , 10r is constituted by a hydraulic pump P, an electromagnetic directional control valve (2), an oil tank T, and the like. Solenoid directional valve ■ is 4
The button is a three-position switchable type, and when the solenoid Sl (or solenoid Sr) is excited, pressure oil from the hydraulic pump P is supplied to the oil chamber of the clutch cylinder 10j' (or clutch cylinder 10r). Then, the piston rod advances and the oil chamber of the clutch cylinder 10r (or clutch cylinder 101) communicates with the oil tank T, and the piston rod retracts and moves to the left or right side as described above.
The traveling roller 1 stops and the aircraft steers left (or right). Solenoid Sj again! , Sr are both demagnetized, the clutch cylinder 10I! , 10r
Both oil chambers communicate with the oil tank T, these piston ronds move in and out, and the left and right running rollers 1 are driven at a constant speed, so that the aircraft moves straight.

Now, the steering sensor 7, which uses an image sensor, is as shown by the two-dot chain line in FIG. In order to image the area within the imaging field of view A, a sensor is placed at a position near the driver's seat DS at the top of the reaping section 6.
It is installed slightly forward and downward. Further, 70 is a floodlight for irradiating the entire imaging field of view A with light, and is installed above the reaping section 6 in parallel with the steering sensor 7.

The steering sensor 7 is, for example, a CCD having a pixel count of nxm.
71, is composed of an optical lens 72 and the like for forming an image of the object on the photosensitive surface of the CCD 71, and its output is sent to an A/D converter 81, a video memory 82, and an arithmetic control section 83. The image signal processing unit 8 is provided with an image signal processing unit 8 consisting of:

Each pixel of the CCU 71 accumulates charges corresponding to the intensity and irradiation time of the light irradiated onto its surface, and each pixel accumulates charges according to the intensity and irradiation time of the light irradiated on its surface, and each pixel accumulates charges depending on the intensity and irradiation time of the light irradiated on its surface, and each pixel accumulates charges depending on the intensity and irradiation time of the light irradiated on its surface. The A/D converter 81 of the image signal processing unit 8 sends image signals of a level corresponding to the charge stored in the pixels in the order in which the direction of movement of the aircraft is the main scanning direction and the direction orthogonal thereto is the sub-scanning direction. Output to. and CCD 7
The image signal from 1 is converted into an analog/digital signal by the A/D converter 81 in, for example, 16 steps according to its level, and stored in the video memory 82 from "0000" representing the darkest part to "1111" representing the brightest part. The image data is stored as 4-bit image data.

The arithmetic control unit 83 using a microcomputer is
Reads the image data stored in the video memory 82, performs an l&if operation from this data, and outputs a control signal to the steering control unit 9 at a level corresponding to the relative position of the aircraft with respect to the rows of shell culms. Depending on the brightness of the imaging visual field A of the steering sensor 7, a signal is output to the floodlight 70 to turn it on or off.

On the output side of the steering control section 9, a solenoid 341. of the electromagnetic directional control valve V is connected. Sr is connected via an excitation circuit (not shown), and when the level of the input signal from the arithmetic control section 83 is equal to or higher than a predetermined value, the steering control section 9 outputs a corresponding output to excite the solenoid SZ. The boat is set to a high level, the aircraft is steered to the left, and when the level of the input signal is below another predetermined value that is smaller than the predetermined value, the corresponding output boat is set to a high level in order to energize the solenoid Sr. level and steer the aircraft to the right to control the harvester's steering.

FIG. 4 is a flowchart showing the control contents of the arithmetic control section 830, and the method of the present invention will be explained below with reference to this figure.

The harvester runs on the field while operating the reaping part 6,
The cutting section 6 cuts off the culms planted on the field surface. During this time, the steering sensor 7 images the rows of culms within the imaging field of view A from above together with the field surface serving as the background, as described above. Fifth
6 are schematic diagrams showing the imaging results of the people steering sensor 7, and FIG. 5 shows the imaging results when the harvester is traveling while maintaining its normal position with respect to the husk row. , FIG. 6 shows the imaging results when the vehicle is traveling to the left, that is, closer to the culm row side than the normal position.

The arithmetic control unit 83 resets a register that stores an index indicating whether or not the light projector 70 is lit. The register contains 1 if the floodlight 70 is lit;
When the light is off, 0 is stored. Next, the arithmetic control unit 83 generates image data D1.corresponding to each pixel of the CCD 71. After resetting the register that stores the total S, the image data DiJ stored in the video memory 82 is sequentially read, and these are added to calculate the value of S, while these are set to a predetermined threshold value, and the value is set as a reference. It is converted into light and dark binarized as "1" representing the bright part and "0" representing the dark part.
” are sequentially written into the memory in the arithmetic control unit 83 as binary image data DiJ'.

nxml stored in the video memory 82 in this way.
IIi] image data D1. After calculating the sum S, divide this by nxm to obtain the average brightness D within the imaging field of view A.
Calculate a. This value Da naturally takes a large value when the imaging field of view is bright, and takes a small value when it is dark.

Therefore, the average value Da should be compared with the preset brightness reference values D2 and D, respectively, and if Da is less than D2, the light emitting 1s70 should be turned on (the corresponding output boat should be set to high level). On the other hand, the next image data is read from the video memory 82 again without performing steering control based on the previously obtained binary image data D, '4, with the [marker set to 1].

On the other hand, if Da is greater than or equal to D3, the corresponding output port is set to low level in order to turn off the floodlight 7°,
When the index is set to 0 and Da is a value between D2 and D3, the output to the projector 70 is not changed, and both are based on the previously obtained binary image data D and J'. Proceed to the next stage to perform steering control.

Now, the part of the culm row shown by hunting in Figs. 5 and 6 is imaged darkly compared to the background under natural light due to diffuse reflection of light, and the image data after binarization Di
'J is "0" in the part where the culm row exists and is "l" in the background part. However, when the amount of natural light is insufficient, such as at dusk or during rain, the difference in brightness between the background and the culm row decreases, making it difficult to distinguish them using the binary image data D,′. becomes. The reference value D2 is set to a value slightly larger than the limit brightness at which it becomes difficult to distinguish between the background and the culm row, and as mentioned above, the average brightness value Da within the imaging field of view A is less than or equal to D2. If this happens, the floodlight 70 that illuminates the person in the imaging field of view is turned on to compensate for the lack of light in the imaging field of view A, making it easier to distinguish between the background and the culm row. Further, the reference value D is set to a value slightly larger than the brightness within the imaging field of view A obtained by lighting the floodlight 70 when the average brightness value Da becomes equal to D2. Even if the floodlight 70 is turned off when the average brightness value t)a becomes D3 or higher, the brightness within the imaging field of view A is maintained at the reference value D2 or higher using natural light, and the difference between the culm row and the background is It does not interfere with discrimination.

In this way, the floodlight 70 is turned on and off depending on the brightness of the image captured by the steering sensor 7, so the brightness within the imaging field of view A is always maintained at an appropriate level, regardless of day or night. It is possible to easily distinguish between the culm row and the background using the binary image data D1', which will be explained below.

Now, as mentioned above, under natural light, the part of the culm row is imaged darkly compared to the background, but under illumination by the floodlight 70, the distance from the floodlight 7o to the upper surface of the culm row, and the distance from the floodlight 70 to the background Due to the difference in distance to the field surface, contrary to the imaging results under natural light, the hatched portions of the culm rows shown in Figures 5 and 6 are imaged brighter than the background. Image data D1' after binarization,
is 1″ in the culm row part and o in the background part.
'' and is written in the memory in the arithmetic control unit 83.The arithmetic control unit 83 compares Da with D2 and D3, and after performing the above processing, determines whether L is I or 0. That is, it checks whether the floodlight 70 is lit or not, and if L is 1, it subtracts the binary image data D i'j from "1" and writes the result into its own memory, When L is 0, the image data D i'J is left as it is and the process proceeds to the next step. In this way, when L is 1, the binary image data "1" becomes "0". , and ′O” is “1”
Binary image data D1 obtained from the imaging result under illumination when the light source 70 is turned on and the floodlight 70 is turned on.
Similarly to the imaging results under natural light, the culm row part is "0" and the background part is "lo", and the imaging results of the steering sensor 7 under natural light and the imaging results under illumination are shown below. It can be processed using the same method as shown in .

Next, the arithmetic control unit 83 sequentially reads out the binary image (image data D1') from its own memory, and
”, the area of the bright part in the imaging visual field A, that is, the plane MB of the background part, is obtained, and a control signal of a level corresponding to this size is output to the steering control unit 9,
Reads the next image data Dij from the video memory 82,
Repeat the same process.

Now, since the steering sensor 7 is fixed to the upper part of the reaping part 6, the steering sensor 7 is fixed to the upper part of the reaping part 6, so the steering sensor 7 is fixed to the upper part of the reaping part 6, so the steering sensor 7 is fixed to the upper part of the reaping part 6, so the steering sensor 7 is fixed to the upper part of the reaping part 6, so it is possible to reduce As the machine moves left and right, when the machine approaches the row of culms to be harvested from its normal running position, the surface of the background part within the imaging field of view A as shown in Figure 6. ! ! ! B becomes small, and as a result, the level of the control signal output from the arithmetic control section 83 to the steering control section 9 becomes small.As mentioned above, the steering control section 9 switches the solenoid Sr of the electromagnetic directional control valve (2). Acting to excite, the harvester is steered to the right away from the row of culms.

On the other hand, when the harvester moves away from the grain culm row, although not shown, the area B of the portion other than the grain culm row within the imaging field of view A becomes larger, contrary to the case in Fig. 6, and as a result, Since the level of the control signal from the calculation control section 83 increases,
The steering control section 9 operates to energize the solenoid SI1 of the electromagnetic directional control valve (2), and the harvester is steered to the left in order to approach the culm row.

In this way, the plane aB of the background part has a value corresponding to the relative position of the harvester with respect to the row of culms, and the steering control unit 9 operates based on a control signal of a level corresponding to this size. The harvester travels over the field while being steered and controlled so as to always maintain a constant relative position to the rows of shell culms planted on the field.

In this example, an example of application to travel control of a harvester has been described, but the method of the present invention can also be applied to various other controls such as handling depth control in a harvester and various controls in other agricultural machinery. Needless to say, it is.

〔effect〕

As detailed above, in the method of the present invention, when an image sensor is used as a sensor for controlling various types of agricultural machinery,
The light is turned on when the image sensor image is not bright enough and it is difficult to distinguish between bright and dark areas, so the image sensor can be used as a control sensor regardless of day or night. When the lights are on, the image signal from the image sensor is binarized, and then the binarized image data is inverted, so the imaging results under daytime natural light and the imaging results under nighttime lighting are different. The necessary control information can be obtained by processing the image signals obtained from the two using the same method, and this provides additional effects such as being able to perform appropriate control regardless of whether it is day or night.

[Brief explanation of the drawing]

The drawings show one embodiment of the present invention, and FIG. 1 is an external perspective view of a harvester equipped with an image sensor according to the method of the present invention as a steering sensor, FIG. 2 is a schematic diagram of reaping work, and FIG. 3 4 is a block diagram of the steering control system, FIG. 4 is a flowchart showing the control contents of the arithmetic control section, and FIGS. 5 and 6 are schematic diagrams showing the imaging results of the steering sensor. 1...Roller that runs 6...Reaping part 7...
Steering sensor 8... Image signal processing section 9... Steering control section 10β, 10r... Clutch cylinder 7
0... Floodlight 83... Arithmetic control unit A... Imaging field of vision Yamanoya Yanmar Agricultural Machinery Co., Ltd. agent Patent attorney Noboru Kawano 2I ¥1 慟 下] Jaw 5 Round step Illustrated Figure 6 Procedure Written amendment (method) October 30, 1988 Patent Application No. 141608 2, Title of the invention Image signal processing method for image sensor 3, Relationship with the case of the person making the amendment Location of the 11i person for patent Location Kita, Osaka City 1-32 Chayamachi, Ward Name (685) Yanmar Agricultural Machinery Co., Ltd. Representative Atsuo Yamaoka 4, Agent Address ■543 Nisshin Building 207-7, 1-14-22 Shitennoji, Tennoji-ku, Osaka City Details of amendments In the 5th line of page 17 of the bookshelf, the text ``The second one'' has been corrected to ``The second figure is''. that's all

Claims (1)

[Claims] 1. An image sensor for performing binarization processing on an image signal of an image captured by an image sensor installed in an agricultural machine and executing various controls based on the binarized image data. The image signal processing method is characterized in that a light is turned on or off depending on the brightness of the image of the image sensor, and when the light is turned on, the binarized image data is inverted. An image processing method for image sensors.