JP4978799B2 - Straight-forward guidance system for moving vehicles - Google Patents

Description

  The present invention relates to a straight traveling guidance system for a traveling vehicle for guiding a traveling vehicle such as a tractor or a rice transplanter or a traveling vehicle such as a work vehicle in the field of civil engineering and construction toward a target.

  In recent years, the agricultural population has been declining, the number of skilled farmers has decreased, and it is difficult to hire them, so in the work process such as sowing and setting up in the field, agricultural vehicles that are one of the moving vehicles There is an increasing demand for technology for automatically running straight ahead.

  Therefore, conventionally, for example, the following straight-ahead control method for agricultural vehicles has been developed. In this agricultural vehicle straight-ahead control method, the agricultural vehicle has a geomagnetic direction sensor and measurement / control means. Further, in this straight traveling control method for agricultural vehicles, teaching traveling is performed by manual operation using the agricultural vehicle, and the target direction in the forward / reverse two directions is determined from the traveling direction of the agricultural vehicle obtained from the geomagnetic direction sensor every moment. After the acquisition, the agricultural vehicle is automatically driven based on a comparison between the traveling azimuth of the agricultural vehicle acquired from the geomagnetic azimuth sensor every moment and the target azimuth obtained during the teaching traveling (see, for example, Patent Document 1). . Hereinafter, this technique is referred to as a first conventional example.

  Conventionally, work vehicle guidance devices have been developed as described below. In this work vehicle guidance device, the beam light projection means for projecting the guide beam light in the length direction of the work process moves in order to project the guide beam light in each of a plurality of work processes parallel to each other. Equipped at the top of the car. The moving vehicle further includes a position adjusting unit that adjusts the position of the beam light projecting unit in the front-rear direction and the width direction with respect to the moving vehicle, and a turning adjustment of the beam light projecting unit around the vertical axis with respect to the moving vehicle. Turning adjustment means is provided (see, for example, Patent Document 2). Hereinafter, this technique is referred to as a second conventional example.

  Further, conventionally, a light emitter position detection device described below has been developed. In this light emitter position detecting device, a light emitter having a first polarizing filter provided at the outer peripheral position is installed at an end of a travel stroke, and the light emitter is mounted on a work vehicle such as a rice planting machine. A first image sensor provided with a second polarizing filter having the same polarization direction as the polarizing filter, and a second image sensor provided with a third polarizing filter having a polarization direction different from that of the first polarizing filter. I'm shooting. And based on the difference information of the imaging information imaged by the two image sensors, the position of the light emitter within the imaging screen is detected, and the work vehicle is steered toward the light emitter ( For example, see Patent Document 3.) Hereinafter, this technique is referred to as a third conventional example.

  Conventionally, a moving vehicle is equipped with a TV camera, and a natural object such as a distant tree, a structure such as a chimney, or a target pole set in advance is photographed by the TV camera and based on the obtained image. In addition, techniques for automatically moving a moving vehicle toward a target have been proposed (see, for example, Non-Patent Documents 1 and 2). Hereinafter, this technique is referred to as a fourth conventional example.

  On the other hand, in the field of civil engineering / architecture, the following automatic operation devices for construction machines have been developed. This automatic operation device for a construction machine includes a guidance instruction device having a guide light for transmitting a blinking signal and a working machine such as a compacting machine used when paving a road. The working machine includes a CCD camera that captures the blinking signal, an image processing apparatus that processes image data from the CCD camera, and a control device that guides the working machine based on the output of the image processing apparatus and controls traveling. (For example, refer to Patent Document 4). Hereinafter, this technique is referred to as a fifth conventional example.

JP 2000-14209 A JP-A 64-10308 JP 7-170809 A JP-A-3-2428 Tetsu Yamazaki et al., “Image Analysis for Autonomous Straight Running of Agricultural Machinery”, Abstracts of the 61st Agricultural Machinery Society Annual Conference, Agricultural Machinery Society, September 17-19, 2002, p. 313-314 Tetsu Yamazaki et al., “Image Analysis for Autonomous Straight Running of Agricultural Machinery (2nd Report)”, Abstracts of the 62nd Agricultural Machinery Society Annual Conference, Agricultural Machinery Society, April 2-4, 2003, p . 155-156

  By the way, it is important that the agricultural vehicle linearly and accurately performs work processes such as sowing, transplanting, and setting up in the field. In particular, in the first work process, the work trace becomes a standard for the subsequent process (second process) and the following, and other work (next work) to be performed next, for example, a standard for crop growth management work and harvesting work It ’s also important. Therefore, in the first work process, it is important to work with the agricultural vehicle running as straight as possible. Therefore, such an important work is usually performed by a person in charge of work such as a farmer's master or a skilled worker. However, work that requires skill and great responsibility in this way is also a burden for the person in charge of the work, and especially when the area of the field to be worked is large, the distance for performing the first work process is long. Therefore, the burden is even greater. Furthermore, as described above, the agricultural population is decreasing year by year, and there is a constant shortage of human resources who can perform the first work process described above.

  However, in the above-described first conventional example, in order to obtain the target direction, the first work process needs to perform teaching traveling by manual operation, and the above problem cannot be solved. Further, in the first conventional example described above, there is a problem that, for some reason, when noise is mixed in the traveling direction of the agricultural vehicle acquired from the geomagnetic direction sensor, the straight traveling of the agricultural vehicle may be disturbed. there were.

  In the second conventional example described above, the beam light projecting means for projecting the guiding beam light is expensive, and there is a problem that it takes time and labor to install and adjust the beam light projecting means. Not yet commercialized.

  Furthermore, in the third conventional example described above, since the first polarizing filter is provided at the outer peripheral position of the light emitter, the amount of light projected from the light emitter is reduced to half or less. Therefore, when the distance between the light emitter and the work vehicle is long, the amount of light projected from the light emitter is insufficient, and it is difficult to detect the light emitter from the imaging information captured by the two image sensors. Become. That is, the third conventional example has a problem that there is a practical limit on the distance between the light emitter and the work vehicle. Further, in the third conventional example described above, since it is necessary to mount two image sensors on the work vehicle, there is a problem that the configuration of the position detecting device for the light emitter is complicated and the price is high. It was.

  Further, in the above-described fourth conventional example, there are not always conspicuous natural objects and buildings suitable for targeting in the entire landscape around the farm field. Therefore, since there are many cases where it is not possible to detect what should be a target from an image captured by a TV camera in the vicinity of a farm field where there are no conspicuous natural objects or buildings, automatic straight traveling of a moving vehicle cannot be stably controlled based on the image. There were many problems. Further, in the above-described fourth conventional example, when the distance between the target pole installed in advance and the moving vehicle becomes long, it is difficult to distinguish the surrounding background and other objects on the image captured by the TV camera. There is. That is, the above-described fourth conventional example has a problem that there is a practical limit on the distance between the target pole and the moving vehicle.

  Furthermore, in the fifth conventional example described above, in addition to the target guide light, for example, a bright part of a house that reflects sunlight (for example, a window or a roof) is included in the image data from the CCD camera. When it is behind, there is a problem that the area of the guide light and the area of the bright part of the house are extracted at the same time, which makes it difficult to detect the position of the guide light.

  The present invention has been made in view of the above-described circumstances, and an example of an object is to solve the above-described problems, and a straight traveling guide for a moving vehicle that can solve these problems The purpose is to provide a system.

  In order to solve the above-described problem, a straight traveling guidance system for a moving vehicle according to the first aspect of the present invention includes a moving vehicle mounting portion mounted on the moving vehicle and an extension line in front of the runway on which the moving vehicle should travel straight. A remote target unit installed, and the moving vehicle mounting unit outputs a travel information for the straight traveling by processing an image obtained from the camera unit and a camera unit that images the far target unit An image processing unit, and a steering control unit that controls the moving vehicle based on the travel information, and the far target unit is equal to or longer than a total time of an imaging cycle and a maximum exposure time of the camera unit. In addition, the image processing unit repeatedly turns on and off the time in a range that is not more than twice the time of the imaging cycle, and the image processing section Of the imaging cycle In the image of the previous frame, when the absolute value of the luminance difference of the corresponding pixel is equal to or higher than the first threshold and the luminance of any one of the pixels is equal to or higher than the second threshold, the pixel is In addition, the absolute value of the luminance difference between the corresponding pixels in the image of the current frame and the image of the previous frame imaged in the previous imaging cycle is extracted as one far target portion candidate pixel. If it is equal to or higher than the first threshold and the luminance of any one of the pixels is equal to or higher than the second threshold, the pixel is extracted as a second far target portion candidate pixel, and the first and first The two far target portion candidate pixels are integrated, the position of the far target portion is detected for the integrated far target portion candidate pixels, and the travel information is generated based on the detection result.

  The invention according to claim 2 relates to the straight-ahead guidance system for a moving vehicle according to claim 1, wherein the image processing unit detects the position of the far target unit in the entire image of the front frame. The image portion corresponding to the window set as a range to be searched is searched from the entire image of the current frame, the searched image portion is set as a first window in the image of the current frame, An image portion corresponding to the window set in the entire image of the previous frame is searched from the entire image of the current frame, and the searched image portion is searched for a second image in the image of the current frame. It is set as a window, and the first and second far target portion candidate pixels are extracted based on the first and second windows.

  The invention according to claim 3 relates to the straight traveling guidance system for a moving vehicle according to claim 1 or 2, wherein the image processing unit extracts the integrated far target portion candidate pixels by grouping processing. For each of the pixels constituting the plurality of pixel groups, the difference between the luminance in the image of the current frame and the luminance in the image of the previous frame for the past few seconds, and the luminance of the image in the current frame And the period of the blinking of the pixel and the time interval of the blinking are detected based on the past several seconds of the difference from the luminance in the image of the previous frame, and based on the detection result, the far target candidate A pixel is determined as the far target portion.

  According to a fourth aspect of the present invention, there is provided the straight traveling guidance system for a moving vehicle according to the second or third aspect, wherein the image processing unit stores past values of positions on the window for the plurality of pixel groups. For several seconds, the standard deviation in the temporal change in the coordinate values in the vertical and horizontal directions is calculated, and when the standard deviation is smaller than a preset threshold, the pixel group is determined as the far target unit. It is characterized by doing.

  The invention according to claim 5 relates to the straight traveling guidance system for a moving vehicle according to any one of claims 1 to 4, wherein the far target portion has a light emitting portion whose contour shape is rectangular or elliptical. Or it is either circular shape, The aspect ratio of the said outline is between 1/2 and 1/4, It is characterized by the above-mentioned.

  According to a sixth aspect of the present invention, there is provided the linear guiding system for a moving vehicle according to any of the second to fifth aspects, wherein the exposure time of the camera unit, the aperture, and the sensitivity adjustment of the amplifier gain of the image signal are adjusted. , And is set based on the luminance distribution of the image in the window partially set in the image of the camera unit.

  A seventh aspect of the present invention relates to the straight traveling guidance system for a moving vehicle according to any one of the first to sixth aspects, in which the camera unit can image the camera unit together with the far target unit. A marker is attached, and the image processing unit detects the position of the vehicle body marker from the image, and detects the direction of the camera unit relative to the moving vehicle based on the position of the vehicle body marker on the image. It is a feature.

  An eighth aspect of the present invention relates to the straight traveling guidance system for a moving vehicle according to any one of the first to seventh aspects, wherein the far target portion is located on an extension line behind the runway on which the moving vehicle should travel straight ahead. The camera unit is mounted on the moving vehicle so as to image the far target unit.

  According to the present invention, the burden on the driver is reduced, and a highly efficient and highly accurate work can be performed even if it is not an expert. Therefore, according to the present invention, it is possible to cope with changes in the outdoor environment such as the direction of sunlight depending on the weather and time, and it is also possible to cope with noises such as various reflected lights and sunbeams. In addition, the target can be recognized stably in a distance range of 100 m or more, and the apparatus can be reduced in size and cost.

  FIG. 1 is a side view showing a schematic configuration of a straight traveling guidance system for a moving vehicle according to an embodiment of the present invention, and FIG. 2 is a plan view showing a schematic configuration of the straight traveling guidance system for a moving vehicle shown in FIG. FIG. 3 is a block diagram showing a schematic configuration of the straight guidance system for the moving vehicle shown in FIG.

  The straight traveling guidance system for a moving vehicle according to the present embodiment includes a moving vehicle mounting unit 1 and a far target unit 2. The moving vehicle mounting unit 1 is generally configured by a camera unit 11, an image processing unit 12, a storage unit 13, a vehicle control unit 14, a steering control mechanism 15, and a display / operation unit 16. The mobile vehicle mounting unit 1 is mounted on a mobile vehicle 3 composed of an agricultural tractor in the examples of FIGS. 1 and 2. The camera unit 11 is installed indoors, on the roof, or on the front part of the moving vehicle 3 so that the imaging direction is directed to the traveling direction of the moving vehicle 3. In the example of FIG. 2, the camera unit 11 is mounted on the vehicle center line MC in the room or on the roof of the moving vehicle 3 and toward the front front of the moving vehicle 3. Moreover, the camera part 11 is attached with a shallow depression angle so that a horizontal line may enter the visual field.

  In the straight traveling guidance system for a moving vehicle according to the present embodiment, the distance of the distant target portion 2 that can be detected is set to a distance of 100 m or more in consideration of the size of the domestic field, while the distance on the vicinity side is 5 m. If it can be detected to a certain extent, it is considered practically sufficient.

  The camera unit 11 includes, for example, an image sensor, an optical system, and the like. The camera unit 11 captures the far target unit 2 and outputs an analog image signal corresponding to the captured image for each frame. The image sensor is made of, for example, a CCD, a CMOS, etc., and is arranged in a matrix at predetermined intervals. The image processing unit 12 includes a central processing unit (CPU), a digital signal processor (DSP), a sequencer, and the like. The image processing unit 12 executes a remote target detection process based on a remote target detection program or the like stored in the storage unit 13 to convert an analog image signal output from the camera unit 11 into digital image data. After the analog / digital conversion processing, processing / calculation is performed, the position of the far target unit 2 in the obtained image data is detected, and the position information of the far target unit 2 is supplied to the vehicle control unit 14. Similarly, the camera unit 11 includes a plurality of image sensors such as a stereo camera, and an image obtained from one of the plurality of image sensors is a processing target. It is a form.

  In the image obtained from the camera unit 11, the range in which the far target unit 2 can be detected is, for example, 512 pixels wide and 400 pixels long. Further, the viewing angle of the camera unit 11 is, for example, about 26 ° in the horizontal direction and about 20 ° in the vertical direction.

  For example, when the image processing unit 12 has a CPU, when a remote target detection program is read from the storage unit 13, the remote processing target detection program is read into the image processing unit 12 and controls the operation of the image processing unit 12. When the far target detection program is activated, the image processing unit 12 executes the far target detection process described above under the control of the far target detection program.

  The storage unit 13 includes a RAM, a ROM, or a semiconductor memory such as a flash memory. The storage unit 13 stores a remote target detection program and other programs executed by the image processing unit 12, image data for each frame supplied from the image processing unit 12, and the like. The storage unit 13 is used for work when the image processing unit 12 executes the far target detection program. Note that various programs and some data stored in the storage unit 13 may be stored in a memory provided in the CPU or the like constituting the image processing unit 12.

  The vehicle control unit 14 includes a CPU, a DSP, a sequencer, and the like. The vehicle control unit 14 controls the steering control mechanism 15 based on the position information of the far target unit 2 supplied from the image processing unit 14, and steers the moving vehicle 3 toward the far target unit 2. . For details of the steering control mechanism 15, refer to, for example, Japanese Patent Laid-Open No. 2003-118615. The display / operation unit 16 includes, for example, a liquid crystal panel, various switches, various buttons, and the like, and is used when an operator activates the mobile vehicle mounting unit 1 and makes various settings.

  The mobile vehicle 3 on which the mobile vehicle mounting portion 1 described above is mounted performs a work process such as sowing and setting up in the field 4 shown in FIGS. 1 and 2. Note that a vehicle body marker 5 is attached to the approximate center of the front upper portion of the moving vehicle 3 as shown in FIGS. 1 and 4. In the example of FIG. 4, the vehicle body marker 5 has a substantially triangular pyramid shape.

  On the other hand, as shown in FIGS. 1 and 2, the far target unit 2 is placed on the ground near the farm field 4 with the front facing the farm field 4 side. Specifically, when the farm field 4 has a rectangular shape shown in FIG. 2 and the moving vehicle 3 performs a work process from one long side 4a side of the rectangular area constituting the farm field 4, for example, the far target unit 2 Is placed on an extension line of the vehicle center line MC1 when the moving vehicle 3 performs the first work process, while being in contact with one short side 4b of the rectangular area constituting the farm field 4.

  As shown in FIGS. 1 and 5, the far target unit 2 includes a light emitting unit 21, a control unit 22, and a power source 23 housed in a housing 24. As the casing 24, for example, a synthetic resin casing that accommodates a harvested product such as potato can be used. The dimensions of the housing 24 are, for example, about 55 cm in length and about 40 cm in width. A control unit 22 and a power source 23 are fixed to the back surface of the housing 24. On the front surfaces of the control units 22 and 23, a partition plate 25 having substantially the same area as the back surface of the casing 24 and having a rectangular shape is provided. A light emitting unit 21 is attached to the approximate center of the partition plate 25. The light emitting unit 21 is configured by a plurality of light emitting diodes (LEDs) arranged in a matrix at predetermined intervals. As the LED, for example, an LED that emits orange light with high luminance used for a turn signal of a vehicle is preferable.

  In addition, the ratio of the vertical width to the horizontal width of the light emitting unit 21 is preferably in the range of 2: 1 to 4: 1, for example. Moreover, the outline shape of the light emission part 21 shall be substantially rectangular shape, substantially elliptical shape, or a substantially circular shape, for example. When the contour shape of the light emitting unit 21 is a substantially rectangular shape, for example, 200 LEDs are arranged in 25 rows and 8 columns in a range of about 30 cm in length and about 10 cm in width. The control unit 22 controls blinking of all LEDs constituting the light emitting unit 21 at a predetermined cycle.

  The predetermined period is not less than the total of the imaging period of the camera unit 11 and the maximum exposure time, and lighting for a preset time in a range that is not more than twice the imaging period, This is a cycle that repeats turning off the light for a preset time in a range that is equal to or longer than the total of the imaging cycle and the maximum exposure time of the camera unit 11 and less than twice the imaging cycle. For example, when the imaging period of the camera unit 11 is about 0.1 second, all the LEDs constituting the light emitting unit 21 are simultaneously turned on for about 0.2 seconds and turned off for about 0.2 seconds. The control unit 22 is set.

  The reason why the predetermined cycle is set will be described. In the present embodiment, from the viewpoint of cost reduction, the flashing of the light emitting unit 21 and the imaging of the camera unit 11 are not synchronized, and the far target unit 2 and the moving vehicle mounting unit 1 are independently operated at the light emitting unit 21. And the camera unit 11 is controlled. As described above, when the imaging period of the camera unit 11 is about 0.1 seconds, the light emitting unit 21 is controlled to repeat lighting for about 0.1 seconds and turning off for about 0.1 seconds at the same time. When the unit 22 controls, for example, the light emitting unit 21 changes from lighting to off during the exposure of the camera unit 11, and the light emitting unit 21 lights up from the off state during the exposure of the camera unit 11 after the next 0.1 second. May change to In such a case, the brightness of the portion corresponding to the light emitting unit 21 in the image obtained from the camera unit 11 is intermediate brightness, and there is a possibility that the situation does not change with time. .

  Therefore, during the exposure of the camera unit 11, it is possible to reliably capture either an image in which the light emitting unit 21 is continuously turned on or an image in which the light emitting unit 21 is continuously turned off. Therefore, as described above, the control unit 22 is set so that all the LEDs constituting the light emitting unit 21 are simultaneously turned on for about 0.2 seconds and turned off for about 0.2 seconds. Moreover, if it becomes longer than twice the imaging cycle, the time interval at which the position of the light emitting unit 21 can be detected becomes unnecessarily long, resulting in inconvenience that the system performance is degraded.

  Next, the operation of the straight traveling guidance system for a moving vehicle having the above configuration will be described. First, as shown in FIG. 2, when the camera unit 11 is mounted on the vehicle center line MC <b> 1 of the moving vehicle 3, the worker is supposed to guide the moving vehicle 3 (hereinafter referred to as “guidance locus”). .) Place the far target unit 2 with the light emitting unit 21 facing the moving vehicle 3 so that the center line of the light emitting unit 21 is on the extended line of GL1. In the example of FIG. 2, the vehicle center line MC1 and the guidance locus GL1 overlap.

  On the other hand, when the camera unit 11 is mounted offset to the left or right from the vehicle center line MC1 of the moving vehicle 3, the operator can set the same amount as the offset amount due to the mounting position of the camera unit 11, The far target portion 2 is placed with the light emitting portion 21 facing the moving vehicle 3 at a position where the center line of the light emitting portion 21 is offset with respect to the guidance locus GL1.

  Next, the worker turns on the power switch of the power source 23 constituting the far target unit 2 to start the blinking operation of the light emitting unit 21. In this case, assuming that the imaging period of the camera unit 11 is about 0.1 seconds, all the LEDs constituting the light emitting unit 21 are turned on for about 0.2 seconds and turned off for about 0.2 seconds all at once. The control unit 22 is set so as to repeat. As a result, all the LEDs that are controlled by the control unit 22 and constitute the light emitting unit 21 are repeatedly turned on for about 0.2 seconds and turned off for about 0.2 seconds.

  Next, the operator drives the moving vehicle 3 and enters the farm field 4 and positions the moving vehicle 3 on the guidance trajectory GL1 so that the forward direction faces the far target portion 2 as shown in FIG. The mounting unit 1 is activated. As a result, the camera unit 11 starts photographing of the far object unit 2 with an imaging cycle of about 0.1 seconds, and the image processing unit 12 of the moving vehicle mounting unit 1 initializes each unit, and thereafter, FIG. 6 and FIG. The far target detection process shown in the flowchart is executed. Here, the blinking of all the LEDs constituting the light emitting unit 21 and the image capturing of the camera unit 11 are not synchronized, and the far target unit 2 and the moving vehicle mounting unit 1 are independent of each other with the light emitting unit 21 and the camera unit at independent operation timings. 11 is controlled.

  First, the image processing unit 12 proceeds to the process of step S1 shown in FIG. 6 and determines whether or not an image signal for one frame is supplied from the camera unit 11. If the determination result in step S1 is “NO”, the image processing unit 12 repeats the determination. On the other hand, the camera unit 11 shoots the distant target unit 2 placed on the ground near the farm field 4 with the front side facing the farm field 4 in an imaging cycle of about 0.1 seconds, and corresponds to the photographed image 1 Image signals for frames are sequentially supplied to the image processing unit 12.

  Thereby, the determination result of step S1 is “YES”, and the image processing unit 12 proceeds to step S2. In step S2, the image processing unit 12 performs analog / digital conversion processing of the analog image signal for one frame supplied from the camera unit 11 into digital image data, stores the digital image data in the storage unit 13, and then proceeds to step S3. . In this case, the size of the light emitting unit 21 30 cm long on the image is less than 2 pixels 100 m away. However, in the actual image, the image of the light emitting unit 21 expands due to the influence of the aberration of the optical system constituting the camera unit 11 and it becomes possible to detect the far target unit 2 100 m or more away.

In step S3, the image processing unit 12 performs the first window position setting process, and then proceeds to step S4. The first window position setting process will be described below. First, examples of images taken by the camera unit 11 and digitized by the image processing unit 12 are shown in FIGS. FIG. 8 is an example of an image (hereinafter, referred to as “image of the previous frame”) that has been captured and digitized during the imaging cycle immediately before the current time (that is, approximately 0.1 seconds before). In FIG. 8, the far target unit 2 exists at substantially the center, and a window W _-1 surrounding the far target unit 2 and performing detection processing is set. The size of the window W ₋₁ is, for example, about 56 pixels wide and about 40 pixels long. The sizes of the other first windows W _{0 and the} like are substantially the same. On the other hand, FIG. 9 shows an example of an image (hereinafter referred to as “image of the current frame”) that is captured and digitized during the next imaging cycle of the image shown in FIG. 8 (that is, after about 0.1 second). It is.

When the moving vehicle 3 travels forward, the traveling direction and the inclination of the vehicle body change according to the surface state of the field 4 and the like, so that the image captured by the camera unit 11 is also captured at the previous imaging cycle. In many cases, the image is shifted up and down, left and right, or both, as compared with the displayed image. The first window W ₀ in the current frame image shown in FIG. 9 is the same position as the window W ₋₁ in the previous frame image shown in FIG. 8, ie, the same position from the reference point BP in the one frame image. It is set. However, since the moving vehicle 3 has moved from the position where the image shown in FIG. 8 is taken, the relative position of the far target portion 2 in the window W- ₁ in the image of the previous frame shown in FIG. 8 and the current frame shown in FIG. This is different from the relative position of the far target portion 2 in the first window W ₀ in this image. Therefore, when the difference between the image shown in FIG. 8 and the image shown in FIG. 9 is simply calculated in such a state, it is difficult to accurately extract the flashing light emitting unit 21.

Therefore, a correspondence search process for extracting an image portion corresponding to the window W ₋₁ in the image of the previous frame shown in FIG. 8 from the image of the current frame shown in FIG. 9 is performed. First, in the current frame image shown in FIG. 9, a search range SA as shown by a broken line is set around the same position as the window W- ₁ in the previous frame image shown in FIG. The search range SA is determined in consideration of, for example, the speed of movement of the moving vehicle 3 assumed in about 0.1 seconds that is the imaging cycle, the calculation processing capability of the image processing unit 12, and the like, for example, up and down, left and right Each of these is searched for a range of about 10 pixels.

In this correspondence search process, for example, a method is used in which, within the search range SA of the current frame image shown in FIG. 9, a portion having the maximum correlation value with the window W ₋₁ in the previous frame image is used. As this correlation value, for example, a value obtained by a known method such as a sum of squares or a sum of absolute values of differences between two frames of images can be applied. The image processing unit 12 extracts the image portion corresponding to the window W ₋₁ in the previous frame image shown in FIG. 8 from the current frame image shown in FIG. Is set as the first window W ₀ in the image and the position is stored in the storage unit 13. By performing this process, the visual field range in which the far target unit 2 can be detected can be widened by moving the window over the entire image of the camera unit 11 and, for example, an automobile with blinking blinkers Even if it enters into the peripheral part within the field of view of the camera unit 11, it can be made less susceptible to the influence. Further, limiting the detection process within the window can greatly contribute to shortening of the processing time.

In step S4, the image processing unit 12 performs the second window position setting process, and then proceeds to step S5. The second window position setting process will be described below. First, as in step S3, an image taken by the camera unit 11 and digitized by the image processing unit 12 is taken at the time of an imaging cycle two times before the current time (not shown) (that is, about 0.2 seconds before). There is a photographed and digitized image (hereinafter referred to as “image of the previous frame”). In the image of the previous frame, a window W _- 2 that surrounds the far target portion 2 and is a range in which detection processing is performed is set. The size of the window W- _{2 is the same as} the size of the window W- ₁ . The moving vehicle 3 travels forward for about 0.2 seconds before the imaging of the frame and the current frame, and the traveling direction and the inclination of the vehicle body change. Or it is often shifted to the left or right or both.

Therefore, correspondence search processing is performed to extract an image portion corresponding to the window W _-2 in the image of the previous frame from the image of the current frame shown in FIG. First, similarly to step S3, in the current frame image shown in FIG. 9, a search range SA as indicated by a broken line is set around the same position as the window W- ₂ in the previous frame image. As in step S3, for example, the search range SA searches for a range of about 10 pixels vertically and horizontally.

In this correspondence search process, for example, a method is used in which, within the search range SA of the current frame image shown in FIG. 9, a portion having the maximum correlation value with the window W _-2 in the image of the previous frame is used. As this correlation value, as in step S3, for example, a value obtained by a known method such as a square sum or a sum of absolute values of differences between images of two frames can be applied. Using the method described above, the image processing unit 12 extracts an image portion corresponding to the window W _-2 in the previous frame image from the current frame image shown in FIG. The second window W ₀₂ is set and the position is stored in the storage unit 13.

In step S5, the image processing unit 12 performs the first far target portion candidate pixel extraction process, and then proceeds to step S6. Hereinafter, the first far target portion candidate pixel extraction process will be described. An example of each of the first window W ₀ in the image of the current frame and the window W ₋₁ in the image of the previous frame is shown in the left and center of FIG. In the first far target portion candidate pixel extraction process, the first window W ₀ in the current frame image is compared with the window W ₋₁ in the previous frame image, and the following equations (1) and (2) are obtained. Pixels that satisfy all of the conditions shown are binarized and extracted as first far target portion candidate pixels.

Tdf ≦ | P ₀ (i, j) −P ₋₁ (i, j) | (1)
Tbr ≦ P ₀ (i, j) or Tbr ≦ P ₋₁ (i, j) (2)
In Expressions (1) and (2), P ₀ (i, j) is the luminance of each pixel in the first window W ₀ in the image of the current frame, and P ₋₁ (i, j) is the previous frame. The luminance of each pixel in the window W ₋₁ , Tdf is a first threshold relating to the luminance difference, and Tbr is a second threshold relating to the luminance. For example, when the image is digitized with 256 gradations, Tdf is about 36 and Tbr is about 160.

  That is, in the first far target portion candidate pixel extraction process, the luminance difference between the current frame image and the previous frame image is greater than or equal to the first threshold Tdf, and the second difference is detected in any image. A pixel having a luminance greater than or equal to the threshold value Tbr is binarized and extracted as a candidate pixel that is highly likely to be the far target portion 2, that is, the first far target portion candidate pixel.

Here, an example of the first far target portion candidate pixel is shown on the left side of FIG. In the example on the left in FIG. 10B, the light emitting unit 21 is lit in both the first window W ₀ in the current frame image and the window W ₋₁ in the previous frame image. This indicates that there is no luminance difference in the portion, and it has not been extracted as the first far target portion candidate pixel.

On the other hand, for example, when the branches and leaves of the trees in the background of the far target unit 2 are shaken by the wind, or when extracting the image portion corresponding to the window W ₋₁ in the image of the previous frame in the correspondence search process in step S3, When an error or a residual occurs in the position of the window W ₋₁ and a position shift occurs, an erroneous extraction may occur as the first far target portion candidate pixel. In the example on the left in FIG. 10B, the white portion shown at the top indicates an example of a pixel erroneously extracted other than the far target portion 2 described above.

In step S6, the image processing unit 12 performs the second far target portion candidate pixel extraction process, and then proceeds to step S7. Hereinafter, the second far target portion candidate pixel extraction process will be described. An example of the window W _-2 in the image of the previous frame is shown on the right side of FIG. In the second far target portion candidate pixel extraction process, the second window W ₀₂ in the current frame image (not shown) is compared with the window W _-2 in the previous frame image, and the expressions (3) and ( Pixels satisfying all the conditions shown in 4) are binarized and extracted as second far target candidate pixels. Here, since the position of the window is reset so that the portion of the light emitting unit 21 becomes the center of the window after the detection processing of the far target unit 2, the window W- ₁ and the window W- _{2 are set.} Are substantially the same position, and therefore, the first window W ₀ and the second window W ₀₂ in the image of the current frame are also substantially the same position.

Tdf ≦ | P ₀ (i, j) −P ₋₂ (i, j) | (3)
Tbr ≦ P ₀ (i, j) or Tbr ≦ P ₋₂ (i, j) (4)
In Expressions (3) and (4), P ₀ (i, j) is the luminance of each pixel in the second window W ₀₂ in the current frame image, and P ₋₂ (i, j) is the previous time. The luminance of each pixel in the window W _-2 in the image of the frame, Tdf is a first threshold relating to the luminance difference, and Tbr is a second threshold relating to the luminance.

The reason for performing the second far target portion candidate pixel extraction process is as follows. That is, in the present embodiment, the lighting time and extinguishing time of the light emitting unit 21 are set to about twice the imaging cycle of the camera unit 11. Therefore, when the current frame image and the previous frame image are both images of the light emitting unit 21 in the lit state, or conversely, both are images of the light emitting unit 21 in the unlit state. There is. Therefore, in the first far target portion candidate pixel extracted by the difference processing between the first window W ₀ in the current frame image and the window W ₋₁ in the previous frame image, the target far target portion 2 may not be extracted. Therefore, in this second distal target portion candidate pixel extraction process, with respect to the window W _-2 in the second window W ₀₂ and before the previous frame of the image in the image of the current frame, the above-mentioned equations (3) and ( The second far target portion candidate pixel is extracted by performing the difference processing by applying the condition shown in 4).

  That is, in the second far target portion candidate pixel extraction process, the luminance difference between the current frame image and the previous frame image is greater than or equal to the first threshold value Tdf, and any image has the first difference. A pixel having a luminance greater than or equal to the threshold value Tbr of 2 is binarized and extracted as a candidate pixel that is highly likely to be the far target portion 2, that is, a second far target portion candidate pixel.

Here, an example of the second far target portion candidate pixel is shown on the right side of FIG. Right example of FIG. 10 (b), although the light emitting portion 21 in the second window W ₀₂ in the image of the current frame is the lighting state, the light emitting portion 21 in the window W _-2 in the image of the second previous frame is turned off It is. Therefore, there is a luminance difference in the portion of the light emitting unit 21, indicating that it has been extracted as the second far target portion candidate pixel. In the right example of FIG. 10B, the white portion shown in the upper part is a pixel that is erroneously extracted other than the above-described far target portion 2 as in the left example of FIG. 10B. An example is shown.

  In step S7, the image processing unit 12 performs the integration process, and then proceeds to step S8. Hereinafter, the integration process will be described. In this integration process, the first far target portion candidate pixel obtained in step S5 and the second far target portion candidate pixel obtained in step S6 are integrated based on the logical sum condition. As a result, one piece of image data is generated. Here, FIG. 10C shows an example of the first far target portion candidate pixel shown on the left of FIG. 10B and an example of the second far target portion candidate pixel shown on the right of FIG. An example of the result of integrating and is shown. In this way, by integrating the far target portion candidate pixels into one, the processing after step S8 described later can be simplified.

  In step S8, the image processing unit 12 performs the grouping / evaluation process, and then proceeds to step S9 shown in FIG. Hereinafter, the grouping / evaluation process will be described. Since the image data of the far target portion candidate pixels extracted by the integration process shown in step S7 includes pixels due to noise or the like, these pixels need to be removed. Each pixel constituting the image data of the extracted far target candidate pixel is binarized, and these pixels are adjacent to each other. For example, if a certain pixel is a far target candidate pixel, Then, the pixels adjacent in the four directions of top, bottom, left and right are examined, and if they are far target part candidate pixels, they are grouped as the same group to generate a pixel group. Next, the area, aspect ratio, center coordinates, and the like are calculated for each pixel group. Next, a pixel group whose area is smaller than a predetermined threshold or whose aspect ratio is extremely larger than the predetermined threshold is regarded as noise and removed. Further, even in the pixel group that remains without being removed, if the projection includes an unnatural projection or the like, the projection is removed and shaped. In this way, by grouping individual far target candidate pixels into pixel groups, it is possible to extract new feature quantities such as area and aspect ratio, and further evaluate these feature quantities to determine noise, etc. There is an effect that it can be discriminated.

  In step S9, the image processing unit 12 performs temporal tracking processing, and then proceeds to step S10. Hereinafter, the temporal tracking process will be described. A pixel group that has already been supplied from the camera unit 11 before the processing related to the current frame image, extracted by performing the above-described steps S1 to S8, and stored in the storage unit 13 is referred to as a registered group. In this temporal tracking process, the correspondence between the plurality of pixel groups extracted by the processes in steps S1 to S8 related to the image of the current frame and the plurality of registered groups stored in the storage unit and read out. For time tracking.

  In the search for the correspondence relationship, first, the similarity of parameters such as the area, aspect ratio, and center coordinates of each pixel group is evaluated, and a combination in which the similarity is equal to or greater than a predetermined threshold value and has a maximum value is obtained. A corresponding pixel group is determined. Next, the registered group parameters stored in the storage unit 13 are updated with the parameters of the plurality of pixel groups extracted by the processes in steps S1 to S8 related to the image of the current frame. In addition, among the plurality of pixel groups extracted by the processes in steps S1 to S8 related to the image of the current frame, a pixel group that does not have a corresponding relationship with the registered group is added to the storage unit 13 as a new registered group. In this way, by performing the temporal tracking process, the parameter calculation process in step 10 described later becomes possible.

  In step S10, the image processing unit 12 performs parameter calculation processing, and then proceeds to step S11. Hereinafter, the parameter calculation process will be described. In this parameter calculation process, the image processing unit 12 averages the luminance in the image of the current frame (hereinafter referred to as “current luminance average value”) for each pixel constituting each registered group stored in the storage unit 13. And an average value of luminance in the image of the previous frame (hereinafter referred to as “previous luminance average value”), and then a value obtained by subtracting the previous luminance average value from the current luminance average value (hereinafter referred to as “luminance difference”). ) Is stored in the storage unit 13. When the luminance difference is a positive value, the light emitting unit 21 is brightened, that is, it can be recognized as “lighting”, while when the luminance difference is a negative value, the light emitting unit 21 is dark. That is, it can be recognized that it is “lights out”.

  For example, when the flashing cycle of the light emitting unit 21 is about 0.4 seconds, the value of the luminance difference is stored for 2 to 4 seconds, for example, for the past 5 to 10 flashing cycles. Store in the unit 13. Next, the image processing unit 12 analyzes the time interval between “brighter” and “darker” from the brightness difference value for each image of each frame stored in the storage unit 13, and periodically blinks. And whether or not the blinking cycle is calculated.

  Next, for each registered group, the image processing unit 12 stores, for example, the value of the center coordinate on the set window W in the storage unit 13 for, for example, processing for the past 10 to 20 times, and the center coordinate therebetween. After calculating the standard deviation (variance) value as the degree of variation, the value is stored in the storage unit 13.

  In step S11, the image processing unit 12 performs the far target portion determination process, and then proceeds to step S12. Hereinafter, the far target portion determination process will be described. As shown in FIG. 10C, the image data of the far target portion candidate pixels extracted by the integration process shown in step S7 described above includes a noisy pixel in addition to the far target portion 2 pixels. Among the extracted registration groups, there are also registration groups due to noise factors.

  First, the temporal change in brightness of an image that occurs when the branches and leaves of trees in the background of the far target unit 2 sway in the wind may be periodic. However, pixels that exhibit such a periodic change in brightness tend to have a greater degree of variation in the detected position than the fixed far target portion 2, and the value of the standard deviation of the center coordinate described above. Therefore, when the standard deviation exceeds a predetermined threshold, it is discriminated / excluded from the registered group by this index.

  Next, when the moving vehicle 3 travels forward, the traveling direction and the inclination of the vehicle body change according to the surface state of the farm field 4 and the like. Compared to images taken periodically, there are many cases where the entire image is shifted vertically or horizontally or both. Therefore, as described above, in steps S3 and S4, the first and second window position setting processes are performed. Due to the error and residual of the window setting position at this time, the extracted registration groups are not registered. There are also registered groups due to noise factors. However, noisy pixels due to such factors tend not to cause periodic brightness changes. For this reason, when the value of the detected blinking cycle is significantly different from the blinking cycle of the light emitting unit 21 constituting the far target unit 2, or when the intensity of the periodicity is equal to or less than a predetermined threshold, this periodicity It is discriminated and excluded from the registered group by the index of.

  Furthermore, for the remaining registered groups, the values of the standard deviation of the central coordinates, the blinking period and the strength of periodicity, the area of each registered group, the aspect ratio, the number of detections, etc. As the evaluation, the score is integrated and integrated, and the registered group whose value is equal to or greater than a predetermined threshold value and the maximum is determined to be the far target unit 2.

  In step S12, the image processing unit 12 performs control data output processing, and then proceeds to step S13. Hereinafter, the control data output process will be described. In this control data output process, the image processing unit 12 uses the center coordinates icl in the horizontal direction in the entire image as shown in FIG. After comparing the coordinate value iv in the entire image of the front direction of the moving vehicle 3 that is measured in advance and calculating the deviation angle θ1 between the moving vehicle 3 and the direction of the far target portion 2 based on the equation (5) To the vehicle control unit 14.

θ1 = tan ⁻¹ ((icl−iv) × PWH) (5)
In equation (5), icl is the i coordinate on the image of the center point of the registered group of the far target unit 2, iv is the i coordinate in the front direction of the moving vehicle 3, and PWH is the viewing angle per pixel. It should be noted that the value of iv is measured and input in advance and stored in the storage unit that constitutes the vehicle control unit 14, or the camera unit 11 is installed correctly facing the moving vehicle 3. One of the premise is that

  Thereby, the vehicle control unit 14 controls the steering control mechanism 15 based on the deviation angle θ1 between the moving vehicle 3 and the direction of the far target unit 2, thereby moving the moving vehicle 3 in the direction of the far target unit 2. Drive straight ahead.

  In step S13, the image processing unit 12 performs the far target portion approach detection process, and then proceeds to step S14. Hereinafter, the far target portion approach detection process will be described. In the present embodiment, the image processing unit 12 may generate a distance image by performing stereo image processing or the like prior to the far target portion detection processing. In this case, the distance image data of the portion of the light emitting unit 21 constituting the detected far target unit 2 is extracted, and the distance from the moving vehicle 3 to the far target unit 2 is calculated. Alternatively, it is possible to determine that the distance to the far target unit 2 has approached when the size of the image group determined as the far target unit 2 is equal to or greater than the threshold. Then, when the moving vehicle 3 approaches a preset distance, the image processing unit 12 transmits a message to that effect to the moving vehicle 3 and alerts the driver by sounding an alarm in the moving vehicle 3. You can go. In addition, this far target part approach detection process is arbitrarily performed.

  In step S14, the image processing unit 12 performs duplication evaluation / integration processing of registered groups, and then proceeds to step S15. The registration group duplication evaluation / integration process will be described below. In the temporal tracking process shown in step S10 described above, when a parameter is changed due to a disturbance or the like added to the original pixel group generated based on the image of the far target portion 2 to be detected, the pixel group has been changed so far. In some cases, it is determined that the pixel group is different from the registered group that is tracked and registered in the storage unit 13. As a result, the far target unit 2 may be registered in the storage unit 13 as being a new pixel group, and the registered group of the far target unit 2 may be duplicated in the storage unit 13 in some cases.

  When such a situation occurs, in the temporal tracking process shown in step S9 for the subsequent frames and thereafter, one pixel group generated based on the image of the far target unit 2 is converted into a plurality of different registration groups. Is determined to be a pixel group corresponding to itself, and there is a possibility that a situation in which correct temporal tracking processing is not performed may occur.

  Therefore, in the overlapping evaluation / integration processing of the registered groups, the image processing unit 12 compares the parameters with each other for a plurality of registered groups that are already registered in the storage unit 13. If there are a plurality of registration groups that are judged to be highly similar and track the same object, a process for integrating such a plurality of registration groups into one registration group is regularly performed. Run it.

In step S15, the image processing unit 12 performs an in-window image update process for the current frame, and then proceeds to step S16. Hereinafter, the in-window image update process of the current frame will be described. In the in-window image update process of the current frame, the image processing unit 12 uses the detected center coordinates on the image of the registered group of the far target unit 2 as the centers of the first and second windows W ₀ and W _02. As described above, the positions of the first and second windows W ₀ and W ₀₂ are reset and stored in the storage unit 13 again as the first window W ₀ in the current frame. The first and second windows W ₀ and W ₀₂ are set, for example, as horizontal 56 pixels and vertical 40 pixels.

  In step S16, the image processing unit 12 performs a window-related exposure time control process, and then terminates a series of processes related to the current frame, and performs step S1 shown in FIG. 6 in order to perform imaging and image processing of the next frame. Return to. The window-corresponding exposure time control process will be described below. In this window-corresponding exposure time control process, the exposure time of the camera unit 11 is set so that the luminance distribution within the range of the window W is optimized for detection of the far target unit 2. Control.

  That is, for example, when the exposure time of the camera unit 11 is controlled for the entire image, the far target unit 2 and its surroundings are bright in the entire image and the brightness is relatively high. The brightness of the area to be included is saturated, and the far target unit 2 installed in the area cannot be detected. On the other hand, when the exposure time of the camera unit 11 is controlled so that the set window W is targeted and the luminance distribution within the range of the window W is optimized for detection of the far target unit 2 Even if the far target section 2 is set in a bright area compared to the entire image, the far target section 2 can be detected. For example, the sensitivity of imaging such as exposure time is controlled so that the luminance of all pixels in the window is histogrammed and the areas of the dark part and the bright part become equal.

  As described above, according to the embodiment of the present invention, the distant portion provided with the light emitting portion 21 in which a large number of LEDs that are relatively inexpensive, have high light emission efficiency, and can easily configure the contour shape and the overall size of the light emitting portion are arranged. The target unit 2 is installed to give a characteristic to the adjustment / setting of the camera unit 11 that captures the flashing of the light emitting unit 21, and further has a difference process and a judgment process characteristic of the image processing in the image processing unit 12. .

  Therefore, according to the embodiment of the present invention, it is possible to cope with changes in the outdoor environment such as the direction of sunlight depending on the weather and time, and it is also possible to cope with various reflected light and noise such as sunbeams. . In addition, the target can be recognized stably in a distance range of 100 m or more, and the apparatus can be reduced in size and cost.

  In addition, according to the embodiment of the present invention, since the synchronization of the blinking of the camera unit 11 and the light emitting unit 21 is omitted, the light emitting unit 21 is turned on or off or is turned on when the camera unit 11 is imaging. The image of the light emitting unit 21 to be imaged may have a state in which the luminance is intermediate between lighting and extinguishing. Assuming that the lighting and extinguishing time of the light emitting unit 21 is the same as the imaging cycle of the camera unit 11, the images captured by the camera unit 11 are continuously turned on and off or from off to on. The image 21 continues to be in an intermediate luminance state, and the temporal change in luminance becomes small, making it difficult to detect the flashing light emitting unit 21.

  Therefore, according to the embodiment of the present invention, the light emitting unit 21 has a range that is equal to or longer than the sum of the imaging cycle and the maximum exposure time of the camera unit 11 and equal to or less than twice the imaging cycle. It is set to repeat the lighting of the time and the turning off of the time in this range. As a result, even if the light emitting unit 21 is turned on or off during imaging by the camera unit 11 in a certain frame, or is turned on from off, the next frame is surely kept on during the exposure time or turned off all the time. The change in luminance of the image of the light emitting unit 21 can be maximized, and the detection performance of the light emitting unit 21 can be improved.

  In this regard, if the lighting time or the light-off time of the light emitting unit 21 is lengthened, the image of the camera unit 11 is easily turned on or turned off all the time. However, in the embodiment of the present invention, the light is turned off or turned off. Since the light-emitting part 21 is detected by the change in lighting, the opportunity to detect the light-emitting part 21 is reduced and the original function of the system is lowered.

  In the embodiment of the present invention, “a time in a range that is equal to or longer than the total of the imaging period and the maximum exposure time of the camera unit 11 and equal to or less than twice the imaging period” is surely the exposure time. This is a setting that maximizes the original function of the system, with a state in which the state is always lighted or lighted off and is the minimum time. In many cases, the maximum exposure time of the camera unit 11 is substantially the same as the imaging cycle. Therefore, the light emitting unit 21 has a lighting time that is not more than twice the imaging cycle and a turn-off time that is not more than twice the imaging cycle. It becomes setting to repeat.

  Further, according to the embodiment of the present invention, the image processing unit 12 extracts an image portion corresponding to the window W set as a range for detecting the position of the far target unit 2 in the entire image of the previous frame. Search from the entire image of the current frame, set the searched image portion as the first window in the image of the current frame, and image portion corresponding to the window set in the entire image of the previous frame, The entire image of the current frame is searched for, and the searched image portion is set as the second window in the image of the current frame. Then, based on the first and second windows, first and second far target portion candidate pixels are extracted. Therefore, even in a situation where the moving speed is fast and the image captured for each frame changes greatly vertically and horizontally, the corresponding image is searched, so that the present system can be applied.

  According to the embodiment of the present invention, in the current frame image and the previous frame image captured by the camera unit 11, the absolute value of the luminance difference of the corresponding pixels is equal to or greater than the first threshold, and When the luminance of any pixel is equal to or higher than the second threshold, the pixel is extracted as a candidate pixel of the far target unit 2. Therefore, the luminance feature on the image can be accurately detected for the flashing light emitting unit 21.

  Further, according to the embodiment of the present invention, in addition to the difference process between the current frame image captured by the camera unit 11 and the previous frame image, the difference process between the current frame image and the previous frame image is also performed. ing. As described above, when the light emitting unit 21 is set to repeatedly turn on and off the total time of the imaging cycle and the maximum exposure time of the camera unit 11 and turn off the total time or more, the light emitting unit 21 is turned on in the current frame. Alternatively, even if a light-off image is obtained, the previous frame may also be a light-on or light-off image, or conversely a light-on or light-off or light-on-light image. However, the previous frame is surely turned off or turned on opposite to the current frame, and the light emitting unit 21 can be detected.

  In addition, according to the embodiment of the present invention, the amount of change in luminance of the image portion is stored over the past several seconds, the amount of change is analyzed to detect the periodicity of blinking and its time interval, When the periodicity is high and the time interval coincides with the set time interval of turning on / off the far target portion 2, it is determined that the pixel group is highly likely to be the far target portion 2 to be detected. Accordingly, it is possible to remove pixel groups that are erroneously extracted due to noise factors.

  In addition, according to the embodiment of the present invention, the position of the extracted pixel group in the window is stored for the past several seconds, and the temporal change amount of the coordinate value in the vertical and horizontal directions of this position is stored. When the deviation is calculated and the standard deviation is small, it is determined that the possibility of the far target unit 2 is high. Therefore, in this case as well, it is possible to remove pixel groups that are erroneously extracted due to noise factors.

  In addition, according to the embodiment of the present invention, the camera unit 11 is adjusted precisely in parallel with the direction of the front front of the moving vehicle 3, or in the image of the camera unit 11, The position in the direction needs to be detected accurately. By configuring in this way, as shown in FIG. 12, the camera unit 11 is installed in advance so that the vehicle body marker 5 installed on the engine hood or the like of the moving vehicle 3 is shown. Using the shape pattern as shown in FIG. 4, the position in the image of the camera unit 11 can be detected by the pattern matching technique, and the position in the front direction of the moving vehicle 3 can be automatically detected.

  Further, according to the embodiment of the present invention, the light emitting unit 21 constituting the far target unit 2 has a contour shape of any one of a rectangular shape, an elliptical shape, and a circular shape, and the contour aspect ratio is 2. Between a quarter and a quarter. Usually, the light-emitting portion of the light-emitting unit 21 has a large area, and the more light that is projected in the direction of the camera unit 11, the more advantageous is detection from a distance. However, since the position of the light emitting unit 21 is calculated from the center coordinates of the light emitting unit 21 on the image, the image of the light emitting unit 21 becomes large at a short distance, and the accuracy of the detected position tends to decrease particularly when the horizontal width is widened. It becomes. Therefore, as in the embodiment of the present invention, when the light emitting portion of the light emitting portion 21 is long in the vertical direction and narrow in the horizontal direction, both the area expansion and the accuracy can be ensured. On the other hand, if the light emitting portion of the light emitting portion 21 becomes too long in the vertical direction, the shape of the light emitting portion 21 on the image is similar to a pixel group that is erroneously extracted due to noise factors, so the aspect ratio of the contour is It is optimal to be between 1/2 and 1/4.

  In addition, according to the embodiment of the present invention, the far target portion 2 is painted in a dark color such as black, and a blank portion having a width greater than or equal to the width of the light emitting portion of the light emitting portion 21 is provided around the light emitting portion 21. A partition plate 25 is provided behind the light emitting unit 21. As described above, the camera unit 11 has a structure in which the imaging elements are arranged in a grid pattern. FIG. 13 shows an example of the far target unit 2, but at a far distance near the detection limit, an image obtained by capturing the far target unit 2 becomes small, and the size of one image sensor is one grid shown in FIG. 13. Thus, the output of the image sensor becomes the average brightness of a plurality of objects existing in one grid, and a mosaic image state as shown in FIG. 14 is obtained.

  If there is no background plate around the light emitting unit 21 and there is a bright object behind the light emitting unit 21, both the light emitting unit 21 and the bright background enter the same image sensor, and the light emitting unit 21 is turned off. However, the pixel receives a bright background light and remains in a high luminance state, and the change in luminance is reduced, making it difficult to detect the light emitting unit 21. Therefore, when the partition plate 25 is provided, the influence of the object behind on the image of the light emitting unit 21 can be reduced, and the detection performance is improved particularly at a distant distance.

  Since the image of the light emitting unit 21 needs to be dark when the light is turned off, it is desirable that the partition plate 25 be a color with little light reflection such as black. In the actual camera unit 11, the influence of light from a certain object diffuses not only to the image sensor at that position but also to the surrounding image sensors due to the influence of lens aberration and the like. Therefore, it is desirable that the partition plate 25 has a width around the light emitting unit 21 that is at least as large as the lateral width of the light emitting portion of the light emitting unit 21. Further, a lid may be attached to the casing 24, the back of the lid may be painted black, and the lid may be unfolded when the light emitting unit 21 is used, and used as an extension of the partition plate 25.

  In addition, according to the embodiment of the present invention, the far target portion 2 has a structure in which the light emitting portion 21 is installed on the partition plate 25 provided near or near the bottom of the rectangular container-like casing 24. As shown in FIG. 5, it is considered that the far target unit 2 often uses the light emitter of the light emitting unit 21 in a substantially vertical position. By using the rectangular parallelepiped housing 24, as shown in FIG. 5, the far target section 2 can be stably installed only by placing it on a flat place, and the handling becomes simple. The casing 24 can also serve as a container for storage as it is.

  In addition, when the direct sunlight is applied to the light emitting unit 21, the portion of the light emitting unit 21 shines due to the reflection of sunlight, and the image remains bright even when the light emitting unit 21 is turned off. By installing the light emitting unit 21 in the recessed portion of the rectangular parallelepiped housing 24, the sunshade is also attached to the upper and left and right sides.

  In addition, according to the embodiment of the present invention, sensitivity adjustments such as the exposure time of the camera unit 11, the aperture, and the amplifier gain of the image signal are performed in the window W partially set in the image of the camera unit 11. It is set based on the luminance distribution of the image. In imaging by the camera unit 11, it is important to adjust the sensitivity of the camera unit 11 so that the brightness change due to the flashing of the light emitting unit 21 is maximized with respect to the brightness of the background portion. When sensitivity adjustment is performed on the entire image as in a general camera, the portion of the image of the light emitting unit 21 that is far away and small may not be optimized. Therefore, the detection performance of the light emitting unit 21 is improved by performing sensitivity adjustment on the image in the window W that is the range in which the detection processing of the light emitting unit 21 is performed.

  Here, an example of the experimental result is shown. A tractor is used as the moving vehicle 3, and the vehicle mounting unit 1 is mounted on the tractor to detect a flashing far target unit 2 installed far away outdoors in the daytime, and the moving vehicle 3 becomes the far target unit 2. Traveling control was performed. In the example shown in FIG. 15, the camera unit 11 is mounted on the front part of the tractor, and the state of the kite formed by performing the traveling control toward the far target unit 2 by installing a vertical machine is shown in FIG. FIG. 17 shows the measurement results of the position of the far target portion 2 and the position of the formed eyelid at this time.

  Here, the position of the eyelid was measured by the experimenter, holding the reflecting prism in the approximate center of the eyelid while walking, and measuring the position of the reflecting prism at that time with an automatic tracking laser positioning device. The position of the far target unit 2 is also measured by the reflecting prism and the automatic tracking laser type positioning device.

  The accuracy of the measurement is about ± 1 cm as a single unit of the automatic tracking laser positioning device, but the position of the eyelid is subject to factors such as variations due to visual measurements and the reflection prism swinging due to walking. The fine variation in the lateral direction of the position of the heel shown in FIG. 17 is mainly due to shaking accompanying walking, and the actual heel is smooth as shown in FIG.

  In FIG. 17, the heel is formed with a deviation within about ± 30 mm with respect to the far target portion 2, and it can be seen that the straight guidance toward the good far target portion 2 was performed. This deviation of ± 30 mm includes a swing of the reflecting prism accompanying the walk of the experimenter at the time of measuring the heel position, and the accuracy of the straight guidance is considered to be practically sufficient.

  The tractor changes straight ahead depending on the work implement to be installed and the state of the field, so the driver needs skilled and frequent steering correction to perform linear work. However, it is possible to work with high accuracy and at the same time reduce the burden on the driver by automation. Moreover, it can be said that it is a simple and intuitive operation method of placing the far target unit 2 at a target position.

  In order to detect the far target unit 2, it is necessary that the brightness of the far target unit 2 be sufficiently higher than the brightness of the portion serving as the background of the far target unit 2. While the brightness of the background changes depending on the type and sunshine conditions, the brightness of the far target portion 2 becomes dark in inverse proportion to the square of the distance. For this reason, the distance which can detect the far target part 2 cannot be specified unconditionally. This system has also succeeded in guiding and running by detecting the far target part 2 that is 400 m or more in the cloudy eyes.

The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and design changes and the like within a scope not departing from the gist of the present invention are possible. Even if it exists, it is included in this invention.
For example, in the above-described embodiment, an example in which the present invention is applied to an agricultural tractor has been described. However, the present invention is not limited to this, and can be applied to work by various agricultural vehicles other than the agricultural tractor. This is effective when performing linear movement with high accuracy, when driving straight ahead at low speed for a long time, or when it is desired to perform linear glazing. Further, the present invention can be used not only in the agricultural field but also for direct guidance of work vehicles in the civil engineering and construction fields.

  Further, in the above-described embodiment, the far target unit 2 is installed on the extension line in front of the running path on which the moving vehicle 3 should travel straight, and the camera unit 11 is moved to capture the far target unit 2. Although the example which mounts on the vehicle centerline MC toward the front front of the moving vehicle 3 was shown, it is not limited to this. For example, the distant target unit 2 is installed on an extension line behind the runway on which the moving vehicle 3 should travel straight, and the camera unit 11 is placed indoors or on the roof of the moving vehicle 3 in order to capture the distant target unit 2. Therefore, it may be mounted on the vehicle center line MC toward the rear front of the moving vehicle 3.

1 is a side view showing a schematic configuration of a straight traveling guidance system for a moving vehicle according to an embodiment of the present invention. It is a top view which shows schematic structure of the linear guide system of the moving vehicle shown in FIG. It is a block diagram which shows the structure of the linear guide system of the moving vehicle shown in FIG. It is the schematic which shows an example of a vehicle body marker. It is a perspective view which shows schematic structure of the far target part which comprises the rectilinear guidance system of the moving vehicle shown in FIG. It is a flowchart which shows the far target part detection process which an image process part performs. It is a flowchart which shows the far target part detection process which an image process part performs. It is a figure which shows an example of the image of a front frame. It is a figure which shows an example of the image of the present flame | frame. It is a conceptual diagram for explaining a far target candidate extraction process of a current frame and a previous frame, a far target candidate extraction process of a current frame and a previous frame, and an integration process. It is explanatory drawing which shows the state of the image of the extracted far target part. It is a figure which shows an example of the image by which the vehicle body marker was imaged. It is explanatory drawing in the state of the image of the far target part in long distance. It is a figure which shows an example of the enlarged image of the light emission part which comprises the far target part in a long distance. It is a figure which shows an example of the photograph which image | photographed the tractor with which the vehicle mounting part in the experiment of driving | running | working was mounted, and a far target part. It is a figure which shows an example of the photograph which image | photographed the kite and the far target part which were formed by carrying out driving control experiment with the tractor shown in FIG. It is a figure which shows the position of the formed kite and the position of a far target part which is an example of the experimental result of the traveling control shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Moving vehicle mounting part, 2 ... Distant target part, 3 ... Moving vehicle, 4 ... Agricultural field, 4a ... Long side, 4b ... Short side, 5 ... Body marker, 11 ... Camera part, 12 ... Image processing part, 13 ... Storage unit, 14 ... Vehicle control unit, 15 ... Steering control mechanism, 16 ... Display / operation unit, 21 ... Light emitting unit, 22 ... Control unit, 23 ... Power source, 24 ... Housing, 25 ... Partition plate, SA ... Search Range, W ₀ ... first window in current frame image, W ₀₂ ... second window in current frame image, W _-1 ... window in previous frame image, W _-2 ... window in previous frame image

Claims (8)

A moving vehicle mounting portion mounted on the moving vehicle, and a far target portion installed on an extension line in front of the road on which the moving vehicle should travel straight ahead,
The moving vehicle mounting unit is based on a camera unit that images the far target unit, an image processing unit that processes an image obtained from the camera unit and outputs traveling information for the straight traveling, and the traveling information And a steering control unit for controlling the moving vehicle,
The far target unit is turned on for a time in a range that is equal to or longer than the sum of the imaging cycle and the maximum exposure time of the camera unit, and less than or equal to twice the imaging cycle. Repeatedly turning off
In the image processing unit, the absolute value of the luminance difference between corresponding pixels in the image of the captured current frame and the image of the previous frame captured in the previous imaging cycle is equal to or greater than a first threshold. And when the luminance of any one of the pixels is equal to or higher than the second threshold, the pixel is extracted as the first far target portion candidate pixel, and the image of the current frame and the previous two pixels are extracted. In the image of the previous frame imaged in the imaging cycle, the absolute value of the luminance difference of the corresponding pixel is not less than the first threshold value, and the luminance of any one of the pixels is the second If the threshold is equal to or greater than the threshold, the pixel is extracted as a second far target portion candidate pixel, the first and second far target portion candidate pixels are integrated, and the far end relative to the integrated far target portion candidate pixel Detects the position of the target part and Straight guidance system of the mobile vehicle and generates the travel information based on. The image processing unit searches the entire image of the previous frame for an image portion corresponding to a window set as a range for detecting the position of the far target unit from the entire image of the current frame. The searched image part is set as a first window in the image of the current frame, and an image part corresponding to the window set in the whole image of the previous frame is set as the window of the current frame. Searching from within the entire image, setting the searched image portion as a second window in the image of the current frame, and based on the first and second windows, the first and second remote targets The part candidate pixel is extracted. The straight traveling guidance system for a moving vehicle according to claim 1, wherein: The image processing unit, for each pixel constituting a plurality of pixel groups extracted by grouping the integrated far target portion candidate pixels, brightness in the image of the current frame, and the pixel of the previous frame The blinking of the pixels based on the past few seconds of the difference between the brightness in the image and the past in the difference between the brightness in the image of the current frame and the brightness in the image of the previous frame. 3. The straight traveling guidance system for a moving vehicle according to claim 1, wherein the time interval between periodicity and blinking is detected, and the far target portion candidate pixel is determined as the far target portion based on the detection result. . The image processing unit calculates a standard deviation in temporal change amounts of vertical and horizontal coordinate values for the past several seconds of position values on the window for each of the plurality of pixel groups, and the standard The straight traveling guidance system for a moving vehicle according to claim 2 or 3, wherein when the deviation is smaller than a preset threshold value, the pixel group is determined as the far target portion. The far target portion has a shape of a contour of the light emitting portion that is any one of a rectangular shape, an elliptical shape, and a circular shape, and an aspect ratio of the contour is between a half and a quarter. The straight traveling guidance system for a moving vehicle according to any one of claims 1 to 4. The exposure time of the camera unit, the aperture, and the sensitivity adjustment of the amplifier gain of the image signal are set based on the luminance distribution of the image in the window that is partially set in the image of the camera unit. A straight traveling guidance system for a moving vehicle according to any one of claims 2 to 5. A vehicle body marker that can be imaged together with the far target unit is attached to the moving vehicle,
The image processing unit detects a position of the vehicle body marker from the image, and detects a direction of the camera unit with respect to the moving vehicle based on a position of the vehicle body marker on the image. Item 7. A straight traveling guidance system for a moving vehicle according to any one of Items 1 to 6. The far target portion is installed on an extension line behind the runway on which the moving vehicle should travel straight,
The straight traveling guidance system for a moving vehicle according to any one of claims 1 to 7, wherein the camera unit is mounted on the moving vehicle so as to image the far target portion.