JP2010066117A5 - - Google Patents

Description

Stereo image processing device for work machines

  The present invention is provided in a work machine that performs work outdoors such as a hydraulic excavator and a wheel loader, and has a stereo image processing of the work machine that includes a stereo camera that includes a plurality of cameras for measuring a three-dimensional shape of an object to be photographed. Relates to the device.

  A technique for measuring the three-dimensional shape of an object to be photographed based on the parallax of two or more camera images is disclosed in Patent Document 1. Parallax is a small n × n region of a camera image, for example, finds a corresponding corresponding point from the images of two cameras arranged on the left and right, and indicates where the corresponding point is on the image. Parallax is required. By calculating the distance between corresponding points based on this parallax, the position of a small n × n region is measured. As described above, by calculating the positions of a plurality of small n × n regions, it is possible to realize the three-dimensional shape measurement of the photographing object. In such three-dimensional shape measurement, it is important to find corresponding points. In some cases, three or more cameras are installed for measurement. In such a case, the increased number of cameras is used to increase the position detection accuracy of corresponding points.

  In addition, as a method of finding corresponding points in the n × n region, generally, the absolute value of the light and shade difference between the n × n region of one camera and each pixel in the n × n region of the other camera. An area having a small SAD (Sum of Absolute Differences) obtained by adding n × n is calculated as a corresponding point. For this reason, the difference in brightness when the left and right cameras are photographed becomes an obstacle to the corresponding point search. Therefore, it is necessary to adjust the brightness so that the left and right cameras have the same brightness.

A method of adjusting the brightness of a camera is known as a technique such as auto exposure (AE) or automatic exposure as a general camera. Further, as shown in the prior art disclosed in Patent Document 2, as a single camera, a technique for adjusting the brightness of the entire camera image or a technique for adjusting the brightness in a specific area such as the center of the screen. It has been known.
JP-A-7-253310 JP 58-72131 A

  As shown in Patent Document 2 described above, when the method of individually adjusting the brightness of a camera is applied to a stereo camera as in the present invention, a plurality of devices installed at different positions such as left and right are installed. Images acquired between cameras differ, and the brightness of the left and right cameras with respect to the object to be photographed differs not only when adjusting the brightness at the center of the image but also when adjusting the brightness of the entire image. . This is natural for stereo cameras that measure the shape and distance by obtaining parallax, but can be applied in places where the brightness does not change significantly, such as under indoor lighting (300-500 lux). It is.

  However, the environment such as the outdoors where the work by the work machine having the joint portion is performed is an environment in which the brightness changes greatly from, for example, about 100,000 lux in the daytime to about 1,000 lux one hour before sunset. is there. In the shade, it becomes darker and the brightness changes. Furthermore, there is an individual difference in the increase in pixel brightness of the camera output with respect to the increase in the amount of incident light of the camera. For this reason, in an environment where the brightness varies greatly, there is a problem that even if the brightness is adjusted at a certain brightness, it cannot be used as it is. There is also a problem that the brightness varies with time. Moreover, since brightness changes also in the direction which sunlight irradiates, it is necessary to adjust the brightness in the measurement direction.

The present invention has been made from the actual situation in the above-described prior art, and an object of the present invention is to capture images taken by the respective cameras at the time of measuring a three-dimensional shape in an environment in which the brightness greatly varies, and the brightness of the imaging reference part to each other. An object of the present invention is to provide a stereo image processing apparatus for a work machine that can be acquired as an equivalent captured image.

In order to achieve this object, a stereo image processing apparatus for a work machine according to the present invention is provided in a work machine having a work tool having a joint, and three-dimensionally captures an imaging object positioned around the work tool. In a stereo image processing apparatus of a work machine having a stereo camera composed of a plurality of cameras for measuring a shape , a marker indicating a reference position for brightness adjustment provided on the work tool, and each captured image of the camera and imaging the reference region detector for detecting a photographing reference position in the vicinity of the marker, respectively determined the brightness of the imaging reference position detected by the imaging reference position detecting section for the plurality of cameras, these imaging reference region and brightness comparison processing unit for calculating the difference in brightness, the difference between the calculated the imaging reference position whereby luminance by the luminance comparison processing unit is reduced To is characterized in that it consists of an adjustment unit for adjusting the shutter speed of the respective camera.

In the present invention configured as described above, in an environment where the brightness greatly fluctuates, for example, in a work site such as outdoors where work with a work machine is performed, a plurality of cameras are used to capture an imaging reference region near the marker provided on the work tool. The imaging reference region near the marker is detected by the imaging reference region detection unit. The brightness comparison processing unit calculates the brightness of each imaging reference part near the marker by each camera and the brightness difference of each imaging reference part, and the adjustment part reduces the difference between each imaging reference part by each camera. Thus, the shutter speed of each camera is adjusted. With such a process, it is possible to acquire a photographed image by each camera at the time of three-dimensional shape measurement in an environment in which the brightness varies greatly as a photographed image in which the brightness of the photographing reference part is equal to each other. It is possible to realize highly accurate three-dimensional shape measurement with respect to an imaging target portion located in the vicinity of.

The stereo image processing apparatus for a work machine according to the present invention includes, in the above invention, a monitor that displays a captured image of the camera, and the imaging reference region detection unit displays an image range on the monitor screen by a cursor display. It is characterized by comprising designation means for designating and a brightness comparison position determination processing section for performing processing for determining the image range designated by the designating means as the brightness comparison position.

The stereo image processing apparatus for a work machine according to the present invention is the stereo image processing apparatus according to the present invention, wherein the imaging reference part detection unit includes a rotation angle detector provided at a joint part of the work tool, and a detection signal of the rotation angle detector. wherein the imaging reference region calculating unit for calculating the imaging reference position in the vicinity of the marker, the imaging reference part calculating unit performs processing for determining the brightness comparison position computed photographed reference part in brightness compared position determination process based on the It consists of parts.

According to the present invention configured as described above, a difference in brightness of each imaging reference part by each camera is reduced through the rotation angle detector, the imaging reference part calculation unit, and the brightness comparison position determination processing unit. in the process of adjusting the shutter speed of each camera can automatically be performed without the intervention of the manual operation of the brightness comparison position specified by the O-Perret over data.

  In the invention, the stereo image processing apparatus for a work machine according to the present invention may include a measurement unit that measures the posture of the work tool, a storage unit that stores a shutter speed corresponding to the posture of the work tool, and the measurement. And a processing unit that reads out the shutter speed corresponding to the posture of the work tool measured by the unit from the storage unit and sets the target shutter speed of each camera.

  The present invention configured as described above is a process for setting a target shutter speed according to the posture of the work tool measured by the measurement unit during automatic operation of the work machine performed through measurement of the posture of the work tool by the measurement unit. By the processing of the unit, it is possible to realize the three-dimensional shape measurement of a plurality of photographing objects with high accuracy.

The present invention includes a marker indicating a reference position for the brightness adjustment provided in the implement, and imaging the reference region detection unit that detects a plurality of respective imaging reference position in the vicinity of the captured image marker camera, the imaging reference It obtains the brightness of the photographed reference parts detected by the part detection unit, respectively, and the brightness comparison processing unit for calculating the difference in brightness between these imaging reference site, imaging the reference region, which is calculated by the brightness comparison processing unit An adjustment unit that adjusts the shutter speed of each camera so as to reduce the difference in brightness of each camera is provided. Therefore, an image captured by each camera at the time of three-dimensional shape measurement in an environment where the brightness varies greatly is provided. The captured image can be acquired as a captured image in which the brightness of the imaging reference region is equal to each other. As a result, it is possible to realize highly accurate three-dimensional shape measurement with respect to the imaging target portion located in the vicinity of the work tool, and to improve the work accuracy and work efficiency of work performed on the work machine as compared with the conventional case. it can.

  The best mode for carrying out a stereo image processing apparatus for a work machine according to the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram for explaining the outline of the operation of the stereo image processing apparatus according to the first embodiment of the present invention. FIG. 1 (a) is a perspective view of a main part showing a hydraulic excavator cited as an example of a work machine. The figure shows the screen of the monitor provided in the first embodiment, (c) the figure shows the image where the area where the marker is displayed is cut out from the screen shown in the figure (b), (d) figure (C) is a figure which shows the average brightness of the area of a figure, FIG. 2 is a block diagram which shows the principal part structure of 1st Embodiment of the stereo image processing apparatus based on this invention.

  The first embodiment of the stereo image processing apparatus shown in FIGS. 1 and 2 is provided in a work machine such as a hydraulic excavator, and this hydraulic excavator is not illustrated as shown in FIG. A revolving structure 1 disposed on a traveling body, a boom 2 that is attached to the revolving structure 1 by a boom pin 21 and rotatable in the vertical direction, and an arm pin 31 that is attached to the boom 2 and can be rotated in the vertical direction. And a bucket 4 attached to the arm 3 by a bucket pin 41 and rotatable in the vertical direction. The boom 2, the arm 3, the bucket 4, and the like constitute a work tool having a joint portion for performing excavation work and the like.

The first embodiment includes a work tool provided in the hydraulic excavator, for example , a marker 6 provided on the back surface of the arm 3 and indicating a reference position for brightness adjustment. Moreover, the several camera which image | photographs the marker 6, for example, the 1st camera 51 and the 2nd camera 52 which are arrange | positioned at the right and left of the roof of a driver's cab are provided. The first camera 51 and the second camera 52 constitute a stereo camera.

  Images taken by the first camera 51 and the second camera 52 are displayed as an image 61 on the screen of the monitor 8 as shown in FIG. The image 61 includes an image in which the marker 6 described above is displayed, that is, a marker image 62. In FIG. 1B, the left image 61 is an image taken by the first camera 51, and the right image 61 is an image taken by the second camera 52.

  In the first embodiment, as will be described later, from the image shown in FIG. 1B, an image required for the calculation for obtaining the brightness is the marker image 62 as shown in FIG. Is designated by the operator as an area 63 including. The left area 63 in FIG. 1C is an area designated in the image photographed by the first camera 51, and the right area 63 is designated in the image photographed by the second camera 52. Area.

  In the first embodiment, as will be described later, for example, average brightness A1 and A2 of the image indicated by the area 63 is obtained. The left diagram in FIG. 1D shows the average brightness A1 of the area 63 specified in the image photographed by the first camera 51, and the right diagram was photographed by the second camera 52. The average brightness A2 of the area 63 designated in the image is shown.

  As shown in FIG. 2, the stereo image processing apparatus according to the first embodiment includes, for example, a personal computer 7, the monitor 8 that displays the processing result of the personal computer 7, and a mouse 9 attached to the personal computer 7. I have.

  The personal computer 7 stores the input / output unit 76 connected to the first camera 51 and the second camera 52, and the captured images of the first camera 51 and the second camera 52 input from the input / output unit 76. An image storage unit 75 for displaying the captured image on the monitor 8, a command processing unit 77 for outputting a command signal such as image acquisition to the first camera 51 and the second camera 52, and photographing by the first camera 51 and the second camera 52. And a three-dimensional shape measurement processing unit 74 that performs three-dimensional shape measurement based on the image.

Further, as shown in FIG. 1C, a brightness comparison position determination processing unit 71 that performs a process of determining an image on the screen of the monitor 8 designated by the cursor display by the mouse 9 as a brightness comparison position. I have. A mouse 9 described above, the brightness comparison position determination processing unit 71, to constitute an imaging reference region detector for detecting a photographing reference position in the vicinity of the marker 6 from the first camera 51 and the second camera 52 each captured image Yes.

The comparison was determined in brightness compared position determining processor 71 position, i.e. to determine the average brightness A1, A2 shown in (d) of Figure of Figure 1 described above each imaging standard site specified by the mouse 9, these Is provided with a brightness comparison processing unit 72 for calculating the difference between the average brightness A1 and A2. Assuming that the average brightness A2 of the imaging reference part by the second camera 52 is darker than the average brightness A1 of the imaging reference part by the first camera 51, the average brightness A1-the average brightness A2 is Calculated as the difference in average brightness. Further, the desired brightness A0 is set empirically, and the average brightness A1-A0 and the average brightness A2-A0 are respectively obtained as the desired brightness differences.

Further, a shutter speed changing table 78 in which the functional relationship between the brightness and the shutter speed is set individually corresponding to each of the first camera 51 and the second camera 52 is provided. Functional relationship set in the shutter speed change table 78, a shutter speed higher brightness becomes brighter is quick Kunar functional relationship. Further, in the shutter speed changing table 78, the difference between the average brightness A1 and A2 of the imaging reference parts obtained by the first camera 51 and the second camera 52 obtained by the brightness comparison processing unit 72 is reduced. A brightness change processing unit 73 that calculates the shutter speed based on the set function relationship is provided.

The shutter speed changing table 78 and the brightness changing processing unit 73 described above are the average brightness A1 and A2 of the imaging reference parts obtained by the first camera 51 and the second camera 52 obtained by the brightness comparison processing unit. An adjustment unit that adjusts the shutter speeds of the first camera 51 and the second camera 52 is configured to reduce the difference.

  FIG. 3 is a flowchart showing a processing procedure in the personal computer provided in the first embodiment.

In the first embodiment configured as described above, each process shown in FIG. 3 is performed when the brightness of the imaging reference portion is adjusted by the first camera 51 and the second camera 52. That is, the images of the first camera 51 and the second camera 52 that photographed the vicinity of the marker 6 shown in FIG. 1A are displayed on the screen of the monitor 8 as shown in FIG. The operator moves the mouse 9 and designates an area 63 including the marker image 62 as shown in FIG. 1C (step S100). Next, as shown in FIG. 1D, the brightness comparison processing unit 72 determines the average brightness A1 and A2 of the photographing reference parts by the photographing of the first camera 51 and the second camera 52, that is, the respective areas 63. Further, average brightness A1 and average brightness A2 are calculated (step S101).

Next, based on the functional relationship among the brightness A1 related to the first camera 51, the desired brightness A0, and the shutter speed set by the brightness change processing unit 73 in the shutter speed change table 78, the first change is made. A shutter speed corresponding to the average brightness A1 of the imaging reference portion corresponding to the camera 51 is obtained. The shutter speed of the second camera 52 is obtained from the average brightness A2 of the second camera 52, the desired brightness A0, and the functional relationship set in the shutter speed change table. For example, when the first camera 51 has the same brightness as the desired brightness and A0 = A1, the shutter speed of the first camera 51 does not change, and only the second camera 52 changes from the previous time (step S102).

  In this case, at least one of the next shutter speed of the first camera 51 and the next shutter speed of the second camera 52 obtained by the left and right cameras, that is, the brightness change processing unit described above, is the current corresponding shutter speed. When it is determined that they are different (step S103), the shutter speed related to the first camera 51 and the shutter speed related to the second camera 52 are changed to the first through the command processing unit 77 and the input / output unit 76 of the personal computer 7, respectively. It is output to the camera 51 and the second camera 52, and the shutter speed of the corresponding camera, in this case, the shutter speed of the second camera 52 is changed (step S104).

  While the first camera 51 and the second camera 52 continue to capture the vicinity of the marker 6, the processes in steps S101 to S104 in FIG. 3 are repeated.

  When the next values of the shutter speeds of the first camera 51 and the second camera 52 are considered to be the same as the current values in step S103 of FIG. 3, the brightness adjustment is finished. In such a state, the process shifts to the measurement processing of the photographing object by the three-dimensional shape measurement processing unit 74.

The first embodiment configured as described above is in the vicinity of the marker 6 provided on the arm 3 constituting the work tool in an environment where the brightness greatly fluctuates, for example, in an outdoor work environment where work is performed using a hydraulic excavator. As described above, the imaging reference part is photographed by each of the first camera 51 and the second camera 52 that are arranged on the left and right and constitute a stereo camera, so that the imaging reference part near the marker 6 is manually operated by the operator. 9 is detected. The brightness comparison processing unit 72 included in the personal computer 7 obtains the average brightness A1 and A2 of the imaging reference portions of the first camera 51 and the second camera 52 designated by the mouse 9, and further desirable brightness A0. Then, the difference between the average brightness A1 and A2 is obtained, and the adjustment unit including the brightness change processing unit 73 and the shutter speed change table 78 determines the average brightness A1 of the imaging reference region by the first camera 51 and the second camera. The shutter speeds of the first camera 51 and the second camera 52 are adjusted so that the difference in brightness of the average brightness A2 of the imaging reference region by 52 becomes a difference from the desired brightness A0. By such processing, the captured images of the first camera 51 and the second camera 52 at the time of measuring the three-dimensional shape in an environment where the brightness varies greatly are acquired as captured images in which the brightness of the imaging reference part is equivalent. be able to. Thereby, highly accurate three-dimensional shape measurement can be realized, and work accuracy and work efficiency of work performed by the hydraulic excavator can be improved.

  It should be noted that the angle of the arm 3 provided with the marker 6 is set so as to match the arrangement angle of the object to be photographed at the work site, and the first camera 51 and the second camera 2 are kept in a state in which the form of the arm 3 is maintained in this way. If the vicinity of the marker 6 is photographed by the camera 52, the brightness of the first camera 51 and the second camera 52 that are more suitable for the photographing object can be adjusted. For this reason, three-dimensional measurement with a stereo camera is possible even in an environment where the brightness of a subject to be photographed at a work site varies greatly depending on the arrangement angle.

  FIG. 4 is a block diagram showing a main configuration of the second embodiment of the present invention.

The second embodiment shown in FIG. 4 is also provided, for example, in the hydraulic excavator shown in FIG. In the second embodiment, each of the photographing as the photographing reference portion detecting unit for detecting a photographing reference position in the vicinity of the marker 6 from the image, the mouse 9 and brightness in the first embodiment of the first camera 51, second camera 52 In place of the comparison position determination processing unit 71, an angle for detecting a rotation angle of the boom 2 provided on a rotation angle detector attached to the joint part of the work tool, for example, the boom pin 21 shown in FIG. The sensor 22, the arm pin 31, the angle sensor 32 that detects the rotation angle of the arm 3, and the personal computer 7, and the imaging reference region near the marker 6 is calculated based on the detection signals of the angle sensors 22 and 32. imaging the reference region calculation unit for, that the marker position calculating processing unit 79, a process of determining the calculated photographing the reference part in the marker position calculating section 79 as the brightness comparison position And a Nau brightness comparison position determination processing unit 71. Other configurations are the same as those of the first embodiment described above.

  FIG. 5 is a flowchart showing a processing procedure in a personal computer provided in the second embodiment.

  In the second embodiment, as shown in FIG. 5, the positional relationship between the first camera 51 and the second camera 52 and the marker 6 provided on the arm 3 is calculated based on the detection values of the angle sensors 22 and 32 ( Procedure S200). In this case, when the marker 6 is in a range that is not reflected on the first camera 51 and the second camera 52, it is determined that there is an abnormality, and the process ends (step S201). When the marker 6 is projected by the first camera 51 and the second camera 52 and is determined to be normal, the pixel range (brightness comparison location) in which the marker 6 is reflected on the first camera 51 and the second camera 52 is a personal computer. 7 is calculated by the marker position calculation processing unit 79. The subsequent processing procedures are equivalent to the processing procedures S101 to S104 shown in FIG. 3 according to the first embodiment.

The second embodiment configured as described above has the same effect as that of the first embodiment, and also has an effect that the instruction of the marker by manual operation of the mouse or the like can be omitted. In particular, through the angle sensors 22 and 32 and the marker position calculation processing unit 79 of the personal computer 7, the first camera so that the difference in brightness between the imaging reference parts of the first camera 51 and the second camera 52 is reduced. The shutter speed of the 51 and the second camera 52 can be automatically adjusted. Therefore, it is suitable for three-dimensional shape measurement with respect to an object to be imaged located near the marker 6 in automatic operation of the hydraulic excavator.

  FIG. 6 is a diagram showing an outline of the operation of the third embodiment of the present invention. FIG. 6A is a side view showing a wheel loader given as an example of a work machine, and FIG. 6B is provided in the third embodiment. FIG. 7 is a diagram showing a monitor screen, FIG. 7 is a diagram showing an automatic excavation / loading operation performed by the wheel loader shown in FIG. 6, and FIG. 8 is a block diagram showing a main part configuration of a third embodiment of the present invention. .

  In the third embodiment, as shown in FIG. 6A, the work machine includes a wheel loader 10, and the work tool to which a marker (not shown) for brightness adjustment is attached includes a bucket 14. The captured images of the first camera 51 and the second camera 52 are displayed on the screen of the monitor 8 as shown in FIG. 6B shows an image 61 including a marker (not shown) photographed by the first camera 51, and the right diagram shows an image 61 including a marker (not shown) photographed by the second camera 52. Yes.

  The wheel loader 10 according to the third embodiment is automatically operated, for example. As shown in FIG. 7, the gravel mountain 101 is excavated with the bucket 14 and the gravel is loaded on the bucket 14. Gravel excavation / loading work is performed automatically to approach and release gravel from the bucket 14 to the loading platform of the dump truck 100 at a predetermined position.

The stereo image processing apparatus according to the third embodiment includes a first camera 51 and a second camera 52 equivalent to those of the second embodiment described above, and a personal computer 7 configuration, and as shown in FIG. The imaging reference part detection unit for detecting the imaging reference part of the rotation angle detector attached to the joint part of the work tool, for example, the angle sensor 43 for detecting the rotation angle of the arm holding the bucket 14, and the bucket 14 with respect to the arm. And an angle sensor 44 for detecting the rotation angle. Further, an automatic operation controller 80 that is connected to the personal computer 7 and controls the automatic operation of the wheel loader 10 is provided.

  The automatic operation controller 80 measures a work implement, that is, a measuring unit that measures the posture of the bucket 14 according to signals output from the angle sensors 43 and 44, and a plurality of shutter speeds, for example, when measuring the shape of the gravel mountain 101. A storage unit that stores the shutter speed 1 and the shutter speed 2 when measuring the shape of gravel loaded on the dump truck 100, and the shutter speeds 1 and 2 measured by the above-described measurement unit are read from the storage unit and the target And a processing unit that performs processing for setting the shutter speed.

  FIG. 9 is a flowchart showing a processing procedure in the personal computer and the automatic operation controller provided in the third embodiment.

In the third embodiment, as shown in FIG. 8, when the shape of the gravel mountain 101 is measured before excavating the gravel mountain 101 with the wheel loader 10, the shutter stored in the storage unit of the automatic operation controller 80. The speed 1 is read (step S300), and the gravel mountain shape three-dimensional measurement is performed by the three-dimensional shape measurement processing unit 74 of the personal computer 7 by the shutter speed 1 (step S301). At this time, it is determined whether it is necessary to adjust the brightness of the imaging reference parts of the first camera 51 and the second camera 52 (step S302). If it is necessary to adjust the brightness, the brightness adjustment processing is performed in advance using each processing unit of the personal computer 7 as described above (step S303). The brightness adjustment processing in the personal computer 7 at this time is equivalent to the processing described in relation to the second embodiment.

  When it is determined in step S302 that the brightness adjustment is unnecessary, a process of storing the currently used shutter speed 1 in the storage unit of the automatic operation controller 80 is performed (step S304). Subsequently, the excavation start position of the gravel mountain 101 is determined by the processing of the automatic operation controller 80 (step S305), the wheel loader 10 is automatically traveled to the excavation start position (step S306), and the gravel mountain by the bucket 14 is processed. 101 is automatically excavated (step S307).

  Subsequently, a process of creating a travel route to the dump truck 100 is performed (procedure S308), and a process of automatically traveling the wheel loader 10 to the loading start position with gravel loaded on the bucket 14 is performed (procedure S309). ), A process of loading gravel loaded in the bucket 14 onto the dump truck 100 is started (step S310).

At this time, in order to measure the shape of gravel loaded on the dump truck 7100, the shutter speed 2 stored in the storage unit of the automatic operation controller 80 is read (step S311). The three-dimensional shape measurement of gravel loaded with gravel is performed by the three-dimensional shape measurement processing unit 74 of the personal computer 7 (step S312). At this time, it is determined whether or not it is necessary to adjust the brightness of the imaging reference parts of the first camera 51 and the second camera 52 '(step S313). If brightness adjustment is necessary, brightness adjustment processing is performed in advance by the personal computer 7 (step S314). The processing in the personal computer 7 at this time is equivalent to the processing described in relation to the second embodiment. When it is determined in step S313 that the brightness adjustment is unnecessary, a process of storing the currently used shutter speed 2 in the storage unit of the automatic operation controller 80 is performed (step S315).

  Subsequently, a process for creating a travel route from the vicinity of the dump truck 100 to the gravel mountain 101 is performed by the automatic operation controller 80 (step S316), and the wheel loader 10 is again three-dimensionally measured for the gravel mountain shape near the gravel mountain 101. Processing for automatically traveling to the position is performed (procedure S317), and a determination unit in the automatic operation controller 80 determines whether or not the next loading is necessary (procedure S318). If it is determined that the next loading is necessary, the process returns to step S300. When it is determined that the next loading is unnecessary, the wheel loader 10 is in a standby state, for example.

  Here, there is the following method as an example of determining whether or not the brightness adjustment in steps S302 and S313 is necessary. As a measurement result of gravel as a measurement object, it can be determined that brightness adjustment is necessary when the current measurement is significantly smaller than the measurement area (number of measurement points) of gravel measured last time or initially.

  In addition, the brightness adjustment processing in steps S303 and S314 is performed by adjusting the angle of gravel (preferably the repose angle) that is the measurement object and the angle of the marker 61 as described in the first embodiment. By doing so, it is possible to adjust the brightness more suitable for shape measurement. Further, the target marker 61 angle is good even with the average angle of the object shape previously measured by the stereo camera.

The third embodiment configured as described above includes the personal computer 7 capable of adjusting the brightness of the imaging reference region by the first camera 51 and the second camera 52 having the same configuration as in the second embodiment. In addition to the operational effects equivalent to those of the second embodiment, in particular, when the wheel loader 10 is automatically operated through the automatic operation controller 80, the target shutter speed corresponding to the measurement target, that is, the shape of the gravel mountain 101 is obtained. The three-dimensional shape dimensions of a plurality of objects to be photographed are processed by the processing unit of the automatic operation controller 80 in which the shutter speed is 1 in the measurement and the gravel shape of the dump truck 100 is measured in the shutter speed 2. It can be realized with high accuracy in accordance with each automatic driving operation performed by the loader 10.

  In addition, by appropriately adjusting the brightness, it is possible to realize a stable measuring means without being affected by changes in the weather or the position of the sun. In the present embodiment, the automatic operation controller 80 reads the shutter speed for each measurement object from the storage unit, and stores the shutter speed after the three-dimensional measurement and the brightness adjustment. Instead of this, a system may be used in which the shutter speed is read and measured for each work posture such as a bucket or an arm. The working posture of the bucket, arm, etc. when measuring the gravel shape of the gravel mountain 101 before excavation and the dump truck 100 after loading is almost the same, and the optimum shutter speed can be read out.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure explaining the outline | summary of operation | movement in 1st Embodiment of the stereo image processing apparatus which concerns on this invention, (a) A principal part perspective view which shows the hydraulic shovel mentioned as an example of a working machine, (b) The figure is FIG. The figure which shows the screen of the monitor with which 1 embodiment is equipped, (c) The figure is the figure which shows the image where the area where the marker is displayed was cut out from the screen which is shown in (b) figure, (d) The figure is (c) FIG. 6 is a diagram showing average brightness in the area of the figure. It is a block diagram which shows the principal part structure of 1st Embodiment of the stereo image processing apparatus which concerns on this invention. It is a flowchart which shows the process sequence in the personal computer with which 1st Embodiment is equipped. It is a block diagram which shows the principal part structure of 2nd Embodiment of this invention. It is a flowchart which shows the process sequence in the personal computer with which 2nd Embodiment is equipped. It is a figure which shows the outline | summary of operation | movement of 3rd Embodiment of this invention, (a) A side view which shows the wheel loader mentioned as an example of a working machine, (b) A figure of the screen of the monitor with which 3rd Embodiment is equipped. FIG. It is a figure which shows the automatic excavation and loading work implemented with the wheel loader shown in FIG. It is a block diagram which shows the principal part structure of 3rd Embodiment of this invention. It is a flowchart which shows the process sequence in the personal computer and automatic operation controller with which 3rd Embodiment is equipped.

DESCRIPTION OF SYMBOLS 1 Revolving body 2 Boom 3 Arm 4 Bucket 6 Marker 7 Personal computer 8 Monitor 9 Mouse (designating means) [imaging reference part detection part]
10 Wheel loader (work machine)
14 bucket 21 boom pin 22 angle sensor (rotation angle detector) [imaging reference part detector]
31 arm pin 32 angle sensor (rotation angle detector) [imaging reference part detector]
41 bucket pin 43 angle sensor (rotation angle detector) [imaging reference part detection unit]
44 Angle sensor (rotation angle detector) [imaging reference part detector]
51 First camera 52 Second camera 61 Image 62 Marker image 63 Area 71 Brightness comparison position determination processing unit (imaging reference part detection unit)
72 Brightness Comparison Processing Unit 73 Brightness Change Processing Unit (Adjustment Unit)
74 Three-dimensional shape measurement processing unit 75 Image storage unit 76 Input / output unit 77 Command processing unit 78 Shutter speed change table (adjustment unit)
79 Marker position calculation processing unit (imaging reference part calculation part) [imaging reference part detection part]
80 Automatic operation controller (measurement unit) (storage unit) (processing unit)
100 dump truck 101 gravel mountain

Claims (4)

Stereo image processing of a work machine provided with a work machine having a joint and having a stereo camera composed of a plurality of cameras for measuring a three-dimensional shape of an object to be photographed positioned around the work tool In the device
A marker indicating a reference position for brightness adjustment provided on the work implement;
And imaging the reference region detector for detecting a photographing reference position in the vicinity of the marker from each of the captured image of the camera,
Each determined, and brightness comparison processing unit for calculating the difference in brightness between these imaging reference position for the imaging reference position detection unit and the plurality of cameras has been brightness of the photographed reference part detected,
A stereo image of a work machine, comprising: an adjustment unit that adjusts a shutter speed of each camera so that a difference in brightness between the imaging reference parts calculated by the brightness comparison processing unit is reduced. Processing equipment. The stereo image processing apparatus for a work machine according to claim 1,
A monitor that displays the image captured by the camera;
The imaging reference part detection unit includes a designation unit that designates an image range on the monitor screen by a cursor display, and a brightness comparison position that performs a process of determining the image range designated by the designation unit as a brightness comparison position A stereo image processing apparatus for a work machine, comprising: a decision processing unit. The stereo image processing apparatus for a work machine according to claim 1,
The imaging reference position detecting section includes a rotation angle detector that is provided to the joint portion of the working tool, the photographing reference part calculator for calculating the imaging reference position in the vicinity of the marker based on a detection signal of the rotation angle detector A stereo image processing apparatus for a working machine, comprising: a brightness comparison position determination processing unit that performs a process of determining the imaging reference region calculated by the imaging reference region calculation unit as a brightness comparison position. In the stereo image processing apparatus of the working machine of any one of Claims 1-3,
A measuring unit that measures the posture of the work tool, a storage unit that stores a shutter speed according to the posture of the work tool, and a shutter speed that corresponds to the posture of the work tool measured by the measuring unit. A stereo image processing apparatus for a working machine, comprising: a processing unit that performs processing for reading out from the camera and setting the target shutter speed of each camera.