JP2009060289A - Exposure controller for camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an exposure controller for a camera mounted on a mobile object that enables the camera to image an object with suitable brightness even when a bright light source such as the sun is reflected within the camera's angle of view. <P>SOLUTION: The camera is mounted on a robot, generates a brightness histogram of an image picked up by at least two image pickup devices (CCD cameras) which pick up an image utilizing incident light from external world in which an object such as human being is present, sets exposure parameters including a shutter speed based on the generated brightness histogram (S10-S22), when the set exposure parameters are found to be out of the predetermined range, when a high-brightness imaging region picked up by high-brightness incident light is present in the image, judges whether it is necessary to remove the high-brightness imaging region (S24, S26), when it is determined that removal of the high-brightness imaging region is necessary, extracts the high-brightness imaging region and removes it from the image (S28, S30, S32). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a camera exposure control device, and more specifically to a camera exposure control device mounted on a moving body.

Conventionally, for example, in Patent Document 1 below, when a high-intensity light source such as sunlight is reflected within the angle of view of a camera, the position, shape, and the like of a ghost generated by internal reflection on the image sensor surface are predicted, There has been proposed a technique for determining which part of a photographed image is a ghost, and correcting (reducing processing) the determined ghost part by designation of a photographer or automatically.
JP 2006-129084 A

  The technique described in Patent Document 1 is a so-called digital camera or video camera, which is a technique for a camera that is photographed by a user operating a shooting target such as a person. It may be used as a visual sensor.

  In such a case, since it is unclear whether or not a subject such as a person who wants to shoot is included in the image, exposure parameters are adjusted based on distance information. However, as shown in FIG. When a high-intensity light source such as a light is captured within the angle of view of the camera, if you increase the shutter speed or close the aperture to avoid saturation of the brightness value, the brightness value of the subject will be crushed and shot An image suitable for subsequent processing such as target extraction may not be obtained. In FIG. 15, the person is actually present on the right side of the screen.

  Accordingly, an object of the present invention is to solve the above-described problems, and is a camera mounted on a moving body, and when a high-luminance light source such as sunlight is reflected within the angle of view of the camera, An object of the present invention is to provide an exposure control device for a camera that takes pictures with values.

  In order to solve the above-described problem, in the camera exposure control apparatus according to claim 1, at least two image sensors that are mounted on a moving body and capture an image with incident light from the outside including the subject to be imaged. An exposure parameter setting means for generating a brightness value histogram of an image captured by the image sensor and setting an exposure parameter including a shutter speed based on the generated brightness value histogram, and the set exposure parameter It is determined whether or not the image is within a predetermined range. When it is determined that the image is not within the predetermined range, if the image has a high-intensity imaging region imaged by incident light with high luminance, the high-intensity imaging region is removed. A high-brightness imaging part removal necessity determination unit that determines whether or not it is necessary, and the high-brightness imaging when it is determined that the high-brightness imaging part needs to be removed Was composed as and a high-brightness imaging region extraction and removal means for removing from said image by extracting position.

  In the above, “high luminance incident light” means incident light having a relatively high luminance such as a light source, and “high luminance imaging portion” means a portion that is imaged by high luminance incident light.

  In the camera exposure control apparatus according to claim 2, the exposure parameter setting means calculates a luminance value of the photographing target in the image and a maximum luminance value of the entire image, and the calculated luminance is calculated. A luminance value histogram is generated for the values, and an exposure parameter including a shutter speed is set based on the generated luminance value histogram.

  In the camera exposure control apparatus according to claim 3, the photographing object is a person, and the exposure parameter setting means is configured to calculate a luminance value of a human face and a maximum luminance value of the entire image. did.

  In the exposure control apparatus for a camera according to claim 4, the high-luminance imaging region extraction / removal unit calculates an average value of luminance values for each predetermined block of the image, and the average value of luminance values is predetermined. A block group equal to or greater than the value is extracted, and the center position of the extracted block group is calculated to extract the high-luminance imaging region.

  In the camera exposure control apparatus according to claim 5, the high-intensity imaging region extraction / removal unit calculates an average value of luminance values for each predetermined block of the image, and the calculated average value is the largest. A large block is extracted, and the high luminance imaging region is extracted based on the extracted block.

  In the camera exposure control apparatus according to claim 6, the high-intensity imaging part extraction / removal unit specifies a position and an angle of the high-intensity imaging part, and the specified high-intensity imaging part is The moving body is configured to generate a predetermined action so as to be removed from the image.

  In the camera exposure control apparatus according to claim 1, a luminance value histogram of an image captured by at least two imaging elements mounted on a moving body and capturing an image with incident light from the outside including the subject to be imaged. The exposure parameter including the shutter speed is set based on the image, and it is determined whether or not the set exposure parameter is within the predetermined range. When it is determined that the exposure parameter is not within the predetermined range, the image is picked up with high-intensity incident light. If there is a high-brightness imaging part, it is determined whether it is necessary to remove it, and when it is determined that it is necessary to remove it, the high-brightness imaging part is extracted and removed from the image Therefore, even when a high-intensity light source such as sunlight is reflected within the angle of view of the camera, it is possible to photograph the photographing object with an appropriate luminance value.

  In the camera exposure control apparatus according to claim 2, the luminance value of the photographing target in the image and the maximum luminance value of the entire image are calculated, a luminance value histogram is generated for the calculated luminance value, and based thereon In addition to the effects described above, the exposure parameters must be set accurately even when a high-intensity light source such as sunlight is reflected within the angle of view of the camera. Therefore, it is possible to shoot an object to be shot with an appropriate luminance value.

  In the exposure control apparatus for a camera according to claim 3, since the photographing object is a person, the exposure parameter setting means is configured to calculate the luminance value of the human face and the maximum luminance value of the entire image. In addition to the effects described above, the exposure parameters can be accurately adjusted when not only high brightness such as sunlight but also low-brightness imaging parts such as human faces are reflected within the angle of view of the camera. Therefore, the object to be imaged can be imaged with an appropriate luminance value.

  In the camera exposure control apparatus according to claim 4, the average value of the luminance values for each predetermined block of the image is calculated, and a block group having the predetermined value or more is extracted, and the center of the extracted block group Since the high luminance imaging part is extracted by calculating the position, the high luminance imaging part can be reliably extracted and removed from the image in addition to the above-described effects.

  In the exposure control apparatus for a camera according to claim 5, an average value of luminance values for each predetermined block of the image is calculated, a block having the largest calculated average value is extracted, and based on the extracted block Since the configuration is such that the high-intensity imaging region is extracted, in addition to the above-described effect, similarly, the high-intensity imaging region can be reliably extracted and removed from the image.

  In the camera exposure control device according to claim 6, the position and angle of the high-intensity imaging part are specified, and a predetermined action is generated in the moving body so as to remove the specified high-intensity imaging part from the image. Since it is configured as described above, in addition to the above-described effects, the high-luminance imaging region can be reliably removed from the image.

  The best mode for carrying out an exposure control apparatus for a camera according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a front view of a legged mobile robot as a moving body on which the camera exposure control apparatus according to this embodiment is mounted, and FIG. 2 is a side view of the robot shown in FIG.

  As shown in FIG. 1, a legged mobile robot (moving body; hereinafter simply referred to as “robot”) 10 includes two left and right leg portions 12L and 12R (L on the left side and R on the right side; the same applies hereinafter). . The leg portions 12L and 12R are connected to the lower portion of the base 14. A head portion 16 is connected to the upper portion of the base body 14, and two left and right arm portions 20L and 20R are connected to the side. Hands (end effectors) 22L and 22R are connected to the tips of the left and right arm portions 20L and 20R, respectively. In this embodiment, a humanoid robot having a height of about 1.3 m and having two legs and two arms is taken as an example of a legged mobile robot.

  As shown in FIG. 2, a storage portion 24 is provided on the back of the base 14, and an electronic control unit (hereinafter referred to as “ECU”) 26, a battery (not shown), and the like are accommodated therein.

  FIG. 3 is an explanatory diagram showing the robot 10 shown in FIG. 1 as a skeleton. Hereinafter, the internal structure of the robot 10 will be described with a focus on joints with reference to FIG. Since the illustrated robot 10 is bilaterally symmetric, L and R are not described below.

  The left and right legs 12 include a thigh link 30, a crus link 32, and a foot 34, respectively. The thigh link 30 is connected to the base body 14 via a hip joint. In FIG. 3, the base body 14 is simply shown as the base link 36, but the base link 36 is configured such that the upper half 36 a and the lower half 36 b can be relatively displaced via a joint 38.

  The thigh link 30 and the lower leg link 32 are connected via a knee joint, and the lower leg link 32 and the foot 34 are connected via an ankle joint. The hip joint includes a rotation axis 40 around the Z axis (yaw axis), a rotation axis 42 around the Y axis (pitch axis), and a rotation axis 44 around the X axis (roll axis). That is, the hip joint has three degrees of freedom.

  The knee joint is composed of a rotation axis 46 around the Y axis and has one degree of freedom. The ankle joint includes a rotation axis 48 around the Y axis and a rotation axis 50 around the X axis, and has two degrees of freedom. In this way, each of the left and right leg parts 12 is provided with six rotation axes (degrees of freedom) constituting three joints, and the leg part as a whole is provided with a total of twelve rotation axes.

  The leg 12 is driven by an actuator (not shown). The leg actuator that drives the leg 12 includes twelve electric motors arranged at appropriate positions of the base 14 and the leg 12, and individually drives the twelve rotating shafts described above.

  The left and right arm portions 20 include an upper arm link 52 and a lower arm link 54, respectively. The upper arm link 52 is connected to the base body 14 through a shoulder joint. The upper arm link 52 and the lower arm link 54 are connected via an elbow joint, and the lower arm link 54 and the hand 22 are connected via a wrist joint.

  The shoulder joint includes a rotation axis 56 around the Y axis, a rotation axis 58 around the X axis, and a rotation axis 60 around the Z axis, and has three degrees of freedom. The elbow joint is composed of a rotation shaft 62 around the Y axis and has one degree of freedom. The wrist joint includes a rotation axis 64 around the Z axis, a rotation axis 66 around the Y axis, and a rotation axis 68 around the X axis, and has three degrees of freedom. In this manner, each of the left and right arm portions 20 is provided with seven rotation axes (degrees of freedom) constituting three joints, and a total of 14 rotation axes are provided as the entire arm portion.

  The arm portion 20 is also driven by an actuator (not shown) as with the leg portion 12. The arm actuator for driving the arm 20 is composed of 14 electric motors arranged at appropriate positions of the base 14 and the arm 20 and individually drives the 14 rotation shafts described above. In the robot 10, the movement of the leg actuator or the arm actuator is controlled so that each rotation axis is driven at an appropriate angle, whereby a desired movement is given to the leg 12 or the arm 20.

  The hand 22 is provided with five finger portions 70. The finger unit 70 can be operated by a hand actuator (not shown), and can perform operations such as gripping an object in conjunction with the movement of the arm unit 20 or pointing in an appropriate direction.

  The head 16 is connected to the base 14 via a neck joint. The neck joint includes a rotation shaft 72 around the Z axis and a rotation shaft 74 around the Y axis, and has two degrees of freedom. The rotary shafts 72 and 74 are also individually driven by a head actuator (not shown). By controlling the operation of the head actuator and driving the rotary shafts 72 and 74 at an appropriate angle, the head 16 can be directed in a desired direction. The base link 36 also drives an actuator (not shown) disposed at the joint 38, whereby the upper half 36a and the lower half 36b are relatively rotated.

  A force sensor (6-axis force sensor) 76 is attached to each of the left and right legs 12, and the three-direction components Fx, Fy, Fz of the floor reaction force acting on the leg 12 from the floor and the three-direction components Mx of the moment. , My, and Mz are output. The same type of force sensor 78 is also attached to the left and right arm portions 20 between the hand 22 and the wrist joint, and the three-direction components Fx, Fy, Fz of the external force acting on the arm portion 20 and the three-direction components Mx, My, A signal indicating Mz is output.

  A tilt sensor 80 is installed on the base 14 and outputs a signal indicating a state quantity such as the tilt angle of the base 14 with respect to the vertical axis and its angular velocity. The head 16 is provided with two (left and right) CCD cameras (hereinafter referred to as “cameras”) 82 that capture images by incident light from the outside including a subject such as a person. The head 16 is provided with a voice input / output device 84 including a microphone 84a and a speaker 84b.

  The output of the above-described sensor or the like is input to the ECU 26 (shown in FIG. 2). The ECU 26 includes a microcomputer and includes a CPU, an input / output circuit, a ROM, a RAM, and the like (not shown).

  FIG. 4 is a block diagram showing the configuration of the robot 10 with the input / output relationship of the ECU 26 as the center.

  As shown in the figure, the robot 10 is carried by a rotary encoder group 86, a gyro sensor 88, a GPS receiver 90, and a person (photographing target) arranged on each of the rotating shafts 40 in addition to the above-described sensors. An IC tag signal receiver (reader) 94 that is connected to the IC tag 92 wirelessly and receives IC tag information transmitted from the IC tag 92 is provided.

  Each of the rotary encoder groups 86 outputs a signal corresponding to the rotation angle of the rotary shaft 40 or the like, that is, the joint angle. The gyro sensor 88 outputs a signal corresponding to the moving direction and distance of the robot 10. The GPS receiver 90 receives a radio wave transmitted from a satellite, acquires position information (latitude and longitude) of the robot 10 and outputs it to the ECU 26. The IC tag signal receiver 94 is stored in the IC tag 92 and transmitted from the identification information (RFID (Radio Frequency ID), specifically, identification information for identifying the person who carries the IC tag 92). Is received by the wireless system and output to the ECU 26.

  The ECU 26 performs gait control by generating a gait based on the outputs of the force sensor 76, the tilt sensor 80, and the rotary encoder group 86. Specifically, the operation of the above-described leg actuator (indicated by reference numeral 100) is controlled to drive the leg 12 to move (walk) the robot 10. Since gait generation and walking control are described in Japanese Patent No. 3726081 proposed by the present applicant, detailed description thereof is omitted.

  The ECU 26 controls the operation of the arm actuator (indicated by reference numeral 102) and the hand actuator (indicated by reference numeral 104) in association with walking control and the like to drive the arm section 20 and the hand 22, and the head actuator. The direction of the head 16 is adjusted by controlling the operation (indicated by reference numeral 106).

  Further, the ECU 26 operates as an exposure control device for the camera 82, that is, executes exposure control for the camera 82.

  FIG. 5 is a block diagram functionally showing a configuration when the ECU 26 operates as an exposure control device of the camera 82.

  As shown in the figure, when the operation of the ECU 26 at that time is viewed by function, the ECU 26 includes a stereo processing unit 26a, a histogram generation unit 26b, an exposure parameter setting unit 26c, a high-brightness imaging region removal necessity determination unit 26d, It consists of a high-intensity imaging region extraction unit 26e, an image processing unit 26f, and an action generation unit 26g.

  The stereo processing unit 26a is mounted on the moving body (robot) 10 and outputs outputs of at least two cameras (imaging devices) 82 that capture an image with incident light from an imaging target, specifically, an external environment including a person. Then, distance information for each pixel is calculated from the parallax of the image input by stereo processing. The number of pixels of the camera 82 is 320 × 240. The stereo processing unit 26a calculates and outputs three-dimensional (3D) data from the grayscale image data.

  The histogram generation unit 26b creates a luminance value histogram of the photographed image and performs weighting for each distance or according to the distance.

  The exposure parameter setting unit 26c sets the exposure parameter (more specifically, the shutter speed) from the luminance value at the distance to be imaged. Since the camera 82 is mounted on the robot 10 and functions as a visual sensor, the camera 82 does not search and shoot the shooting target itself, but must extract the shooting target from the shot image. Therefore, the diaphragm of the camera 82 is fixed to the minimum, and is adjusted so as to be focused at a short distance, specifically, about 0.5 m to 2.5 m. Therefore, only the shutter speed can be adjusted as an exposure parameter.

  The high-brightness imaging part removal necessity determination unit 26d removes from the image a high-brightness imaging part imaged by high-luminance incident light such as a light source in the image in order to obtain an appropriate exposure for photographing the subject. Determine if you need to do that.

  When it is determined that the high-brightness imaging part needs to be removed from the image, the high-brightness imaging part extraction unit 26e extracts the high-brightness imaging part and determines the stage coordinate system based on the camera parameters of the camera 82, that is, The position and angle of the high-luminance imaging region in the orthogonal coordinate system with one point on the screen as the origin and the floor surface as the x and y planes are calculated. Extraction of the high-intensity imaging region is performed by determining an area where the average luminance value is greater than the threshold as an area of the high-intensity imaging region, or calculating an average value of luminance values for each predetermined block of the image, The largest block is determined as the center of the high brightness imaging region.

  The image processing unit 26f performs image processing as a visual sensor when the robot 10 moves. The behavior generation unit 26g causes the robot 10 to generate a predetermined behavior so as to remove the high luminance imaging region from the image based on the size of the high luminance imaging region.

  Next, the operation of the ECU 26 will be described in more detail.

  FIG. 6 is a flowchart showing processing when the ECU 26 operates as an exposure control device of the camera 82, as in FIG. The illustrated program is executed at a predetermined time, for example, every arbitrary time between 1 msec and 300 msec.

  In the following, in S10, it is determined whether or not there is a subject to be imaged, that is, a person. This is determined from the output of the IC tag signal receiver 94 that receives the identification information transmitted from the IC tag 92 carried by the subject.

  When the result in S10 is affirmative, the process proceeds to S12, where the brightness value of the image to be photographed in the screen, that is, the image obtained by photographing the object to be photographed by the camera 82 is calculated. The maximum luminance value is calculated. If the result in S10 is negative, the process in S12 is skipped.

  Next, the process proceeds to S16, and the calculated brightness values are compared with each other. If the maximum brightness value is larger, the process proceeds to S18, where the maximum brightness value is determined as a high brightness imaging region, and the brightness value of the imaging target is greater. When the process proceeds to S20, it is determined to be a high-luminance imaging region. The processing from S10 to S20 is executed by the stereo processing unit 26a of FIG. Next, in S22, exposure parameter setting processing is executed.

  FIG. 7 is a sub-routine flow chart showing the processing, which is executed by the histogram generation unit 26b and the exposure parameter setting unit 26c.

  In the following, the target pixel is extracted in S100. Here, “target pixel” means a pixel having distance information for which exposure is desired to be adjusted. This extraction of the target pixel corresponds to generating a luminance value histogram of an image obtained by shooting the shooting target.

  Next, in S102, it is determined whether or not the number of extracted pixels exceeds a threshold A that is set as appropriate. The threshold A is a value set in advance by calculating the area that should occupy the entire screen to be imaged at a distance where exposure is desired to be adjusted.

When the result is affirmative in S102, the process proceeds to S104, and the average value of the luminance values is calculated according to the following equation 1.
Average value of luminance value = average value of previous frame × (1−number of extracted pixels × coefficient / number of area pixels)
+ Current frame average (considering distance) x number of extracted pixels / number of area pixels
... Formula 1

Next, in S106, the value of the counter (up counter) is reset. On the other hand, when the result in S102 is negative, the process proceeds to S108, where it is determined whether or not the counter value is less than a threshold value B that is set as appropriate. 2 is calculated, and the process proceeds to S112 to update (up) the counter value.
Average luminance value = previous frame average value × (1−coefficient) + current frame average value × coefficient
... Formula 2

When the result in S108 is negative, the process proceeds to S114, the average value of the luminance values is calculated according to the following expression 3, and then the process proceeds to S112.
Average value of luminance value = Average value of current frame (3)

  In the above formula 1, etc., the “previous frame average value” is a value obtained by averaging the luminance values of all the pixels of the past several frames (that is, the entire screen) of the image (frame) taken every 200 msec. , “The number of extracted pixels” is the number of pixels extracted in S100, “the number of area pixels” is the number of all (that is, the entire screen) pixels including the extracted pixels, “coefficient” is a correction coefficient obtained through experience, The “current frame average value” is a value obtained by averaging the luminance values of all the pixels photographed this time (that is, the entire screen). As described above, since all the pixels have distance information, the luminance value of the pixels is averaged in consideration of the distance.

  Next, the process proceeds to S116, where it is determined whether or not the average value of the luminance values calculated in S104 or the like is less than a threshold C that is set as appropriate or exceeds a threshold D. The thresholds C and D are set by empirically selecting values that allow image processing.

When the result in S116 is affirmative, the process proceeds to S118, and the shift amount (change amount) of the shutter speed, which is one of the exposure parameters, is calculated according to the following expression 4 (in other words, the exposure parameter is set).
Shift amount = (average luminance value−threshold value C, D / 2) × current shutter speed × coefficient 1
... Formula 4

  That is, in S118, the shutter speed shift amount is calculated so that the average value of the luminance values is in the middle of the threshold value C and the threshold value D. The coefficient 1 is a correction coefficient set from experience. As described above, the aperture of the camera 82 is fixed to the minimum, and only the shutter speed can be adjusted as an exposure parameter.

On the other hand, when the result in S116 is negative, the process proceeds to S120, the reference value is calculated according to the following expression 5, and the process proceeds to S122, where the shutter speed shift amount (change amount) is calculated using the calculated reference value and the like. 6 is calculated (in other words, an exposure parameter is set). In equations 5 and 6, coefficients 2 and 3 are also correction coefficients set from experience.
Reference value = average value of luminance value− (number of pixels exceeding the effective luminance threshold−number of pixels less than the effective luminance threshold) / coefficient 2.
Shift amount = (average luminance value−reference value) × current shutter speed × coefficient 3
... Formula 6

  The processing of FIG. 7 will be described. In this embodiment, for example, as shown in FIG. 8, an appropriate luminance value is set for a subject to be photographed in a situation in which external light such as sunlight enters from the lighting window and is reflected on the screen. Shooting with

  In the shooting situation shown in FIG. 8, when a normal luminance value histogram as shown in FIG. 9 is calculated, the white squashed (saturated) pixels greatly exceed the black clogged (insufficient sensitivity) pixels. If the exposure is adjusted based on this, the subject to be photographed will be clogged with black. Further, in FIG. 8, since the object to be photographed is located slightly to the left from the vicinity of the center of the screen, the same result is obtained even if the exposure is adjusted based on the luminance information near the center of the screen.

  Therefore, a parallax image as shown in FIG. 10 is obtained, and a luminance value histogram weighted by the distance as shown in FIG. 11 is calculated. In the histogram shown in FIG. 11, when the value of a pixel close to the camera 82 is regarded as important, the frequency of white crushing and black clogging is approximately the same, and exposure adjustment is optimal. FIG. 8 shows a case where exposure is adjusted based on a luminance value histogram weighted by such distance.

  Further, in the processing of FIG. 7, as shown in FIG. 8, there is a situation in which the object to be imaged once exists in the screen and then moves with time to disappear from the screen. Therefore, if the object to be photographed exists in the screen in the affirmative in S102, or the counter value is determined to be less than the threshold value B (for example, several frames) in S108 even if the object is denied in S102, the average of the luminance values As a value, a weighted average of the average luminance value of the current frame and that of the previous frame is calculated (S104 or S110). When the result in S108 is negative and the threshold value is exceeded, the image data is separated from the past shooting data, and the average value of the pixels of the entire screen is set as the average value of the luminance values (S114).

  Further, the processing of FIG. 7 includes a case where the existence of the imaging target is denied and the maximum luminance value of the screen is determined as the high luminance imaging region. Even in such a case, when the result in S108 is negative and the counter value reaches the threshold value B, the process proceeds to S114, and the average value of the pixels on the entire screen is set as the average value of the luminance values (S114).

  The shift amount of the shutter speed will be described. When the average value of the luminance values is between the threshold value C and the threshold value D, the shift amount is determined according to Equation 4 so that the average value falls in the center of the two threshold values. Calculated (S118).

  On the other hand, if not, the reference value is first calculated according to Equation 5. When Expression 5 is described with reference to FIG. 11, “the number of pixels exceeding the effective luminance threshold” is, for example, the number of pixels exceeding the luminance value 25, and “the number of pixels less than the effective luminance threshold” is, for example, the luminance value. The number of pixels is less than 225. That is, for example, a difference obtained by subtracting the average value of pixels exceeding the luminance value 25 and less than 225 from the luminance average value is used as a reference value, and the reference value is used instead of the threshold values C and D. The amount of change in the shut speed is calculated by dividing from the average value.

  Returning to the description of the flowchart of FIG. 6, the process then proceeds to S24, where the shutter speed shift amount (in other words, the set exposure parameter) calculated in S118 or S122 of the flowchart of FIG. 7 is within a predetermined range. If the result is affirmative and the result is affirmative, it is possible to shoot with an appropriate luminance value by changing the shutter speed according to a routine not shown in accordance with the calculated shift amount, so that the subsequent processing is skipped.

  On the other hand, when the result in S24 is negative, the program proceeds to S26, in which high luminance imaging part removal necessity determination processing is executed.

  FIG. 12 is a sub-routine flowchart showing the processing, which is executed by the high-brightness imaging region removal necessity determination unit 26d.

  To explain below, in S200, it is determined whether or not there is position information of an imaging target. This is judged from the information obtained in S12 of the flowchart of FIG. 6 or the processing of the flowchart of FIG.

  When the result in S200 is affirmative, the process proceeds to S202, and the average value of the luminance values at the positions to be photographed is calculated. When the result is negative, the process proceeds to S204, and pixels within a relatively short distance, for example, several meters within the screen. The average value of the luminance values is calculated. The processing from S200 to S204 is similar to the processing from S10 to S14 in the flowchart of FIG.

  Next, in S206, it is determined whether or not the calculated average value of luminance values is less than the threshold value E. Since the determination in S206 is for determining whether the photographing situation is backlight or direct light, a value sufficient to determine the threshold value E is appropriately set.

  When the result in S206 is affirmative, it is determined that the subject is backlit and the subject to be photographed is black and the luminance value is low, and therefore it is determined that an appropriate exposure for photographing the subject to be photographed cannot be obtained. It is determined that it is necessary to remove the luminance imaging region. On the other hand, when the result in S206 is negative, since such inconvenience is not seen, the process proceeds to S210, and it is determined that it is not necessary to remove the high-luminance imaging region.

  Returning to the description of the flow chart of FIG. 6, the process then proceeds to S28, where it is determined whether it is determined that it is not necessary to remove the high-intensity imaging region. If the determination is affirmative, the subsequent processing is skipped and denied. If YES in step S30, the flow advances to step S30 to execute high-intensity imaging part extraction processing.

  FIG. 13 is a sub-routine flowchart showing the processing, which is executed by the high-intensity imaging region extraction unit 26e.

  In the following, in S300, an average value of luminance values for each block of the screen is calculated. Here, “block” means, for example, a 10 × 10 pixel. Since the camera 82 has 320 × 240 pixels, the average value of the luminance values is calculated for each unit area.

  Next, the process proceeds to S302, where a group of blocks with high luminance, more specifically, a high-luminance block that exceeds a threshold value that is set as appropriate, is extracted, and the process proceeds to S304, where the variance value is calculated. Calculate the spread of the high part.

  Next, in S306, it is determined whether or not the calculated variance value exceeds a threshold value F that is set as appropriate. When the result in S306 is affirmative, it is determined that the bright area is wide as shown on the right side of FIG. 8, and the process proceeds to S308, where the center position of the high-luminance block group is determined based on the camera parameters input from the camera 82. Is calculated.

  Next, in S310, the central angle of the high-luminance imaging part, that is, the angle with respect to the calculated center position is calculated. In S312, it is determined that the size of the high-luminance imaging part is large, and the determination result and S310 are calculated. The center angle of the high brightness imaging region is output. As described above, when the average value of the luminance values is larger than the threshold value F, the region is determined as the region of the high luminance imaging region, and the position and angle of the high luminance imaging region in the stage coordinate system are calculated.

  On the other hand, when the result in S306 is negative, the process proceeds to S314, where the block having the maximum average value is extracted from the high-luminance block group, and the process proceeds to S316, where the central angle of the high-luminance imaging region, that is, the extracted average value is extracted. Is calculated based on the camera parameters, and the process proceeds to S318, where it is determined that the size of the high-intensity imaging region is small, and the determination result and the center angle of the high-intensity imaging region calculated in S316 are output. To do.

  That is, when the average value of the luminance values is not larger than the threshold value, the block having the largest average value of the luminance values for each predetermined block of the image is determined as the center of the high-intensity imaging region, and high-intensity imaging in the stage coordinate system The position and angle of the part are calculated.

  Returning to the description of the flowchart of FIG. 6, the process then proceeds to S32 to execute an action generation process.

  FIG. 14 is a sub-routine flow chart showing the processing, which is executed by the behavior generating unit 26g.

  To explain below, in S400, it is determined whether or not the size of the high-luminance imaging region output in S312 or S318 of the flowchart of FIG. 13 exceeds 1 / 4S. Since “S” indicates the area of the screen, the determination in S400 means that it is determined whether the size of the high-luminance portion exceeds 1/4 of the screen.

  When the result in S400 is affirmative, the process proceeds to S402, where the size of the high-intensity imaging region is relatively large, so the leg portion 12 of the robot 10 is driven to change the direction of the base 14 (and the head 16) to increase the luminance. While the operation (generation of action) is performed so as to avoid the incident light (no incident light enters), the process proceeds to S404 when the result is negative, and the size of the high-intensity imaging region is relatively small. The unit 20R or 20L is driven to extend as much as possible, and the hand 22R or 22L operates (generates an action) so as to block incident light with high luminance, thereby removing the high luminance imaging region from the image.

  As described above, this embodiment is mounted on the moving body (robot 10) and takes an image with incident light from the outside including the subject to be photographed (person), more specifically, takes at least two images based on the stereo principle. An exposure parameter setting means for generating a brightness value histogram of an image picked up by the image pickup device (CCD camera 82) and the image pickup device and setting an exposure parameter including a shutter speed based on the generated brightness value histogram (Stereo processing unit 26a, histogram generation unit 26b, exposure parameter setting unit 26c, S10 to S22, S100 to S122) and determines whether the set exposure parameter is within a predetermined range, and is not within the predetermined range When it is determined that there is a high-brightness imaging portion imaged by high-luminance incident light in the image, The high-brightness imaging part removal necessity determination means (high-brightness imaging part removal necessity determination unit 26d, S24, S26, S200 to S210) for determining whether or not it is necessary to remove the high-brightness imaging part, and the high luminance When it is determined that the imaging region needs to be removed, a high-intensity imaging region extraction / removal unit (a high-intensity imaging region extraction unit 26e, an action generation unit 26f, S28, S30, S32, S300 to S318, and S400 to S404). Therefore, even when a high-intensity light source such as sunlight is reflected in the angle of view of the camera 82, the object to be imaged has an appropriate luminance. You can shoot with values.

  Further, the exposure parameter setting means calculates a luminance value of the photographing target in the image and a maximum luminance value of the entire image, generates a luminance value histogram for the calculated luminance value, and generates the generated luminance value histogram. Since the exposure parameter including the shutter speed is set based on the brightness value histogram, more specifically based on the magnitude of the generated brightness value histogram (S10 to S22, S100 to S122), the above-described effect is obtained. In addition, when a high-luminance light source such as sunlight is reflected within the angle of view of the camera 82, the exposure parameter can be set accurately, so that the subject can be photographed with an appropriate luminance value. .

  In addition, since the object to be photographed is a person and the exposure parameter setting means is configured to calculate the luminance value of the human face and the maximum luminance value of the entire image (S10 to S22, S100 to S122), When not only high brightness such as light but also a low brightness imaging part such as a human face is reflected in the angle of view of the camera, the exposure parameter can be set accurately, so A subject to be photographed can be photographed with an appropriate luminance value.

  The high-brightness imaging region extraction / removal means calculates an average value of luminance values for each predetermined block of the image, extracts a block having an average luminance value equal to or greater than a predetermined value, and extracts the extracted block Since the high-brightness imaging region is extracted by calculating the center position of the light source (S300 to S312), in addition to the above-described effects, the high-brightness imaging region reliably imaged by the high-brightness incident light such as the light source Can be extracted and removed from the image.

  Further, the high-brightness imaging part extraction / removal means calculates an average value of luminance values for each predetermined block of the image, extracts a block having the largest calculated average value, and based on the extracted block Therefore, in addition to the above-described effects, the high-intensity imaging part can be reliably extracted and removed from the image.

  The high-brightness imaging part extraction / removal means specifies a position and an angle of the high-brightness imaging part and performs a predetermined action on the moving body so as to remove the specified high-brightness imaging part from the image. Is generated (S400 to S404), in addition to the effects described above, the high-luminance imaging region can be reliably removed from the image.

  In the above description, a person is scheduled as an object to be photographed, but other objects such as tools and works scheduled for the work of the robot 10 may be used.

  In the above description, the presence / absence of the photographing target is determined from the output of the IC tag signal receiver 94 that receives the identification information transmitted from the IC tag 92 carried by the photographing target. Also good. Further, it may be taught by inputting a command from the outside.

  In the above description, a legged mobile robot, specifically a bipedal walking robot, is illustrated as an example of the moving body. However, the mobile body is not limited to this, and any mobile body may be used as long as it can move autonomously.

1 is a front view of a legged mobile robot as a moving body on which a camera exposure control device according to an embodiment of the present invention is mounted. It is a side view of the robot shown in FIG. It is explanatory drawing which shows the robot shown in FIG. 1 with a skeleton. FIG. 2 is a block diagram showing the configuration of the robot shown in FIG. 1 focusing on the input / output relationship of an electronic control unit (ECU). It is a block diagram which shows a structure functionally when the electronic control unit shown in FIG. 4 operate | moves as an exposure control apparatus of a camera. FIG. 6 is a flowchart showing processing when the electronic control unit operates as an exposure control device for a camera, as in FIG. 5. 7 is a sub-routine flowchart showing the exposure parameter setting process of FIG. 6. It is an example of the screen which shows the process of FIG. FIG. 9 is a normal luminance value histogram (not considering distance) of the screen of FIG. 8. FIG. It is a parallax image of the screen of FIG. FIG. 9 is a luminance value histogram in consideration of the screen distance of FIG. 8. FIG. FIG. 7 is a sub-routine flow chart showing a process for determining whether or not to remove a high-intensity imaging region in FIG. 6. FIG. 7 is a sub-routine flowchart showing the high-intensity imaging region extraction process of FIG. 6. FIG. FIG. 7 is a sub-routine flow chart showing the action generation process of FIG. 6. FIG. It is an example of the scene which plans imaging | photography with the exposure control apparatus of the camera which concerns on this Example.

Explanation of symbols

  10: Legged mobile robot (moving body, robot), 12: Leg, 14: Base, 20: Arm, 26: ECU (Electronic Control Unit), 26a: Stereo processor, 26b: Histogram generator, 26c: Exposure parameter setting unit, 26d: High-brightness imaging region removal necessity determination unit, 26e: High-brightness imaging region extraction unit, 26f: Image processing unit, 26g: Action generation unit, 82: CCD camera (imaging device, camera), 92 : IC tag, 94: IC tag signal receiver

Claims (6)

  At least two image sensors that are mounted on a moving body and capture an image with incident light from the outside including the subject to be imaged, and a brightness value histogram of the image captured by the image sensor, and the generated brightness value An exposure parameter setting means for setting an exposure parameter including a shutter speed based on a histogram; and determining whether or not the set exposure parameter is within a predetermined range. A high-brightness imaging part removal necessity determination means for determining whether or not the high-brightness imaging part needs to be removed, and the high-brightness imaging part A high-brightness imaging part extraction / removal means for extracting the high-brightness imaging part and removing it from the image when it is determined that it is necessary to remove Exposure control device for a camera according to claim.   The exposure parameter setting means calculates a luminance value of the photographing target in the image and a maximum luminance value of the entire image, generates a luminance value histogram for the calculated luminance value, and generates the generated luminance value 2. The exposure control apparatus for a camera according to claim 1, wherein an exposure parameter including a shutter speed is set based on the histogram.   3. The camera exposure control apparatus according to claim 2, wherein the object to be photographed is a person, and the exposure parameter setting means calculates a brightness value of a human face and a maximum brightness value of the entire image.   The high-luminance imaging region extraction / removal means calculates an average value of luminance values for each predetermined block of the image, extracts a block group having an average luminance value equal to or greater than a predetermined value, and extracts the block group The camera exposure control apparatus according to claim 1, wherein the high-luminance imaging region is extracted by calculating a center position of the camera.   The high-luminance imaging region extraction / removal means calculates an average value of luminance values for each predetermined block of the image, extracts a block having the largest calculated average value, and based on the extracted block, 5. The exposure control apparatus for a camera according to claim 1, wherein a high-luminance imaging part is extracted.   The high-intensity imaging part extraction / removal means specifies a position and an angle of the high-intensity imaging part and generates a predetermined action on the moving body so as to remove the specified high-intensity imaging part from the image. An exposure control apparatus for a camera according to claim 1, wherein