JP2009056529A - Legged mobile robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a legged mobile robot configured to take an image of a target with an appropriate brightness value even if a highly bright light source such as solar light comes into a mounted camera (imaging element). <P>SOLUTION: When an imaged region with high brightness taken by highly bright incident light is present on an image picked up by the imaging element, a hand is driven to block the highly bright incident light (S18 to S28), a head is rotated to avoid the highly bright incident light (S36 to S48), or legs are driven to avoid the light. Brightness reduction for reducing the brightness of the highly bright imaged region is thus carried out. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a legged mobile robot, and more specifically, includes a CCD camera (imaging device) as a visual sensor, and sets an appropriate luminance value even when a high-intensity light source such as sunlight is reflected on the camera. This is related to a legged mobile robot that can be used for shooting.

Conventionally, for example, in Patent Document 1 below, when a high-intensity light source such as sunlight is reflected within the angle of view of a camera, the position, shape, and the like of a ghost generated by internal reflection on the image sensor surface are predicted, There has been proposed a technique for determining which part of a photographed image is a ghost, and correcting (reducing processing) the determined ghost part by designation of a photographer or automatically.
JP 2006-129084 A

  The technology described in Patent Document 1 is a so-called digital camera or video camera, which is a technology related to a camera that a user operates to shoot a subject such as a person, and such a camera is a legged mobile robot or the like. It may be mounted on a moving body and used as a visual sensor.

  In such a case, as shown in FIG. 9, when a high-intensity light source such as sunlight or a spotlight is imaged on the image sensor, the luminance value of a subject to be photographed such as a person who wants to shoot in the dark area is destroyed due to the light source In some cases, the subject to be photographed cannot be specified. In FIG. 9, the person is actually present on the right side of the screen.

  Accordingly, the object of the present invention is to solve the problems described above, and to shoot a subject to be photographed with an appropriate luminance value even when a high-intensity light source such as sunlight is reflected on the camera (image sensor) to be mounted. It is to provide a mobile robot.

  In order to solve the above-described problem, in the legged mobile robot according to claim 1, an image pickup device that picks up an image with incident light from the outside including the subject to be imaged, and an image picked up by the image pickup device. When there is a high-brightness imaging part imaged by high-luminance incident light, the apparatus is configured to include a brightness reduction operation executing unit that executes a brightness reduction operation for reducing the brightness of the high-brightness imaging part.

  The legged mobile robot according to claim 2 includes at least a base, an arm connected to the base, and a hand connected to the arm. The hand is driven to block the high-luminance incident light, thereby reducing the luminance of the high-luminance imaging region.

  In the legged mobile robot according to claim 3, the subject to be imaged is a person, and the brightness reduction operation executing means drives the hand while holding the image parameter of the person to increase the brightness of the person. It was configured to block incident light.

  The legged mobile robot according to claim 4 includes at least a base and a head connected to the base and on which the imaging device is mounted, and the brightness reduction operation executing means includes the head The unit is rotated to reduce the luminance of the high-luminance imaging region.

  The legged mobile robot according to claim 5 includes at least a base, a head, a leg, and an arm connected to the base, and the brightness reduction operation executing means drives the leg. Thus, the brightness of the high brightness imaging region is reduced.

  In the above, “high luminance incident light” means incident light having a relatively high luminance such as a light source, and “high luminance imaging portion” means a portion that is imaged by high luminance incident light.

  In the legged mobile robot according to claim 1, the high-brightness imaging part imaged by the high-intensity incident light is captured on the image captured by the imaging element that captures the image by the incident light from the outside including the imaging target. In some cases, since it is configured to include a brightness reduction operation executing means for executing a brightness reduction operation for reducing the brightness, even when there is a high brightness imaging part imaged by high brightness incident light such as sunlight, By reducing the brightness, a subject such as a person can be photographed with an appropriate brightness value.

  The legged mobile robot according to claim 2 has at least a hand or the like, and is configured to reduce the luminance of the high-luminance imaging region by driving the hand and blocking high-luminance incident light. In addition to the effects described above, a subject such as a person can be photographed with an appropriate luminance value without improving the performance of the image sensor or image processing.

  In the legged mobile robot according to the third aspect, since the hand is driven to block the high-intensity incident light while maintaining the image parameters of the person to be imaged, in addition to the effects described above. The person who is the subject of photography can be reliably photographed.

  The legged mobile robot according to claim 4 has at least a head on which an image sensor is mounted and is configured to reduce the luminance of the high-luminance imaging region by rotating the head. In addition, it is possible to shoot a subject such as a person with an appropriate luminance value without improving the performance of the image sensor or image processing, and reliably shoot the subject even when the high-luminance imaging part is relatively large. Can do.

  In the legged mobile robot according to the fifth aspect of the present invention, the legged mobile robot has at least a leg connected to the base body and is configured to drive the leg to reduce the luminance of the high-luminance imaging region. It is possible to shoot a subject such as a person with an appropriate luminance value without improving the performance of the image sensor and image processing, and more reliably shoot the subject even when the high-luminance imaging part is relatively large. Can do.

  The best mode for carrying out a legged mobile robot according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a front view of a legged mobile robot according to this embodiment, and FIG. 2 is a side view of the robot shown in FIG.

  As shown in FIG. 1, a legged mobile robot (moving body; hereinafter simply referred to as “robot”) 10 includes two left and right leg portions 12L and 12R (L on the left side and R on the right side; the same applies hereinafter). . The leg portions 12L and 12R are connected to the lower portion of the base 14. A head portion 16 is connected to the upper portion of the base body 14, and two left and right arm portions 20L and 20R are connected to the side. Hands (end effectors) 22L and 22R are connected to the tips of the left and right arm portions 20L and 20R, respectively. In this embodiment, a humanoid robot having a height of about 1.3 m and having two legs and two arms is taken as an example of a legged mobile robot.

  As shown in FIG. 2, a storage portion 24 is provided on the back of the base 14, and an electronic control unit (hereinafter referred to as “ECU”) 26, a battery (not shown), and the like are accommodated therein.

  FIG. 3 is an explanatory diagram showing the robot 10 shown in FIG. 1 as a skeleton. Hereinafter, the internal structure of the robot 10 will be described with a focus on joints with reference to FIG. Since the illustrated robot 10 is bilaterally symmetric, L and R are not described below.

  The left and right legs 12 include a thigh link 30, a crus link 32, and a foot 34, respectively. The thigh link 30 is connected to the base body 14 via a hip joint. In FIG. 3, the base body 14 is simply shown as the base link 36, but the base link 36 is configured such that the upper half 36 a and the lower half 36 b can be relatively displaced via a joint 38.

  The thigh link 30 and the lower leg link 32 are connected via a knee joint, and the lower leg link 32 and the foot 34 are connected via an ankle joint. The hip joint includes a rotation axis 40 around the Z axis (yaw axis), a rotation axis 42 around the Y axis (pitch axis), and a rotation axis 44 around the X axis (roll axis). That is, the hip joint has three degrees of freedom.

  The knee joint is composed of a rotation axis 46 around the Y axis and has one degree of freedom. The ankle joint includes a rotation axis 48 around the Y axis and a rotation axis 50 around the X axis, and has two degrees of freedom. In this way, each of the left and right leg parts 12 is provided with six rotation axes (degrees of freedom) constituting three joints, and the leg part as a whole is provided with a total of twelve rotation axes.

  The leg 12 is driven by an actuator (not shown). The leg actuator that drives the leg 12 includes twelve electric motors arranged at appropriate positions of the base 14 and the leg 12, and individually drives the twelve rotating shafts described above.

  The left and right arm portions 20 include an upper arm link 52 and a lower arm link 54, respectively. The upper arm link 52 is connected to the base body 14 through a shoulder joint. The upper arm link 52 and the lower arm link 54 are connected via an elbow joint, and the lower arm link 54 and the hand 22 are connected via a wrist joint.

  The shoulder joint includes a rotation axis 56 around the Y axis, a rotation axis 58 around the X axis, and a rotation axis 60 around the Z axis, and has three degrees of freedom. The elbow joint is composed of a rotation shaft 62 around the Y axis and has one degree of freedom. The wrist joint includes a rotation axis 64 around the Z axis, a rotation axis 66 around the Y axis, and a rotation axis 68 around the X axis, and has three degrees of freedom. In this manner, each of the left and right arm portions 20 is provided with seven rotation axes (degrees of freedom) constituting three joints, and a total of 14 rotation axes are provided as the entire arm portion.

  The arm portion 20 is also driven by an actuator (not shown) as with the leg portion 12. The arm actuator for driving the arm 20 is composed of 14 electric motors arranged at appropriate positions of the base 14 and the arm 20 and individually drives the 14 rotation shafts described above. In the robot 10, the movement of the leg actuator or the arm actuator is controlled so that each rotation axis is driven at an appropriate angle, whereby a desired movement is given to the leg 12 or the arm 20.

  The hand 22 is provided with five finger portions 70. The finger unit 70 can be operated by a hand actuator (not shown), and can perform operations such as gripping an object in conjunction with the movement of the arm unit 20 or pointing in an appropriate direction.

  The head 16 is connected to the base 14 via a neck joint. The neck joint includes a rotation shaft 72 around the Z axis and a rotation shaft 74 around the Y axis, and has two degrees of freedom. The rotary shafts 72 and 74 are also individually driven by a head actuator (not shown). By controlling the operation of the head actuator and driving the rotary shafts 72 and 74 at an appropriate angle, the head 16 can be directed in a desired direction. The base link 36 also drives an actuator (not shown) disposed at the joint 38, whereby the upper half 36a and the lower half 36b are relatively rotated.

  A force sensor (6-axis force sensor) 76 is attached to each of the left and right legs 12, and the three-direction components Fx, Fy, Fz of the floor reaction force acting on the leg 12 from the floor and the three-direction components Mx of the moment. , My, and Mz are output. The same type of force sensor 78 is also attached to the left and right arm portions 20 between the hand 22 and the wrist joint, and the three-direction components Fx, Fy, Fz of the external force acting on the arm portion 20 and the three-direction components Mx, My, A signal indicating Mz is output.

  A tilt sensor 80 is installed on the base 14 and outputs a signal indicating a state quantity such as the tilt angle of the base 14 with respect to the vertical axis and its angular velocity. The head 16 is provided with two (left and right) imaging elements, specifically a CCD camera (hereinafter referred to as “camera”) 82, which captures an image with incident light from the outside including a subject such as a person. The head 16 is provided with a voice input / output device 84 including a microphone 84a and a speaker 84b.

  The output of the above-described sensor or the like is input to the ECU 26 (shown in FIG. 2). The ECU 26 includes a microcomputer and includes a CPU, an input / output circuit, a ROM, a RAM, and the like (not shown).

  FIG. 4 is a block diagram showing the configuration of the robot 10 with the input / output relationship of the ECU 26 as the center.

  As shown in the figure, the robot 10 is carried by a rotary encoder group 86, a gyro sensor 88, a GPS receiver 90, and a person (photographing target) arranged on each of the rotating shafts 40 in addition to the above-described sensors. An IC tag signal receiver (reader) 94 that is connected to the IC tag 92 wirelessly and receives IC tag information transmitted from the IC tag 92 is provided.

  Each of the rotary encoder groups 86 outputs a signal corresponding to the rotation angle of the rotary shaft 40 or the like, that is, the joint angle. The gyro sensor 88 outputs a signal corresponding to the moving direction and distance of the robot 10. The GPS receiver 90 receives a radio wave transmitted from a satellite, acquires position information (latitude and longitude) of the robot 10 and outputs it to the ECU 26.

  The IC tag signal receiver 94 is stored in the IC tag 92 and transmitted from the identification information (RFID (Radio Frequency ID), specifically, identification information for identifying the person who carries the IC tag 92). Is received by the wireless system and output to the ECU 26.

  The ECU 26 performs gait control by generating a gait based on outputs from the force sensor 76, the tilt sensor 80, the rotary encoder group 86, and the like. Specifically, the operation of the above-described leg actuator (indicated by reference numeral 100) is controlled to drive the leg 12 to move (walk) the robot 10. Since gait generation and walking control are described in Japanese Patent No. 3726081 proposed by the present applicant, detailed description thereof is omitted.

  The ECU 26 controls the operation of the arm actuator (indicated by reference numeral 102) and the hand actuator (indicated by reference numeral 104) in association with walking control and the like to drive the arm section 20 and the hand 22, and the head actuator. The direction of the head 16 is adjusted by controlling the operation (indicated by reference numeral 106).

  Further, when a high-luminance imaging region is imaged by the camera (imaging device) 82, the ECU 26 executes a luminance reduction operation for reducing the luminance.

  FIG. 5 is a block diagram functionally showing the configuration when the ECU 26 executes the brightness reduction operation.

  As shown in the figure, when the operation of the ECU 26 at that time is viewed by function, the ECU 26 includes a stereo processing unit 26a, a histogram generation unit 26b, an exposure parameter setting unit 26c, an image processing unit 26d, and an action generation unit 26e. Become.

  The stereo processing unit 26a receives the outputs of two cameras (imaging devices) 82 mounted on the moving body (robot) 10, and calculates distance information for each pixel from the parallax by stereo processing. The number of pixels of the camera 82 is 320 × 240. The stereo processing unit 26a calculates and outputs three-dimensional (3D) data from the grayscale image data.

  The histogram generation unit 26b creates a luminance value histogram of the photographed image and performs weighting for each distance or according to the distance.

  The exposure parameter setting unit 26c sets the exposure parameter (more specifically, the shutter speed) from the luminance value at the distance to be imaged. Since the camera 82 is mounted on the robot 10 and functions as a visual sensor, the camera 82 does not search and shoot the shooting target itself, but must extract the shooting target from the shot image. Therefore, the diaphragm of the camera 82 is fixed to the minimum, and is adjusted so as to be focused at a short distance, specifically, about 0.5 m to 2.5 m. Therefore, only the shutter speed can be adjusted as an exposure parameter.

  The image processing unit 26d performs image processing as a visual sensor when the robot 10 moves.

  When the camera 82 has a high-brightness imaging part imaged with high-brightness incident light such as a light source, the behavior generation unit 26e performs a luminance reduction operation for reducing the luminance.

  Next, the operation of the ECU 26 will be described in more detail.

  6 is the first half of a flow chart showing the processing when the ECU 26 performs a luminance reduction operation, and FIG. 7 is the second half thereof, as in FIG.

  In the following, it is determined whether or not there is a person (photographing target) in S10. This is determined from the output of the IC tag signal receiver 94 that receives the identification information transmitted from the IC tag 92 carried by a person.

  When the result in S10 is negative, the above process is repeated. When the result is positive, the process proceeds to S12, moves to approach a person, proceeds to S14, approaches within a predetermined distance, then proceeds to S16, and further approaches. .

  Next, the process proceeds to S18, in which it is determined whether the direction and position of the high-luminance imaging region in the image has been detected. If the result is affirmative, the process proceeds to S20, and the optimum posture of the hand 22 is calculated and the result is negative. Proceed to S22 and adopt the default (default value) as the posture of the hand 22.

  Next, in S24, the hand 22 is held over a light source such as sunlight, more precisely, a high-brightness imaging region, that is, the hand 22 is driven to operate so as to block high-brightness incident light such as a light source.

  Next, the process proceeds to S26, the position of the hand 22 is brought closer (pulled toward the base 14), and the process proceeds to S28, where the hand 22 has become larger than the light source, more precisely, the high-luminance imaging region, in other words, the high-luminance. It is determined whether or not the incident light is blocked by the hand 22, and when the result is negative, the process returns to S24 and the above processing is repeated.

  FIG. 8 is an explanatory diagram for explaining the luminance reduction operation of the high-luminance imaging region by the hand 22 described above.

  As shown in the figure, the robot 10 operates to drive the hand 22R to block high-intensity incident light (optical axis) from the light source. The position of the hand 22R approaches the camera 82 mounted on its own head 16 in the order of A, B, and C, and the area for blocking incident light from the light source increases, while the imaging region for humans can be blocked. Sex is also increased.

  Therefore, in the processing from S18 to S28, the position where the high-luminance incident light from the light source is completely blocked while holding the imaging portion of the person, in other words, holding the image parameter of the person, that is, The hand 22 is driven so as to be as close to A as possible.

  Returning to the description of FIG. 6, the process proceeds to S30, where the exposure parameter of the person, that is, the shutter speed is calculated, and the process proceeds to S32 to determine the possibility of extracting the face from the person image.

  Next, the process proceeds to S34, where it is determined whether or not a face cannot be extracted. If the determination is affirmative, the process proceeds to S36, and the maximum luminance value of the high-luminance imaging region imaged by high-luminance incident light such as a light source It is determined whether or not the value is exceeded. If the result is affirmative, the process proceeds to S38, and the amount of rotation of the head 16 is calculated.

  Next, the process proceeds to S40, and the direction and size of the movement of the head 16 for shifting (avoiding) high-intensity incident light (optical axis) such as a light source is calculated. The process proceeds to S42, and the head 16 is moved relative to the target direction. The movement command value is determined, the process proceeds to S44, the head 16 is moved (rotated) accordingly, the process proceeds to S46, and limit processing is performed. That is, since it is determined that the luminance of the high-luminance imaging region has not been sufficiently reduced by driving the hand 22 from S18 to S28, the head 16 is rotated to avoid high-luminance incident light.

  Next, the process proceeds to S48, a face extraction process is executed, and the process proceeds to S50, in which it is determined whether or not a face has been extracted. When the result in S50 is negative, the program proceeds to S52 (FIG. 7), and it is determined whether or not the average value of the luminance values of the high-luminance imaging region exceeds the extracted face luminance value. When the result in S52 is affirmative, the program proceeds to S54, and the correction amount of the gait (rotation, etc.) is calculated. If the result in S52 is NO, S54 is skipped.

  Next, in S56, a gait command value is output according to the calculated correction amount, and in S58, the leg portion 12 is driven to execute walking control. Thereafter, the process returns to S18, and the above processing is repeated. If the result in S34 is negative or the result in S50 is affirmative, the subsequent processing is skipped because the face can be extracted.

  In the legged mobile robot 10 according to this embodiment, an image sensor (CCD camera) 82 that captures an image with incident light from the outside including a subject such as a person, and an image captured by the image sensor. When there is a high-luminance imaging part imaged by high-luminance incident light, luminance reduction operation executing means (ECU 26, stereo processing unit 26a, histogram generation unit 26b) that executes a luminance reduction operation for reducing the luminance of the high-luminance imaging part The exposure parameter setting unit 26c, the image processing unit 26d, and the action generation units 26e, S10 to S58), so that even when there is a high-luminance imaging part imaged by high-luminance incident light such as sunlight. By reducing the luminance, it is possible to photograph a subject such as a person with an appropriate luminance value.

  In addition, it has at least a base body 14, an arm portion 20 connected to the base body, and a hand 22 connected to the arm portion, and the brightness reduction operation executing means drives the hand 22 to achieve the high brightness. Since the brightness of the high-brightness imaging region is reduced by blocking the incident light (S18 to S28), in addition to the effects described above, it is possible to shoot a person or the like without increasing the performance of the imaging device or image processing. The object can be photographed with an appropriate luminance value.

  In addition, the subject to be imaged is a person, and the luminance reduction operation executing means drives the hand 22 to hold the high-luminance incident light while holding a human imaging part, in other words, a human image parameter. Since it is configured to block (S18 to S28), in addition to the above-described effects, it is possible to reliably photograph the person who is the subject of photographing.

  In addition, at least a head 14 connected to the base 14 and mounted with the imaging device is mounted, and the brightness reduction operation executing means rotates the head 16 to perform the high brightness imaging. Since it is configured to reduce the luminance of the part (S36 to S46), similarly, it is possible to photograph a photographing target such as a person with an appropriate luminance value without increasing the performance of the image sensor and image processing. Even when the luminance imaging region is relatively large, the imaging object can be reliably imaged.

  In addition, it has at least a base 14, a head 16, a leg 12, and an arm 20 connected to the base, and the brightness reduction operation executing means drives the leg 12 to thereby provide the high brightness imaging part. In the same manner, it is possible to shoot a shooting target such as a person with an appropriate luminance value without increasing the performance of the image sensor or the image processing, and the high luminance. Even when the imaging region is relatively large, the imaging object can be imaged more reliably.

  In the above description, a person is scheduled as an object to be photographed, but other objects such as tools and works scheduled for the work of the robot 10 may be used.

  In the above description, the presence / absence of the photographing target is determined from the output of the IC tag signal receiver 94 that receives the identification information transmitted from the IC tag 92 carried by the photographing target. Also good. Further, it may be taught by inputting a command from the outside.

  In the above description, a legged mobile robot, specifically a bipedal walking robot, is illustrated as an example of the moving body. However, the mobile body is not limited to this, and any mobile body may be used as long as it can move autonomously.

1 is a front view of a legged mobile robot according to an embodiment of the present invention. It is a side view of the robot shown in FIG. It is explanatory drawing which shows the robot shown in FIG. 1 with a skeleton. FIG. 2 is a block diagram showing the configuration of the robot shown in FIG. 1 focusing on the input / output relationship of an electronic control unit (ECU). FIG. 5 is a block diagram functionally showing a configuration when the electronic control unit shown in FIG. 4 performs a luminance reduction operation of a high-luminance imaging region. FIG. 6 is a first half of a flow chart showing processing when the electronic control unit performs a luminance reduction operation of the high-luminance imaging region, as in FIG. 5. 6 is the latter half of the flow chart. It is explanatory drawing explaining the brightness reduction operation | movement by the hand in the process of FIG. It is an example of the scene which image | photographs imaging | photography with the camera (imaging element) which concerns on this Example.

Explanation of symbols

  10: Leg type mobile robot (moving body, robot), 12: Leg, 14: Base, 20: Arm, 26: ECU (Electronic Control Unit), 26a: Stereo processing unit, 26b: Histogram generation unit, 26c: Exposure parameter setting unit, 26d: image processing unit, 26e: action generation unit, 82: CCD camera (imaging device, camera), 92: IC tag, 94: IC tag signal receiver

Claims (5)

  When there is an imaging device that captures an image with incident light from the outside including the imaging target, and a high-luminance imaging region that is imaged with high-luminance incident light in the image captured by the imaging device, the high-luminance imaging region A legged mobile robot comprising a luminance reduction operation executing means for executing a luminance reduction operation for reducing luminance.   The apparatus includes at least a base, an arm connected to the base, and a hand connected to the arm, and the brightness reduction operation execution unit drives the hand to block the high-luminance incident light. The legged mobile robot according to claim 1, wherein the luminance of the high luminance imaging region is reduced.   3. The luminance reduction operation execution means drives the hand and blocks the high-intensity incident light while holding a human image parameter while the photographing object is a person. Legged mobile robot.   At least a base connected to the base and a head on which the imaging element is mounted, and the brightness reduction operation executing means rotates the head to reduce the brightness of the high brightness imaging region. The legged mobile robot according to any one of claims 1 to 3, wherein:   And at least a head, a leg, and an arm connected to the base, and the luminance reduction operation executing means drives the leg to reduce the luminance of the high-luminance imaging region. The legged mobile robot according to any one of claims 1 to 5, wherein