JPH11213157A - Camera mounted mobile object - Google Patents

Abstract

PROBLEM TO BE SOLVED: To place a camera so that its image pickup part comes at the same position and in the same direction against environment at different time even during move when an image processing is performed by picking up an image of the environment in which not only a moving object but a still object exists against the environment in a mobile object on which the camera is mounted. SOLUTION: A position of a CCD camera 1 against the environment is fixed by providing a single-axis robot 2 on a moving part 3 and moving the CCD camera 1 on the moving part 3 according to the move of the moving part 3. The moving object is extracted by using an image of the CCD camera 1 which is still against the environment and positional relation between a feature point and the traveling object in the environment is calculated by using an extraction result of the moving object and the image of the CCD camera at other time.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile body equipped with a camera, and more particularly to a mobile body equipped with a camera such as an autonomous mobile robot.

[0002]

2. Description of the Related Art Conventionally, an autonomous mobile robot usually includes a moving section for moving the robot and a camera which plays a role of visual function of the robot. The following are known techniques for processing an image captured by this camera.

(1) As a method for extracting a moving object, a method using a time difference between frames of a moving image captured by a camera stationary with respect to an environment is known. This is to extract the moving object area by using the fact that the time difference between frames takes zero because the stationary object does not change over time, but the time difference does not become zero around the moving object. is there. However, the condition is that the camera is stationary with respect to the environment.

[0004] (2) As a technique for obtaining a positional relationship between an environmental feature point and a moving body, there is a technique of calculating the position of an environmental feature point from an optical flow and movement information of a moving body, and a moving stereoscopic view. . The optical flow refers to a local motion in the target image. As typical extraction methods of the optical flow, a feature matching method for examining a correspondence between features between images and a gradient method for obtaining a movement amount from a brightness gradient in spatiotemporal space are known. The feature matching method uses the image f at time t.
A feature is obtained from (x, y, t) and another image f
A corresponding part is obtained from (x, y, t + Δt). In order to take correspondence, correlation between images of a neighboring region including a feature is used, or similarity of attributes such as a direction of a feature and contrast is used. This method has the advantage that the movement of an important part can be obtained. In particular, it is known as MPEG block matching, which is a standard for compressing moving images.

In moving stereoscopic vision, a feature point is associated between an image photographed at a point A and an image photographed at a point B by a camera on a moving body moving through the points A and B. Is a method of calculating the position of a feature point using triangulation. However, correct values cannot be obtained unless the feature points are stationary.

(3) At the University of Southern California, an attempt has been made to detect obstacles from a translating observation system by using polar coordinate transformation called Complex Logarithmic Mapping (CLM). Pp. 66-67).

Further, (4) camera shake prevention technology has been put to practical use in 8-mm video cameras and the like.

[0008]

However, if a camera is fixedly mounted on a moving body, the camera moves with the moving body, so that the camera cannot be kept stationary with respect to the environment. There was no other way but to stop. Therefore, in order to use the above-mentioned (1) mobile object extraction method, it was necessary to stop the mobile object.

[0009] In addition, when optical flow is used in (2) the method of obtaining the positional relationship between the feature points of the environment and the moving object, it is effective when only a stationary object is present in the environment image. , A static object and a moving object in the environmental image
If there is one, optical flow segmentation (division) is required. Flow segmentation often uses the assumption that a scene is approximated by multiple planes. If the moving object does not occupy a large area on the screen, it is assumed that everything is stationary and the relative motion with the camera and the three-dimensional structure are obtained, and the flow is predicted from the motion, and the region having a flow different from the prediction is calculated. Is extracted as a region of a moving object. This flow segmentation has the disadvantage that the computational load increases. In addition, when using the moving stereoscopic vision, there is a disadvantage that a correct value cannot be obtained unless the feature points of the environment are stationary. Therefore, it is necessary to extract a stationary object from the environment image in advance.

The above-mentioned method (3) of detecting an obstacle from a translationally moving observation system requires polar coordinate conversion and logarithmic conversion when an image is converted into a CLM image, so that the calculation load is large. There is.

Further, the above-mentioned camera shake preventing technique (4) corrects a change in the attitude of the camera, and does not correct a change in the position of the camera.

[0012] When the camera mounted on the moving body moves together with the moving body, the camera cannot be maintained at a position that is stationary with respect to the environment. Can't judge it,
Even if it can be determined, the calculation load increases. For example,
The whole moving image obtained by the camera while the moving body is moving is moving, and it is actually difficult to distinguish a stationary wall or door from a moving person.

[0013]

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems, and has a camera-mounted moving body in which an imaging unit is placed in the same position and direction with respect to the environment at different times even during movement. To provide.

It is another object of the present invention to provide a camera-mounted moving body that controls an image pickup unit to be positioned at the same position and direction with respect to the environment independently of movement control of the moving body even during movement. It is.

Still another object of the present invention is to provide a camera-equipped mobile body in which the image pickup unit is placed at the same position and direction with respect to the environment at different times and the image pickup unit is placed at different positions with respect to the environment at different times. To provide.

Still another object of the present invention is to facilitate the extraction of a moving object even when a moving object as well as a stationary object is present in an environmental image taken while moving, and to provide a method for extracting the moving object and the environment. An object of the present invention is to provide a camera-equipped mobile body that accurately determines a positional relationship with a feature point.

Still another object of the present invention is to continuously or intermittently or continuously determine the positional relationship between a moving object and a feature point in an environment while the moving object is moving. It is to provide.

Still another object of the present invention is to provide a camera-mounted moving body that improves the quality of a captured image during movement.

According to one aspect of the present invention, a moving unit;
An imaging unit moving unit attached to the moving unit; an imaging unit held by the imaging unit moving unit; a detection unit that detects a speed at which the moving unit moves; and a position of the imaging unit with respect to the environment according to an output of the detection unit. And a control unit for controlling the imaging unit moving unit such that the direction is the same at the first time and the second time.

According to the present invention, it is possible to provide a camera-mounted moving body in which the imaging unit is located at the same position and in the same direction with respect to the environment at different times even during movement.

According to another aspect of the present invention, a moving unit;
An imaging unit moving unit attached to the moving unit; an imaging unit held by the imaging unit moving unit; a detection unit that detects a moving speed of the imaging unit; and a position of the imaging unit with respect to the environment according to an output of the detection unit. A control unit that controls the imaging unit moving unit so that the direction is the same at the first time and the second time.

According to the present invention, it is possible to provide a camera-mounted moving body that controls the imaging unit to be placed at the same position and direction with respect to the environment independently of the movement control of the camera-mounted moving body.

More preferably, the control means of the camera-mounted moving body controls the image pickup unit moving means so that the image pickup unit is placed at the same position and direction with respect to the environment at the first time and the second time. The first control and the second control for controlling the imaging unit moving means are performed so that the imaging unit is located at different positions with respect to the environment at the first time and the third time.

According to the present invention, the imaging unit is placed at the same position and direction with respect to the environment at different times by the first control,
With the second control, it is possible to provide a camera-mounted moving body that positions the imaging unit at different times with respect to the environment at different times.

More preferably, the camera-mounted moving body further has an image processing section for processing an image picked up by the image pickup section, and the image processing section uses the image obtained by the first control to enter the environment. A first image processing for extracting a moving object is performed, and a second relation for obtaining a positional relationship between a moving object and a feature point in an environment using a result of the first image processing and an image obtained by the second control. Perform image processing.

According to the present invention, even in the case where not only a stationary object but also a moving object is present in the environmental image taken while the moving object with the camera is moving, the moving object can be easily extracted and the moving object and the environment can be easily extracted. It is possible to provide a camera-equipped mobile body that accurately determines a positional relationship with a feature point in the camera.

[0027] More preferably, the control means of the camera-mounted moving body performs the first control and the second control continuously and repeatedly.

According to the present invention, it is possible to provide a camera-mounted mobile body that can accurately and easily determine the positional relationship between the mobile body and characteristic points in the environment while the camera-mounted mobile body moves.

According to another aspect of the present invention, a camera-mounted moving body includes at least two imaging units, and imaging unit moving means capable of holding and moving the imaging units, respectively.
A moving unit to which the imaging unit moving unit is attached, a detecting unit for detecting a speed at which the moving unit moves, and a position and a direction with respect to the environment of at least one imaging unit corresponding to the first time according to an output of the detecting unit. And control means for controlling the imaging section moving means so as to be the same at the second time.

According to the present invention, it is possible to provide a camera-mounted moving body that places at least one image pickup unit at the same position and at the same time with respect to the environment at different times during movement.

According to another aspect of the present invention, a camera-mounted moving body includes at least two imaging units, and imaging unit moving means capable of holding and moving the imaging units, respectively.
A moving unit to which the imaging unit moving unit is attached; a detection unit for detecting the movement of the imaging unit; and a position and a direction of at least one imaging unit with respect to the environment corresponding to the first time and the second time. And control means for controlling the imaging unit moving means so as to be the same at the time.

According to the present invention, even during the movement, at least one time at a different time independently of the movement of the moving body.
It is possible to provide a camera-equipped mobile body in which two image pickup units are placed at the same position and direction with respect to the environment.

[0033] More preferably, the control means of the camera-equipped moving body is arranged such that the position and direction of the at least one imaging unit with respect to the environment are always the same between the first time and the second time. It is characterized by controlling the moving means.

According to the present invention, at least one image pickup unit can always be placed at the same position and direction with respect to the environment at different times even while moving, and a camera mounted to prevent a decrease in image quality due to the flow of images. A moving object can be provided.

[0035] More preferably, the at least two imaging units include a first imaging unit and a second imaging unit, and the camera-mounted moving body further includes an image processing unit that processes images captured by the at least two imaging units. The image processing unit performs first image processing for extracting a moving object in the environment using the image captured by the first imaging unit, and performs the first image processing result and the second imaging The second image processing is performed to obtain the positional relationship between the moving object and the feature point in the environment using the image captured by the unit, and the first imaging unit performs the first image processing on the environment at the first time and the second time. The position and the direction are the same, and the second imaging unit is different from the first imaging unit.

According to the present invention, it is possible to provide a camera-equipped mobile object that can accurately and easily determine the positional relationship between the mobile object and feature points in the environment while the mobile object moves.

[0037] More preferably, the imaging section moving means of the camera-mounted moving body is capable of independently changing the distance between the optical axis of the visual field of the imaging section and the moving section.

According to the present invention, it is possible to provide a camera-mounted moving body that captures only environmental images in the camera's field of view.

[0039]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same reference numerals indicate the same or corresponding parts.

FIG. 1 is a perspective view showing the appearance of a floor moving vehicle (a kind of moving body) according to the first embodiment of the present invention, and FIG. 2 is a functional block diagram thereof. Referring to the figure, a floor moving vehicle includes a CCD camera 1, a single-axis robot 2, and a moving unit 3
, A control unit 6, a drive motor 7, wheels 8, an encoder 9, and an image processing unit 10.

The moving section 3 is capable of moving forward or backward in the direction of the arrow by driving the drive motor 7 and rotating the wheels 8 in accordance with instructions from the control section 6. Single axis robot 2
Is mounted on the moving unit 3 so that its axis is parallel to the traveling direction of the moving unit 3. The CCD camera 1 is supported by a movable base of the single-axis robot 2 such that the optical axis of its visual field is parallel to the traveling direction of the moving unit 3. Therefore CC
The D camera 1 can move in parallel with the moving direction of the moving unit 3.

The control section 6 sends a movement instruction to the drive motor 7 to rotate the wheels 8. As a result, the moving section 3 moves forward or backward. Since the encoder 9 is connected to the wheel, the rotation direction and the number of rotations of the wheel can be obtained. Thereby, the moving direction and the moving distance of the moving unit 3 can be obtained from the output of the encoder 9.

The moving direction and the moving distance of the moving unit 3 may be obtained from the moving instruction of the control unit 6 to the driving motor 7 without using the encoder.

The control unit 6 determines that the CCD camera 1 is at the same position with respect to the environment at the first time and the second time based on the moving direction and the moving distance of the moving unit 3 obtained as described above. To control the single-axis robot 2. Here, the single-axis robot 2 in the case where the CCD camera 1 is positioned at the forefront of the single-axis robot, and the moving unit 3 is moving at a constant speed of V [m] per second (the forward direction is defined as positive). Will be described. When the CCD camera 1 is positioned at the forefront of the single-axis robot at time t ₀ (first time), the time (t ₀ +
At T) (second time), the position of the CCD camera 1 with respect to the environment is controlled by controlling the single-axis robot 2 so that the CCD camera 1 is located VT [m] behind the single-axis robot 2 from the front. It becomes the same at time t ₀ and (t ₀ + T). If the single-axis robot 2 is controlled so that the CCD camera 1 moves at the speed of -V [m] during the time T, CC
The position of the D camera 1 with respect to the environment remains the same during the time T. As a result, it is possible to prevent the image quality from deteriorating due to the flow of the image, and to obtain a high-quality image even while moving.

Then, the time (t₀+ T), the CCD camera
The camera 1 is moved forward of the single-axis robot 2. this
Time of time is t₁(Third time). Later time
(T ₁+ T) the CCD camera 1 is the single-axis robot 2
Single-axis robot so that it is located behind VT [m] from the front
2, the position of the CCD camera 1 with respect to the environment is
Time t₁And time (t₁+ T). like this
At time t₀From time t ₁Repeat the series of operations up to
Even if the moving unit 3 moves at a constant speed,
With the CCD camera 1 placed at the same position with respect to the environment
The obtained image can be obtained intermittently. Image processing unit 10
Processes an image captured by the imaging unit 1. Its processing and
The images obtained at the first time and the second time, respectively,
There is a process of extracting a moving object using an image. The above
In particular, the environment of the CCD camera 1 at the first time and the second time
Position is the same for
Can do it.

Then, an optical flow operation is performed using the image obtained at the first time or the second time and the image obtained at the third time, and the result, the moving direction of the moving unit 3 and A process (three-dimensional position calculation) of calculating the positional relationship between the feature point in the image and the moving object from the moving distance is performed. The positional relationship between the moving object and the feature point means a distance between the moving object and the feature point, a direction of the feature point from the moving object, and the like. At this time, if the feature points used for the optical flow calculation are divided into the feature points in the moving object and the feature points in the stationary object extracted earlier and the optical flow calculation is performed, the speed of the optical flow calculation is increased, and The feature point position can be obtained accurately.

FIG. 3 is a perspective view showing the appearance of a floor moving vehicle according to a second embodiment of the present invention, and FIG. 4 is a functional block diagram thereof. In this embodiment, a turning mechanism 1 is provided so that the optical axis of the field of view of the CCD camera 1 can be rotated horizontally.
1 is added. The other parts are the same as in the first embodiment, and the description thereof will not be repeated. A case where the turning mechanism 11 is controlled so that the optical axis of the visual field of the CCD camera 1 is perpendicular to the traveling direction of the moving unit 3 will be described.

In this case as well, by controlling the single-axis robot 2 while the moving unit 3 is moving in the same manner as in the first embodiment, the same position and direction with respect to the environment can be obtained at different times.
Since the CD camera 1 can be placed, the moving object can be easily extracted. Further, since it is possible to take an image from two different viewpoints on the moving path of the moving unit 3 with the optical axis of the visual field of the CCD camera 1 perpendicular to the moving path, the correspondence between the feature points of these images is obtained, so that triangulation is performed. To perform moving stereoscopic view for calculating the distance to the feature point. In this case, since the moving object is extracted by the preceding moving object extraction processing, an accurate distance can be calculated by dividing the moving object and the stationary object and performing moving stereoscopic viewing.

In this embodiment, a single-axis robot and a turning mechanism are used as the imaging unit moving means, but a robot that performs a rotary motion may be used instead of the single-axis robot. This also allows the CCD camera to be placed at the same position with respect to the environment at the first time and the second time.

FIG. 5 is a perspective view showing the appearance of a floor moving vehicle according to a third embodiment of the present invention using two CCD cameras, and FIG. 6 is a functional block diagram thereof. The CCD cameras 1 and 4 are respectively supported on movable platforms of the single-axis robots 2 and 5 such that the optical axis of the visual field is parallel to the traveling direction of the moving unit 3. The single-axis robots 2 and 5 are attached to the moving unit 3 so that their axes are parallel to the traveling direction of the moving unit 3.
The control unit 6 issues a movement instruction of the drive motor 7 and acquires the rotation speed and rotation direction of the wheels from the encoder 9. Thereby, the control unit 6 calculates the moving direction and the moving distance of the moving unit 3. Further, the control unit 6 includes the single-axis robots 2 and 5
Are controlled independently. Next, control of the single-axis robots 2 and 5 will be described.

FIG. 7 is a plan view of the floor moving vehicle according to the present embodiment. The case where the moving unit 3 is moving at the speed V0 (positive in the right direction in the drawing) will be described. Here, X1 is a CCD from the left end of the movable range of the single-axis robot 2.
Let X be the distance to the camera 1 and X2 be the distance from the left end of the movable range of the single-axis robot 5 to the CCD camera 4 and the distance of the movable range of the single-axis robot. V1 and V2 are each CC
The relative speed of the D camera 1 and the CCD camera 4 with respect to the moving unit 3 (positive in the right direction in the drawing). Therefore, V0 + V1
Represents the speed of the CCD camera 1 with respect to the environment, and V0 + V2 represents the speed of the CCD camera 4 with respect to the environment.

FIG. 8 shows the speed V0 of the moving unit 3 and the speeds V1 and V1 of the CCD cameras 1 and 4 when the horizontal axis is the time axis.
FIG. 2 is a diagram showing a relationship between the position 2 and positions X1 and X2. At time t = 0, the CCD camera 1 moves C1 to the position of X1 = 0.
The CD camera 4 is at the position of X2 = D. The moving unit 3
During acceleration / deceleration, the CCD camera is controlled to be located at the positions of X1 = 0 and X2 = D.

During the period from t = 0 to t = t1, the moving unit 3 accelerates from the stop state, and reaches the target speed at t = t1. At this time, V1 = V2 = 0.

From t = t1 to t = t2, V1 = V
0, the single-axis robots 2 and 5 are controlled so that V2 = −V0. Thereby, V1 + V0 = 2V0, V2 + V0
= 0, the CCD camera 1 moves at twice the speed of the moving unit 3 with respect to the environment, and the CCD camera 4 stops with respect to the environment. The image processing unit 10 performs a moving object extraction process based on the image of the CCD camera 4, performs an optical flow calculation based on the image of the CCD camera 1, and calculates the calculation result and the moving direction of the moving unit 3. The positional relationship between the moving object and the feature point is obtained from the moving distance.

At time t = t2, since the CCD cameras reach the limit of the movable range, the moving direction of each CCD camera is changed.

From t = t2 to t = t3, V1 = −
The single-axis robots 2 and 5 are controlled so that V0 and V2 = V0. Thus, the CCD camera 1 is stationary with respect to the environment, and the CCD camera 4 moves with respect to the environment at twice the speed of the moving unit 3. The image processing unit 10 performs a moving object extraction process based on the image of the CCD camera 1,
An optical flow operation is performed based on the image of the CCD camera 4.

From t = t3 to t = t4, the control from t = t1 to t = t3 is repeated. t = t4 to t = t
5, the moving unit 3 is decelerated, so that X1 = 0 and X2 = D
The single-axis robots 2 and 5 are controlled so that

From t = t5 to t = t6, the CCD camera does not move with respect to the moving unit 3, and the moving unit 3 decelerates to t = t5.
The speed becomes 0 at t6.

During the period from t = t6 to t = t7, the moving unit 3 accelerates from the stop state and reaches the target speed at the time of t = t7. During this time, the CCD camera does not move.

From t = t7 to t = t8, each CCD camera resumes reciprocating motion. The change pattern between V1 and V2 is the same as the change pattern during the period from t = t1 to t = t4, but the moving direction of the moving unit 3 is reversed, so that the image processing unit 10 uses the moving object extraction processing. The image and the image used for the optical flow calculation are reversed. The image of the CCD camera whose speed with respect to the environment becomes zero is used for the moving object extraction processing, and the image of the other CCD camera is used for the optical flow calculation.

Since the moving unit 3 decelerates from t = t8 to t = t9, the single-axis robots 2 and 5 are controlled so that X1 = 0 and X2 = D.

At t = t10, the moving unit 3 stops, and the control ends. This allows one of the two CCD cameras to be placed at the same position with respect to the environment even while the moving unit 3 is moving.

Further, a moving object can be extracted using an image picked up by any one of the CCD cameras, and a three-dimensional position can be calculated using an image picked up by another CCD camera. As compared with the first embodiment using one unit, the period during which the moving object cannot be extracted and the period during which the three-dimensional position calculation cannot be performed can be significantly reduced.

FIG. 9 shows a fourth embodiment of the present invention in which two CCD cameras and two single-axis robots are mounted on the moving unit 3 and an elevating mechanism is added so that the optical axis of the CCD camera can be moved in the vertical direction. FIG. 10 is a perspective view showing the appearance of a floor-moving vehicle in the form, and FIG. 10 is a functional block diagram thereof. An electric laboratory jack is used for the lifting mechanism.

The elevating mechanism 12 is attached to the movable base of the single-axis robot 2 and supports the turning mechanism 11 at the other end. Similarly, the lifting mechanism 14 is attached to the single-axis robot 5 and supports the turning mechanism 13 at the other end.

When the optical axes of the visual fields of the CCD camera 1 and the CCD camera 4 are made perpendicular to the traveling direction of the moving unit 3, the CCD camera (CCD camera 1 in FIG. 9) which is rearward with respect to the environment in which the image is taken, This is inconvenient because the front CCD camera enters the image. Therefore, by raising the optical axis of the field of view of the CCD camera by the lifting mechanism of the rear CCD camera,
It is possible to prevent the front CCD camera from entering the image of the rear CCD camera. In FIG. 9, the lifting mechanism 12 of the rear CCD camera 1 is raised, and the lifting mechanism 14 of the front CCD camera 4 is lowered.

The control of the single-axis robot 2 and the single-axis robot 5 at this time is the same as in the case of the third embodiment.

FIG. 11 is a perspective view showing the appearance of a floor-moving vehicle according to a fifth embodiment of the present invention in which an acceleration sensor 15 is mounted on the CCD camera 1 and an acceleration sensor 16 is mounted on the CCD camera 4. FIG. 12 is a functional block diagram thereof.

The acceleration sensors 15 and 16 detect the acceleration of the CCD cameras 1 and 4 and transmit the detected acceleration to the imaging unit movement control unit 17. The transmitted acceleration is used by the imaging unit movement control unit 17 for controlling the single-axis robot.

The imaging unit movement control unit 17 includes the acceleration sensor 1
One of the single-axis robots is controlled so that the acceleration detected in steps 5 and 16 becomes zero. Thereby, one CCD camera has the same position with respect to the environment regardless of the traveling state of the moving unit 3. Then, the other single-axis robot is controlled so that the other CCD camera moves in a direction opposite to the movement of one CCD camera. For example, the single-axis robot 2 is a CCD
When moving the camera 1 so that the acceleration of the camera 1 becomes zero, since the image used for the moving object extraction processing is only from the CCD camera 1, the single-axis robot 5 controls the CCD camera 4 regardless of the outputs of the acceleration sensors 15 and 16. Just move it. Then, CCD camera 1 has X1 = 0 and CCD camera 4 has X
If 2 = D (initial position) is reached, a moving object extraction process or the like is performed using the image of the CCD camera 4 in the next period. At this time, the movement of the CCD camera 4 is controlled so that the acceleration detected by the acceleration sensor 16 becomes zero, and the CCD camera 1 is moved to X1 = D (initial position) regardless of the outputs of the acceleration sensors 15 and 16. . In this manner, the CCD camera for reducing the acceleration to zero at regular intervals and the C for returning to the initial position
By switching the CD camera, even when the moving unit 3 is accelerating or decelerating, the image processing unit 10 can perform the moving object extraction processing and the optical flow calculation.

Since the control of the single-axis robots 2 and 5 is independent of the control of the moving unit 3, if the moving unit 3 does not have a movement control function or a driving unit (tow the other moving body). The wheel is slipped even if it has a drive unit, and the speed of movement is one pair with the wheel speed. Even if it does not correspond to 1, CCD camera 1,
4 can be kept stationary with respect to the environment.

In the present embodiment, one of the single-axis robots is controlled by the imaging unit movement control unit 17 so that the acceleration detected by the acceleration sensors 15 and 16 becomes zero. Based on the time measured by the imaging unit movement control unit 17 and the outputs of the acceleration sensors 15 and 16, C
The movement amounts of the CD cameras 1 and 4 are calculated, and the first time and the second time are calculated.
The position of the CCD cameras 1 and 4 with respect to the environment may be controlled to be the same at the time.

FIG. 13 is a functional block diagram of a floor moving vehicle according to the sixth embodiment of the present invention when a speed sensor is mounted on the moving unit 3. The control of the single-axis robots 2 and 5 is the same as in the third embodiment, but the moving direction and speed of the moving unit 3 are detected by a speed sensor 18 mounted on the moving unit 3. This embodiment is different from the third embodiment in that the imaging unit movement control unit 17 controls the single-axis robots 2 and 5 based on the detected movement direction and speed.

In the case of the fifth embodiment in which the acceleration sensor is mounted on the CCD camera, the single-axis robot is controlled so that the output of the acceleration sensor becomes zero, so that accurate control by the null method is possible. However, there is a disadvantage that the driving force required for moving the CCD camera is increased because the weight of the CCD camera is increased. On the other hand, when the speed sensor 18 is mounted on the moving unit 3, there is an advantage that the weight of the camera can be reduced and the driving force required for moving the camera can be reduced.

FIG. 14 is a perspective view showing the appearance of a floor moving vehicle according to a seventh embodiment of the present invention using three CCD cameras. The CCD cameras 1, 4, and 19 are supported on movable platforms of the single-axis robots 2, 5, and 20, respectively, such that the optical axis of the visual field is parallel to the traveling direction of the moving unit 3. The single-axis robots 2, 5, and 20 are attached to the moving unit 3 so that the optical axis is parallel to the traveling direction of the moving unit 3.

The control of the single-axis robots 2, 5, and 20 will be described. If the single-axis robot is controlled so that the speed of the CCD camera is the same in the direction opposite to the moving speed of the moving unit 3, the CCD camera can be kept stationary with respect to the environment. Although two CCD cameras are used in the third embodiment, when the CCD camera changes the moving direction, the CCD camera is accelerated and decelerated at the time of the direction switching.
The camera could not be kept stationary. Therefore, in the present embodiment, three CCD cameras are used, and FIG.
As shown in FIG. 5, the phase of the reciprocating CCD camera is set to 1
Control is performed by shifting by 20 °. In FIG. 15, the CCD camera 1 when the moving unit 3 is moving at the speed of V0.
Is represented by V1, the speed of the CCD camera 4 is represented by V2, and the speed of the CCD camera 19 is represented by V3. As a result, any one of the three CCD cameras is stationary with respect to the environment, so that the moving object can always be extracted, and any one of the three CCD cameras moves in the same direction as the moving unit 3. Therefore, the optical flow calculation for three-dimensional position measurement can always be performed.

When the camera used for the optical flow calculation is switched, the optical flow calculation is performed repeatedly using the images of the two cameras moving in the same direction as the moving body before and after the switching. Optical flow calculation can be performed continuously without interruption. For example, between time t1 and t3 in FIG.
The camera 4 and the CCD camera 19 are moving at the speed of V0. During this time, if the optical flow calculation using the image captured by the CCD camera 4 and the optical flow calculation using the image captured by the CCD camera 19 are performed in parallel, the discontinuity of the moving image at the time of switching can be eliminated. Optical flow calculation for three-dimensional position measurement can be performed continuously without interruption.

FIG. 16 is a perspective view showing the appearance of a floor moving vehicle according to an eighth embodiment of the present invention, showing an embodiment in which a fixed CCD camera is added to the third embodiment. . The CCD camera 19 is fixed to the moving unit 3 with the optical axis of the visual field parallel to the traveling direction of the moving unit 3. The control of the single-axis robots 2 and 5 is the same as that in the third embodiment.

The moving object is extracted using either the image of the CCD camera 1 or the image of the CCD camera 4,
Optical flow calculation for three-dimensional position measurement is always performed using the image of the D camera 19. Of course, three CCD cameras for extracting a moving object may be used. In the present embodiment, an image captured by the same CCD camera is always used for the optical flow calculation, so that the calculation of the optical flow can be performed without interruption. In addition, the aforementioned C
Eliminating the need for duplicate optical flow calculations using the images of the two cameras when switching between CD cameras,
Calculation load can be reduced.

FIG. 17 is a functional block diagram of a floor moving vehicle according to a ninth embodiment in which an acceleration sensor and a gyro sensor are mounted on a CCD camera. The CCD camera 1 has an acceleration sensor 15 and a gyro sensor 21 mounted thereon.
It is supported by the turning mechanism 11. The turning mechanism 11 is mounted on a movable base of the single-axis robot 2,
Is attached to the moving unit 3. Imaging unit movement control unit 17
Controls the single-axis robot 2 and the turning mechanism 11 based on the output of the acceleration sensor 15 and the output of the gyro sensor 21 so that the movement amount of the CCD camera 1 becomes zero. The image processing unit 10 processes an image of the CCD camera 1.

Then, the axis of the single-axis robot 2 is set in the moving unit so as to intersect with the rotation axis when the moving unit 3 rotates in place, and the position of the CCD camera 1 when the moving unit 3 is stationary is moved. The single-axis robot 2 is controlled so as to be on the rotation axis when the unit 3 rotates in place. When the moving unit 3 starts rotating in place from a stationary state, the gyro sensor 21
The rotation angle of the CD camera is detected. The imaging unit movement control unit 17 controls the turning mechanism 11 so that the rotational angular velocity output from the gyro sensor 21 becomes zero. Thus, the camera can be kept stationary even when the moving unit 3 is rotating in place, so that the moving object can be extracted.

FIG. 18 shows that the CCD camera is rotated three degrees of freedom.
FIG. 19 is a perspective view showing an appearance of a floor moving vehicle according to a tenth embodiment mounted on a holding table having a total of six degrees of freedom of three degrees of translation, and FIG. 19 is a functional block diagram thereof.

The CCD camera 1 has a three-axis acceleration sensor 2
Two- and three-axis gyro sensors 23 are mounted. CCD
The camera 1 is fixed on a holding table 25. Holder 2
5 is supported at one end of a linear motion actuator 27 via a ball joint 26. The other end of each of the three linear motion actuators 27 is rotatably connected to the movable base of the single-axis robot 2. The three single-axis robots 2
The axes are fixed on the moving unit 3 so as to be orthogonal to each other.

A combination of three single-axis robots 2, a linear actuator 27, a ball joint, and a holding table (hereinafter collectively referred to as a "six degrees of freedom parallel mechanism") provides three degrees of freedom and three degrees of translation. Exercise with a total of six degrees of freedom is possible.

The imaging unit movement control unit 17 determines the output of the three-axis acceleration sensor 22 and the three-axis gyro sensor 23 based on the outputs.
A control for controlling the 6-degree-of-freedom parallel mechanism so that accelerations in three directions and angular velocities around the three axes are all zero;
The control of returning the CD camera 1 to the initial position with respect to the moving unit 3 is repeated. Here, the initial position with respect to the moving body refers to the position of the CCD camera 1 when the single-axis robot 2 and the linear actuator 27 are located at the center of the movable range. With this configuration, the CCD camera can be moved to the environment for a certain period of time, not only when the moving unit 3 moves on a plane, but also when it has a large degree of freedom such as outdoor rough terrain, underwater, and air. Can be kept stationary.

In the present embodiment, a parallel mechanism is used as a drive mechanism of the holding table having six degrees of freedom, but a holding mechanism having six degrees of freedom may be used.

[Brief description of the drawings]

FIG. 1 is a perspective view of a camera-mounted moving body according to a first embodiment of the present invention.

FIG. 2 is a functional block diagram of the camera-equipped moving body according to the first embodiment of the present invention.

FIG. 3 is a perspective view of a camera-mounted moving body according to a second embodiment of the present invention.

FIG. 4 is a functional block diagram of a camera-mounted moving body according to a second embodiment of the present invention.

FIG. 5 is a perspective view of a camera-mounted moving body according to a third embodiment of the present invention.

FIG. 6 is a functional block diagram of a camera-equipped moving body according to a third embodiment of the present invention.

FIG. 7 is a plan view of a camera-mounted moving body according to a third embodiment of the present invention.

FIG. 8 is a diagram illustrating a speed and a position of a camera of a camera-equipped moving body according to a third embodiment of the present invention.

FIG. 9 is a perspective view of a camera-mounted moving body according to a fourth embodiment of the present invention.

FIG. 10 is a functional block diagram of a camera-mounted moving body according to a fourth embodiment of the present invention.

FIG. 11 is a perspective view of a camera-mounted moving body according to a fifth embodiment of the present invention.

FIG. 12 is a functional block diagram of a camera-mounted moving body according to a fifth embodiment of the present invention.

FIG. 13 is a functional block diagram of a camera-equipped moving body according to a sixth embodiment of the present invention.

FIG. 14 is a perspective view of a camera-mounted moving body according to a seventh embodiment of the present invention.

FIG. 15 is a diagram illustrating a camera speed of a camera-equipped moving body according to a seventh embodiment of the present invention.

FIG. 16 is a perspective view of a camera-mounted moving body according to an eighth embodiment of the present invention.

FIG. 17 is a functional block diagram of a camera-mounted moving body according to a ninth embodiment of the present invention.

FIG. 18 is a perspective view of a camera-mounted moving body according to a tenth embodiment of the present invention.

FIG. 19 is a functional block diagram of a camera-mounted moving body according to a tenth embodiment of the present invention.

[Explanation of symbols]

 1,4,19 CCD camera 2,5,20 Single axis robot 3 Moving unit 6 Control unit 7 Drive motor 8 Wheel 9 Encoder 10 Image processing unit 11,13 Turning mechanism 12,14 Lifting mechanism 15,16 Acceleration sensor 17 Imaging unit Movement control unit 18 Speed sensor 21 Gyro sensor 22 3-axis acceleration sensor 23 3-axis gyro sensor 24 6-degree-of-freedom parallel mechanism 25 6-degree-of-freedom holding table 26 Ball joint 27 Linear actuator

Claims (10)

[Claims] A moving unit; an imaging unit moving unit attached to the moving unit; an imaging unit held by the imaging unit moving unit; a detecting unit for detecting a moving speed of the moving unit; Control means for controlling the imaging unit moving means so that the position and direction of the imaging unit with respect to the environment are the same at a first time and a second time in accordance with the output of the detection means; Moving body. A moving unit; an imaging unit moving unit attached to the moving unit; an imaging unit held by the imaging unit moving unit; a detecting unit for detecting a moving speed of the imaging unit; Control means for controlling the imaging section moving means so that the position and direction of the imaging section with respect to the environment are the same at a first time and a second time in accordance with the output of the means. body. 3. The first control unit controls the imaging unit moving unit to place the imaging unit at the same position and direction with respect to the environment at the first time and the second time.
And performing second control for controlling the imaging unit moving means so as to place the imaging unit at a different position with respect to the environment at the second time and the third time. Item 3. The moving object mounted with a camera according to Item 1 or 2. 4. The camera-mounted moving body further includes an image processing unit that processes an image picked up by the image pickup unit, and the image processing unit uses the image obtained by the first control to generate an environment. Performing a first image processing for extracting a moving object in the moving object, and using a result of the first image processing and an image obtained by the second control to determine a positional relationship between a moving object and a feature point in an environment. 4. The second image processing to be performed is performed.
A moving object equipped with a camera according to item 1. 5. The camera-mounted moving body according to claim 3, wherein the first control and the second control are continuously and repeatedly performed. 6. At least two imaging units, an imaging unit moving unit capable of holding and moving the imaging units, a moving unit to which the imaging unit moving unit is attached, and the moving unit moving Detecting means for detecting a speed; and a position and a direction of at least one of the imaging units with respect to an environment are set to a first time and a second time according to an output of the detecting means.
Control means for controlling the imaging unit moving means so as to be the same at the time of (c). 7. An at least two imaging units, an imaging unit moving unit capable of holding and moving each of the imaging units, a moving unit to which the imaging unit moving unit is attached, and a moving speed of the imaging unit And a position and a direction of at least one of the imaging units with respect to the environment are set at a first time and a second time according to an output of the detection unit.
Control means for controlling the imaging unit moving means so as to be the same at the time of (c). 8. The control unit controls the imaging unit moving unit such that a position and a direction of at least one of the imaging units with respect to an environment are always the same between the first time and the second time. The mobile object mounted with a camera according to claim 6 or 7, wherein 9. The at least two imaging units include a first imaging unit and a second imaging unit, and the camera-mounted moving body further processes an image captured by the at least two imaging units. The image processing unit performs first image processing for extracting a moving object in an environment using an image captured by the first imaging unit, and performs a result of the first image processing and the first image processing. Performing a second image processing for obtaining a positional relationship between the moving object and a feature point in an environment using an image captured by a second imaging unit, wherein the first imaging unit determines the first time and the second time. The position and the direction with respect to the environment are the same at the second time and the second time
9. The camera-equipped moving body according to claim 6, wherein the imaging unit is different from the first imaging unit. 10. The imaging unit according to claim 6, wherein the imaging unit moving unit can independently change a distance between an optical axis of a visual field of the imaging unit and the moving unit. Mobile with camera.