JP7101350B2 - Image processing device and movement support device - Google Patents

Description

The present invention relates to an image processing device and a movement support device.

An image processing device that estimates the state of the front space based on an captured stereo image is known.

There is a need for an image processing device that can be worn by visually impaired people when walking and can easily determine whether or not there are obstacles such as steps, stairs, walls, and objects in front of them.

For example, Patent Document 1 discloses an image processing device for detecting a plane when a robot or an automobile moves autonomously. This image processing device calculates the horizontal plane and the vertical plane in the front visual field of the distance sensor.

Patent Document 2 discloses a step edge estimation device for causing a robot that can move autonomously to move up and down stairs. The present invention estimates the position and direction of the step portion on the premise that there is a step in front of the user.

Patent Document 3 discloses a road surface step detection device that detects a step on a road surface from a plurality of image data captured by a stereo camera.

However, in the case of pedestrians, there are many obstacles to be noted other than the horizontal and vertical planes, such as steps, stairs, and objects at the feet. Neither patent document discloses an image processing device that easily discriminates an obstacle in front of a pedestrian himself.

Therefore, there is a need for an image processing device that can easily determine whether or not there are obstacles such as steps, stairs, walls, and objects in the front space.

Japanese Unexamined Patent Publication No. 10-063210 JP-A-2015-0433117

An object of the present invention is to provide an image processing device for determining the presence or absence of an obstacle in the space in front of a moving body with a simple configuration.

The image processing apparatus according to the present invention has an imaging unit that captures a stereo image in the front space, a position selection unit that selects a plurality of measurement points from a surface captured by the stereo image, and a three-dimensional space of a plurality of measurement points. It is characterized by including a plane equation calculation unit that calculates a plurality of plane equations based on coordinates, and a determination unit that determines the presence or absence of an obstacle in the front space based on the plurality of plane equations.

According to the present invention, the presence or absence of an obstacle in the space in front of the moving body is determined with a simple configuration.

It is (a) schematic block diagram which shows the embodiment of the movement support device which concerns on this invention, (b) is the front perspective view which shows the state which the wearer wears the image pickup part which the said movement support device has. It is a functional block diagram of the said movement support device. It is a schematic diagram which shows the measurement point selected by the position selection part of the image processing apparatus included in the movement support device in the space in front of the wearer. It is a schematic diagram which shows the state of the plane passing through the said measurement point, which was drawn on the assumption that the optical axis of the said image pickup part is horizontal. It is a flowchart of the said movement support device. It is an example of analysis by (a) the state of the front space and (b) the image processing apparatus when moving forward at the place where there is a wall on the front right side. It is (a) the state of the front space when it is rotated to the left in the place where there is a wall on the front left side, and (b) the analysis example by the said image processing apparatus. This is an example of analysis by (a) the state of the front space and (b) the image processing device when there is an obstacle in front of the front. This is an example of analysis by (a) the state of the front space and (b) the image processing device when there is a down staircase in front of the front. It is a schematic diagram which shows the state of the measurement point and the normal vector calculated from the said measurement point when there is a down staircase in front of the front. This is an example of analysis by (a) the state of the front space and (b) the above-mentioned image processing device when there is an ascending staircase in front of the front. It is a schematic diagram which shows the state of the measurement point and the normal vector calculated from the said measurement point when there is an ascending stairs in front of the front. It is a schematic diagram which shows the state of the measurement point and the normal vector calculated from the said measurement point when there is an ascending step in the front front.

Hereinafter, embodiments of the image processing device and the movement support device according to the present invention will be described with reference to the drawings.

● Outline of the movement support device As shown in FIGS. 1 (a) and 1 (b), the movement support device 1 includes a fixed unit 2, an image pickup unit 3, a transmission unit 4, an arithmetic unit 5, a battery 6, and the like. To prepare for. The image pickup unit 3 is fixed to the user by the fixing unit 2 so that an image in the traveling direction of the user can be captured. The image pickup unit 3 and the arithmetic unit 5 constitute an image processing device 10.

The fixing unit 2, the imaging unit 3, and the transmitting unit 4 are configured in a ring shape corresponding to the outer periphery of the head of the user 100. The image pickup unit 3 is a stereo camera composed of a pair of cameras. The image pickup unit 3 is fixed in the vicinity of the left and right temples of the user 100, respectively. The pair of cameras are connected by a resin member on the front side of the user 100 so that the distance between the pair of cameras does not change. The stereo camera 3 is connected by a fixed portion 2 on the back side of the user 100. The fixing portion 2 is, for example, a belt having metal fittings, and its length can be adjusted according to the outer peripheral length of the head of the user 100. Further, the fixing portion 2 may be an elastic belt.

The image pickup unit 3 and the transmission unit 4 are connected to the arithmetic unit 5 by wire or wirelessly, respectively. The arithmetic unit 5 analyzes the image in the traveling direction of the user 100 based on the stereo image captured by the image pickup unit 3, and determines the presence or absence of an obstacle. The arithmetic unit 5 outputs the determined result to the transmission unit 4.

Here, the obstacle is a concept that includes a wide range of objects and obstacles that should be noted when moving, such as steps, stairs, walls, and objects at the feet. The steps include one step up and down, and the stairs include up and down stairs. Obstacles include uphill and downhill slopes. The movement is assumed to be walking when the movement support device 1 is worn and used by a visually impaired person.

The transmission unit 4 receives the result calculated by the arithmetic unit 5, and informs the user 100 of information such as whether or not there is an obstacle in the traveling direction of the user 100 and what kind of obstacle is present. It is a mechanism to transmit. The transmission unit 4 is, for example, bone conduction headphones. In the present embodiment, the transmission unit 4 is fixed so as to come into contact with the head of the user 100 at the connecting portion between one camera of the imaging unit 3 and the fixing unit 2. The transmission unit 4 is not limited to the bone conduction headphones, and may be a device such as an earphone or a speaker that transmits information by voice, or may be a device that transmits information by tactile sensation. Further, various devices such as projection on the retina of the user 100 with a laser can be applied.

The battery 6 supplies power to the image pickup unit 3, the transmission unit 4, and the arithmetic unit 5.

The arithmetic unit 5 and the battery 6 are attached to the waist, chest, and the like of the user 100.

The fixed position of the image pickup unit 3 does not have to be the head as long as the front space of the user 100 can take an image. For example, it may be a shoulder or a chest. Depending on the fixed position of the image pickup unit 3, various shapes such as a neck pillow type and a chest band type can be applied to the shape of the fixed portion 2.

● Configuration of the movement support device 1 As shown in FIG. 2, the movement support device 1 includes an image pickup unit 3, an A / D conversion unit 31, a position selection unit 32, a position calculation unit 33, and a plane equation calculation unit 36. Further, the movement support device 1 includes an image pickup unit angle calculation unit 40 for correcting the inclination of the image pickup unit 3, and a coordinate conversion unit 41. Further, the movement support device 1 includes a flat road determination unit 50, a recording unit 60, and a transmission unit 4.

In the following description, the traveling direction of the user 100 on the horizontal plane is defined as the x-axis direction, the direction orthogonal to the x-axis on the horizontal plane is defined as the y-axis direction, and the direction orthogonal to the horizontal plane is defined as the z-axis direction. This xyz coordinate system is also referred to as a user coordinate system. Further, the front direction in the stereo image is defined as the X-axis direction, the left-right direction orthogonal to the X-axis is defined as the Y-axis direction, and the vertical direction orthogonal to the X-axis and the Y-axis is defined as the X-axis direction. This XYZ coordinate system is also referred to as an image pickup unit coordinate system. The image pickup unit coordinate system is obtained by the arrangement of the pixels of the image pickup unit.

The image pickup unit 3 captures a stereo image of the space in front of the user 100 in the traveling direction. In order to image the road surface at the feet of the user 100, the image pickup unit 3 is fixed so as to face slightly downward from the horizontal.

The A / D conversion unit 31 digitally converts the stereo image captured by the image pickup unit 3.

As shown in FIG. 3, the position selection unit 32 selects a plurality of measurement points from the surface captured by the digitally converted stereo image. The plurality of measurement points are points P0, P1, and P2 (hereinafter, also referred to as "first measurement point group") on the surface close to the feet of the user 100, and on the surface slightly in front of the feet of the user. Points P3, P4, and P5 (hereinafter, also referred to as "second measurement point group"), totaling 6 points.

The first measurement point group and the second measurement point group are points that are not on the same straight line with each other. For example, the point P0 is in front of the user 100, the point P1 is diagonally forward to the right with respect to the point P0, and the point P2 is diagonally forward to the left with respect to the point P0. The point P3 is a position in front of the user 100 and slightly farther from the user 100 than the point P0 in the traveling direction. Point P4 is diagonally forward to the right with respect to point P3, and point P5 is diagonally forward to the left with respect to point P3. The measurement points P4 and P5, which are the farthest from the image pickup unit 3 in the second measurement point group, are located at positions farther forward from the image pickup unit 3 than any of the measurement points in the first measurement point group.

The position calculation unit 33 measures the three-dimensional spatial coordinates of the points P0 to P5 in the image pickup unit coordinate system.

● Angle correction of the imaging unit 3 The optical axis of the imaging unit 3 is fixed so as to face slightly downward from the horizontal. In other words, as shown in FIG. 4, if the horizontal in the stereo image captured by the image pickup unit 3 is interpreted to be the same as the horizontal in the user coordinate system, the road surface appears to stand up diagonally.

Therefore, the image pickup unit angle calculation unit 40 and the coordinate conversion unit 41 calculate the angle of the optical axis of the image pickup unit 3 with respect to the horizontal plane in the user coordinate system, and the measurement points P0 to P5 obtained on the image pickup unit coordinate system. Converts 3D space coordinates to 3D space coordinates on the user coordinate system.

As shown in FIGS. 2 to 4, the imaging unit angle calculation unit 40 obtains a vector V1 connecting the points P0 and P1 and a vector V2 connecting the points P0 and P2, respectively. Then, the image pickup unit angle calculation unit 40 calculates the outer product of the vector V1 and the vector V2, and derives the normal vector V100 of the plane P100 passing through the points P0 to P2. Since the plane P100 is perpendicular to the normal vector V100 and passes through the point P0, the image pickup unit angle calculation unit 40 can obtain the plane equation of the plane P100 from the normal vector V100 and the point P0.

The equation of a plane of the plane P100 having the normal vector V100 of the components (A, B, C) is expressed as follows.
AX + BY + CZ + D = 0 (1)
X, Y, and Z are variables, and A, B, C, and D are constants.

The normal vector V100 may be derived by the plane equation calculation unit 36, which will be described later, and the image pickup unit angle calculation unit 40 may receive the information of the normal vector V100 from the plane equation calculation unit 36.

（２）

は、法線ベクトルＶ１００の長さである。 The image pickup unit angle calculation unit 40 calculates the inclination angle A100 of the normal vector V100 of the plane P100 in the xz axis space by the equation (2).

(2)

Is the length of the normal vector V100.

The coordinate conversion unit 41 uses the angle A100 as the angle of the optical axis of the image pickup unit 3 to convert the three-dimensional spatial coordinates of the points P0 to P5 obtained on the image pickup unit coordinate system into the user coordinate system. Assuming that the three-dimensional spatial coordinates obtained on the image pickup unit coordinate system of the point Pn (n is a natural number starting from 0) are (Xn, Yn, Zn), the three-dimensional spatial coordinates of the point Pn in the user coordinate system (xn, yn and Zn) are obtained by the following equations (3) to (5), respectively. In this embodiment, n is a natural number from 0 to 5.
xn = Xn × cos (-π / 2 + angle A100) -Zn × sin (-π / 2 + angle A100) (3)
Zn = Zn × sin (-π / 2 + angle A100) + Zn × cos (-π / 2 + angle A100) (4)
yn = Yn (5)
The unit of the angle A100 is a radian. From the equations (3) to (5), the three-dimensional spatial coordinates of the points P0 to P5 in the user coordinate system can be obtained.

The normal vector V100 may be slightly tilted in the yz-axis space on the actual road surface. Since the influence of the inclination of the yz-axis space is small, the inclination of the yz-axis space is ignored in the present embodiment. However, the image pickup unit angle calculation unit 40 may calculate the inclination of the yz axis space. Further, the coordinate conversion unit 41 may convert the three-dimensional space coordinates of the measurement points P0 to P5 into the user coordinate system based on the inclination of the yz axis space.

As shown in FIG. 3, the equation of a plane calculation unit 36 derives a plane equation representing a plane passing through the first measurement point group and a plane equation representing a plane passing through the second measurement point group, respectively. Specifically, the plane equation calculation unit 36 obtains a vector v1 connecting the points P0 and P1 converted into the user coordinate system and a vector v2 connecting the points P0 and P2 converted into the user coordinate system. .. Then, the plane equation calculation unit 36 calculates the outer product of the vector v1 and the vector v2, and derives the normal vector V10 of the plane P10 passing through the points P0 to P2.

Further, the plane equation calculation unit 36 obtains a vector v3 connecting the points P3 and P4 converted into the user coordinate system and a vector v4 connecting the points P3 and P5 converted into the user coordinate system. The plane equation calculation unit 36 calculates the outer product of the vector v3 and the vector v4, and derives the normal vector V11 of the plane P11 passing through the points P3 to P5.

The equation of a plane of the plane P10 having the normal vector V10 is expressed as follows.
ax + by + cz + d = 0 (6)
x, y, and z are variables, and a, b, c, and d are constants.

● Flat road determination unit 50
As shown in FIG. 2, the flat road determination unit 50 includes a plane height calculation unit 52, a plane angle calculation unit 53, and a determination unit 70.

The plane height calculation unit 52 calculates the z component of the first measurement point group points P0 to P2, that is, the average height, and calculates the height Z10. Further, the plane height calculation unit 52 calculates the z component of the second measurement point group points P3 to P5, that is, the average height, and calculates the height Z11.

The plane angle calculation unit 53 calculates the slopes of the plane P10 and the plane P11 based on the components of the normal vector V10 of the plane P10 and the normal vector V11 of the plane P11. First, the plane angle calculation unit 53 normalizes the normal vectors V10 and V11, respectively, and calculates the normalized normal vectors V20 and V21, respectively.

Assuming that the components of the normalized normal vector V20 are (a20, b20, c20), the tilt angle A10 of the vector V10 with respect to the traveling direction can be obtained by the equation (6).
A10 = π / 2-acos (a20) (6)
The tilt angle B10 of the vector V10 with respect to the left-right direction is obtained by the equation (7).
B10 = π / 2-acos (b20) (7)

Similarly, the plane angle calculation unit 53 calculates the tilt angle A11 with respect to the traveling direction of the equation vector V11 and the tilt angle B11 with respect to the left-right direction using the equations (6) and (7).

The recording unit 60 records the heights Z10 and Z11 calculated in the past, the tilt angles A10 and A11 with respect to the traveling direction of the planes P10 and P11, and the tilt angles B10 and B11 with respect to the left-right direction. The recording unit 60 can record a plurality of each value calculated in the past. The recording unit 60 transmits each value to the determination unit 70 in response to a call from the determination unit 70.

The recording unit 60 may store all the values since the start of measurement, or record a plurality of the latest values one by one, and when receiving new data, the oldest recorded value is used. You may want to delete the data.

The determination unit 70 determines the presence or absence of an obstacle in the space in front of the user 100 based on the heights Z10 and Z11 and the tilt angles A10, A11, B10, and B11 of the planes P10 and P11.

The determination unit 70 may store a predetermined threshold value for each value, and may determine that there is an obstacle in the front space when each value currently calculated exceeds the threshold value. The determination unit 70 may receive each value calculated in the past from the recording unit 60 and determine the presence / absence of an obstacle or the like based on each value calculated in the past and each value currently calculated. Specifically, it may be determined that there is an obstacle in the front space when the threshold value is continuously exceeded in each of the recently calculated multiple times value and each value currently calculated.

● Flow chart of the movement support device 1 The operation of the movement support device 1 described so far will be described. As shown in FIG. 5, first, the movement support device 1 takes an image of the space in front of the user 100 by the image pickup unit 3 and obtains a stereo image (step S1). Then, the movement support device 1 A / D-converts the captured image by the A / D conversion unit 31 (step S2).

Next, the position selection unit 32 selects a plurality of measurement points from the surface of the A / D converted image (step S3). The plurality of measurement points are a total of 6 points, that is, points P0, P1, and P2 on the surface near the feet of the user 100, and points P3, P4, and P5 on the surface slightly in front of the feet of the user. Is.

Next, the position calculation unit 33 calculates the three-dimensional spatial coordinates of the plurality of measurement points P0 to P5 in the image pickup unit coordinate system (step S4).

The image pickup unit angle calculation unit 40 calculates the inclination of the optical axis of the image pickup unit 3 from the three-dimensional spatial coordinates of the first measurement point group points P0 to P2 (step S5). Further, the recording unit 60 records the calculated inclination of the optical axis of the imaging unit 3 (step S61).

The image pickup unit angle calculation unit 40 reads the tilt of the optical axis of the image pickup unit 3 calculated in the past from the recording unit 60, and based on the tilt of the optical axis calculated in the past and the tilt of the optical axis currently calculated. The inclination of the optical axis of the image pickup unit 3 may be calculated. If the currently calculated tilt of the optical axis is too large, the value calculated in the past instead of the currently calculated value may be transmitted to the coordinate conversion unit 41 as the tilt of the optical axis of the imaging unit 3. good. Further, the image pickup unit angle calculation unit 40 may calculate the inclination of the optical axis of the current image pickup unit 3 based on a plurality of values calculated in the past. By using the tilt of the optical axis calculated in the past for the calculation, even if the space under the feet of the user 100 is not flat in a narrow space such as a staircase or while going up and down the stairs, the front It is possible to appropriately determine the presence or absence of obstacles in the space.

The coordinate conversion unit 41 converts the three-dimensional space coordinates of the points P0 to P5 in the image pickup unit coordinate system into the three-dimensional space coordinates in the user coordinate system based on the inclination of the optical axis of the image pickup unit 3 (step S6). ).

The equation of a plane calculation unit 36 calculates the normal vector V10 of the plane P10 passing through the points P0, P1 and P2 (step S7). Further, the equation calculation unit 36 calculates the normal vector V11 of the plane P11 passing through the points P3, P4, and P5 (step S8).

The plane height calculation unit 52 calculates the height Z10 of the first measurement point cloud (step S9). Further, the plane height calculation unit 52 calculates the height Z11 of the second measurement point group (step S10). The recording unit records heights Z10 and Z11 (step S62).

The plane angle calculation unit 53 calculates the tilt angle A10 in the traveling direction and the tilt angle B10 in the left-right direction of the plane P10 with respect to the horizontal (step S11). Further, the plane angle calculation unit 53 calculates the tilt angle A11 of the plane P11 in the traveling direction with respect to the horizontal and the tilt angle B11 in the left-right direction (step S12). The recording unit records the calculated tilt angles A10, A11, B10, and B11 of the plane P10 and the plane P11, respectively (step S63).

The determination unit 70 calls the information of the heights Z10, Z11 and the tilt angles A10, A11, B10, and B11 calculated in the past from the recording unit 60, respectively (step S13). The determination unit 70 determines the presence or absence of an obstacle in the space in front of the user 100 based on the height and tilt angle calculated in the past and the height and tilt angle currently calculated (step S14). Further, the recording unit 60 records the determination result (step S64).

It is assumed that the movement support device 1 is worn by, for example, a walking user and sequentially determines the presence or absence of an obstacle in front of the user. Therefore, the determination unit 70 can capture how the distance between the user and the obstacle gradually changes as he walks by using each value calculated in the past for the determination. Therefore, the accuracy of determining obstacles is improved.

When the determination unit 70 determines that it is necessary to transmit to the user based on the result of the determination, the determination unit 70 transmits to that effect to the transmission unit 4. The transmission unit 4 transmits the determination result to the user by various methods (step S15). If the determination unit 70 determines that it is not necessary to transmit to the user, the determination unit 70 does not transmit information to the transmission unit 4.

The transmission unit 4 may notify the user only when there is an obstacle, or may give a notification notifying that the user can walk safely when there is no obstacle.

Finally, the movement support device 1 returns to step S1 and periodically repeats the operations of steps S1 to S15 and steps S61 to S63.

In this way, the movement support device 1 can determine the presence or absence of obstacles around the user 100 with a simple configuration by analyzing the surface of the space at the feet and the space slightly in front of the feet. ..

● Measurement example 1: When there is a wall in front of the right side in the traveling direction of the user As shown in FIG. 6A, when the wall 200 exists slightly in front of the feet of the user 100, the points of the second measurement point group. P3 and point P5 are on the floor, and point P4 rests on the wall 200. Therefore, the plane P11 is measured tilted from the upper right to the lower left.

FIG. 6B shows the time series change of each value when the user 100 approaches the wall diagonally. In this measurement example, a total of 16 values are shown, each of which is calculated 15 times in the past and the value calculated now. That is, each value up to frame -15 is plotted with the time point of the currently calculated value as frame 0 and the time point of the value calculated one time before as frame-1.

In each of the following measurement examples, the measurement frequency of the movement support device 1 is 10 Hz. That is, the movement support device 1 makes a measurement once every 100 ms and analyzes the state of the front space. In other words, the time interval between frames is 100 ms. The measurement frequency is arbitrary, and the more frequently the measurement is, the higher the accuracy is, but the processing load of the arithmetic unit 5 of the movement support device 1 is increased. When a person wears it and uses it, considering that the walking speed of the person is about 1 m / s, a measurement of about 10 Hz is suitable for analyzing the change in the front space every about 0.1 m. According to such a movement support device 1, false detection due to momentary fluctuation can be reduced, and the front space of the moving user 100 can be determined more accurately.

In the figure, the tilt angle B11 in the left-right direction of the plane P11 rises from around the frame-10 to around 50 °. Further, the tilt angle A11 in the traveling direction of the plane P11 is lowered to the vicinity of −40 °. Further, as the user 100 progresses, the tilt angle B10 in the left-right direction of the plane P10 also gradually rises, reflecting that the point P1 also rides on the wall 200. From this information, the determination unit 70 can determine that a plane inclined diagonally to the right with respect to the traveling direction exists in front. In this case, the determination unit 70 determines that there is a wall in front of the right side, and notifies the user through the transmission unit 4.

● Measurement example 2: When there is a wall in front of the user in the rotation direction As shown in FIG. 7A, when there is a wall in front of the left side of the user 100, the user 100 rotates to the left near the wall. If so, in the second measurement point group, the point P5 is on the floor surface, and the points P3 and P4 ride on the wall 201. Further, as the user 100 rotates, the point P2 of the first measurement point group also rides on the wall 201.

As shown in FIG. 7B, the tilt angles of the plane P10 and the plane P11 gradually increase in the minus direction from the vicinity of the frame-9 in both the traveling direction and the left-right direction. From this, the determination unit 70 determines that an obstacle has appeared in front as the user 100 rotates, and transmits that fact to the user 100 through the transmission unit 4.

● Measurement example 3: When there is an obstacle in front of the user in the traveling direction As shown in FIG. 8A, when the user 100 walks toward the obstacle 202, points P4 and P5 are obstacles almost at the same time. Get on the thing 202.

As shown in FIG. 8B, the tilt angle B11 in the left-right direction of the plane P11 hardly changes, but the tilt angle A11 in the traveling direction suddenly changes from the frame-11 in the minus direction to -50 °. It's down to the vicinity. From this, the determination unit 70 determines that there is an obstacle in front of the user 100 in the traveling direction, and transmits that fact to the user 100 through the transmission unit 4.

● Measurement example 4: When there is a down staircase in front of the user's traveling direction As shown in Fig. 9 (a), when the user 100 approaches the front of the down staircase, the plane P11 is placed on the down staircase. Will be done. As shown in FIG. 9B, the tilt angle A11 of the plane P11 increases in the positive direction from the frame-13. The tilt angle A10 of the plane P10 remains almost constant. Further, the plane height Z11 of the plane P11 changes so as to be located below the plane height Z10 of the plane P10. From these facts, the determination unit 70 determines that a down staircase or a down slope has appeared in front of the user 100 in the traveling direction. The tilt angle A11 of the plane P11 in the traveling direction is about 20 ° when the plane P11 is approaching the stairs. As shown in FIG. 10, since this value corresponds to the slope of the stairs, it can be seen that the image processing device 10 can analyze the state of the down stairs.

● Measurement example 5: When there is an ascending staircase in front of the user in the traveling direction As shown in FIG. 11A, when the user 100 approaches the front of the ascending staircase, the plane P11 is arranged on the ascending staircase. Will be done. As shown in FIG. 11B, the plane P11 tilt angle A11 increases from the vicinity of the frame-10 to the − side. Further, the plane height of the plane P11 has begun to change so as to be located above the plane P10. From these facts, the determination unit 70 determines that an ascending staircase or an ascending slope has appeared in front of the user 100 in the traveling direction. The tilt angle A11 of the plane P11 in the traveling direction is about −20 ° when the plane P11 is approaching the stairs. As shown in FIG. 12, since this value corresponds to the slope of the stairs, it can be seen that the image processing device 10 can analyze the state of the ascending stairs.

● Measurement example 6: When there is a step in front of the user in the traveling direction As shown in FIG. 13, for example, when there is a step between the floor surface of the entrance and the soil, the tilt angles A10, A11, B10, and B11 are small. However, the heights Z10 and Z11 are different from each other. Therefore, the determination unit 70 compares the plane heights Z10 and Z11 and determines whether or not there is a step in the front. This is the same for the up step and the down step.

As described above, the image processing device 10 can determine the presence or absence of an obstacle in the space in front of the moving body with a simple configuration. Further, the movement support device 1 provided with the image processing device 10 can transmit to the user 100 who wears the image processing device 10 whether or not there is an obstacle in the space in front of the moving body.

In explaining the movement support device according to the present invention, the device attached to a human being and used is assumed. However, the technical idea of the present invention can also be applied to, for example, a mechanism unit that is attached to a robot or incorporated into a robot to determine the presence or absence of an obstacle.

1 Movement support device 3 Image pickup unit 10 Image processing device 32 Position selection unit 36 Plane equation calculation unit 70 Judgment unit

Claims (10)

Direction of travel of the moving body ForwardofAn image pickup unit that captures a stereo image of space,
A position selection unit that selects a plurality of measurement points from the stereo image, and
A plane equation calculation unit that calculates a plurality of plane equations based on the three-dimensional spatial coordinates of the plurality of measurement points.
Based on the multiple equations of a plane,Direction of travelForwardofA judgment unit that determines the presence or absence of obstacles in space,
Equipped with,
The plurality of measurement points include a first point, a second point diagonally forward to the right in the traveling direction with respect to the first point, a third point diagonally forward to the left with respect to the first point, and the first point. A fourth point diagonally forward in the traveling direction by a predetermined distance ahead of the point, a fifth point diagonally forward to the right with respect to the fourth point, and a sixth point diagonally forward to the left with respect to the fourth point. And, at least, including
The plane equation calculation unit calculates the first plane equation based on the three-dimensional spatial coordinates of the first measurement point group, with the first point, the second point, and the third point as the first measurement point group. At the same time, the second plane equation is calculated based on the three-dimensional spatial coordinates of the second measurement point group, with the fourth point, the fifth point, and the sixth point as the second measurement point group.
The determination unit determines the presence or absence of an obstacle in the space in front of the traveling direction based on the first equation of a plane and the second equation of a plane.
An image processing device characterized by this. The determination unit calculates the inclination of the plane represented by the plurality of plane equations from the plurality of plane equations, and determines the presence or absence of an obstacle in the space in front based on the inclination of each of the planes. 1 The image processing apparatus according to 1. The determination unit calculates the inclination of the plane represented by the plurality of plane equations in the front-rear direction and the left-right direction orthogonal to the front-back direction from the plurality of plane equations, and the inclination of the plane in the front-back direction and the left-right direction. The image processing apparatus according to claim 1 or 2, wherein the type of an obstacle in the space in front of the object is determined based on the inclination of a plane in a direction. An image pickup unit that captures a stereo image of the front space,
A position selection unit that selects a plurality of measurement points from the stereo image, and
A plane equation calculation unit that calculates a plurality of plane equations based on the three-dimensional spatial coordinates of the plurality of measurement points.
A determination unit that determines the presence or absence of an obstacle in the front space based on the plurality of equations of a plane.
Equipped with
The determination unit calculates the inclination of the plane represented by the plurality of plane equations in the front-rear direction and the left-right direction orthogonal to the front-back direction from the plurality of plane equations, and the inclination of the plane in the front-back direction and the left-right direction. Determining the type of obstacle in the front space based on the inclination of the plane in the direction,
An image processing device characterized by this. The plurality of plane equations are composed of a first plane equation and a second plane equation, and the plurality of measurement points are a first measurement point group consisting of a plurality of measurement points and a plurality of measurements different from the first measurement point group. The plane equation calculation unit is composed of a second measurement point group consisting of points, calculates the first plane equation representing a plane passing through the first measurement point group, and represents the plane passing through the second measurement point group. The image processing apparatus according to claim 4 , which calculates a second plane equation. The fifth aspect of the present invention, wherein the measurement point farthest from the image pickup unit in the second measurement point group is located at a position farther forward from the image pickup unit than any measurement point in the first measurement point group. Image processing device. The determination unit calculates the average height of the first measurement point group and the second measurement point group, respectively, and determines the presence or absence of an obstacle in the space in front based on the average height. The image processing apparatus according to any one of 2, 3, 5 and 6 . The determination unit further includes a recording unit that records the inclination of the plane calculated in the past, and the determination unit is an obstacle in the space in front based on the inclination of the plane calculated in the past and the inclination of the plane currently calculated. The image processing apparatus according to any one of claims 1 to 7 , which determines the presence or absence of an object. Based on the plane equation, the image pickup unit angle calculation unit that calculates the angle of the image pickup unit with respect to the plane represented by the plane equation as the image pickup unit angle, and the coordinate conversion that corrects the inclination of the stereo image based on the image pickup unit angle. The image processing apparatus according to any one of claims 1 to 8 , further comprising a unit. An image processing device that is configured to be fixed to a moving body and has an image pickup unit that captures a stereo image of the space in front of the moving body.
A transmission unit that transmits the processing result of the image processing device to the moving body, and
It is a movement support device equipped with
The image processing device is a movement support device, which is the image processing device according to any one of claims 1 to 9 .

Images