JP2016131367A - Moving body system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten the time required until information on the surroundings of a moving body is acquired.SOLUTION: A moving body system comprises: an imaging apparatus attached to a moving body; a sight line direction changing mechanism for changing the sight line direction of the imaging apparatus; and a controller for controlling the sight line direction of the imaging apparatus in accordance with a change in the traveling direction of the moving body.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a mobile system.

  2. Description of the Related Art Conventionally, a mobile body system that acquires information around a mobile body by attaching a stereo camera to the mobile body is known.

  As a mobile system, for example, a configuration is disclosed in which a plurality of stereo cameras are fixed to a vehicle so as to photograph the same direction or different directions (see, for example, Patent Document 1).

  However, in a state in which the stereo camera is fixed in a certain direction, when the moving body changes its traveling direction to, for example, the left direction and the right direction, it may take a long time to acquire information around the moving body.

  Accordingly, an object of the present invention is to shorten the time required to acquire information around a moving object.

  In one proposal, an imaging device attached to a moving body, a gaze direction changing mechanism that changes a gaze direction of the imaging device, and a gaze direction of the imaging device is controlled according to a change in a traveling direction of the moving body. There is provided a mobile system having a control unit.

  According to one aspect, it is possible to shorten the time required to acquire information around the mobile object.

It is a schematic block diagram (the 1) of the mobile body system which concerns on 1st Embodiment of this invention. It is a control block diagram of the mobile body system which concerns on 1st Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the stereo camera which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the mobile body system which concerns on 1st Embodiment of this invention. It is a table which illustrates the relationship between a steering angle and the viewing direction of a stereo camera. It is a flowchart which shows the other example of operation | movement of the mobile body system which concerns on 1st Embodiment of this invention. It is a figure for demonstrating operation | movement of a mobile body system when a vehicle is steered to the left direction when the advancing direction of a vehicle is a front direction. It is a figure for demonstrating the relationship between the advancing direction of a vehicle, and the gaze direction of a stereo camera. It is a figure for demonstrating operation | movement of a mobile body system when a vehicle is steered rightward when the advancing direction of a vehicle is a front direction. It is a schematic block diagram (the 2) of the mobile body system which concerns on 1st Embodiment of this invention. It is a figure for demonstrating operation | movement of a mobile body system when a vehicle is steered leftward when the advancing direction of a vehicle is a back direction. It is a figure for demonstrating operation | movement of a mobile body system when a vehicle is steered rightward when the advancing direction of a vehicle is a backward direction. It is a block diagram which shows the hardware constitutions of the stereo camera which concerns on 2nd Embodiment of this invention. It is a control block diagram of the mobile body system which concerns on 2nd Embodiment of this invention. It is a control block diagram of the mobile body system which concerns on 3rd Embodiment of this invention. It is a figure for demonstrating the principle of ranging by a stereo camera. It is a figure for demonstrating the process process for producing | generating a parallax image in a stereo camera. It is a figure for demonstrating the image matching by a parallax search. It is the graph which plotted the dissimilarity calculated by image matching according to the amount of parallax.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has substantially the same function structure, the duplicate description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  A moving body system according to an embodiment of the present invention includes an imaging device attached to a moving body, a line-of-sight direction changing mechanism that changes a line-of-sight direction of the imaging device, and an imaging device according to a change in the traveling direction of the moving body. And a control unit for controlling the viewing direction.

  Hereinafter, although the mobile body system at the time of using a vehicle as an example of a mobile body is demonstrated, this invention is not limited in this point.

[First Embodiment]
(Configuration of mobile system)
A mobile system according to a first embodiment of the present invention will be described. FIG. 1 is a schematic configuration diagram (part 1) of a mobile system according to the first embodiment of the present invention. FIG. 2 is a control block diagram of the mobile system according to the first embodiment of the present invention. Note that the arrows in FIG. 1 indicate the traveling direction of the vehicle.

  As shown in FIGS. 1 and 2, the mobile system includes a stereo camera 100, a rotation mechanism 200, an ECU (Electronic Control Unit) 300, and a rotation mechanism control unit 400.

  Stereo camera 100 is an example of an imaging apparatus. The stereo camera 100 includes at least two cameras having different viewpoints, and acquires information around the vehicle 500 by imaging a person or an object to be imaged by these cameras. Specifically, the stereo camera 100 measures the distance between the vehicle 500 and a person or an object approaching from the surroundings of the vehicle 500, for example. The stereo camera 100 is attached on the ceiling (roof) of the vehicle 500 via, for example, a rotation mechanism 200, and rotates together with the rotation mechanism 200 when the rotation mechanism 200 rotates.

  In FIG. 1, the stereo camera 100 is used as an example of the imaging device, but the present invention is not limited in this respect, and a monocular camera may be used instead of the stereo camera 100. In this case, in order to obtain a wide angle of view, it may be configured to measure the distance to a person or an object by arranging a large number of monocular cameras, or to measure the distance to a person or an object using the phase shift method. Good.

  In FIG. 1, the stereo camera 100 is mounted on the ceiling of the vehicle 500, but the present invention is not limited in this respect, and may be mounted on the hood of the vehicle 500, for example.

  A hardware configuration of the stereo camera 100 will be described. FIG. 3 is a block diagram showing a hardware configuration of the stereo camera 100 according to the first embodiment of the present invention.

  As illustrated in FIG. 3, the stereo camera 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an imaging unit 104, and an image processing unit 105. Prepare. Each unit of the stereo camera 100 is connected via a bus 106.

  The CPU 101 controls the stereo camera 100 by executing a program stored in the ROM 102 in the RAM 103 as a work memory.

  The ROM 102 is a non-volatile memory and is connected to the bus 106 via a ROM interface (I / F) 107. The ROM 102 stores programs executed by the CPU 101, data, and the like.

  The RAM 103 is a main storage device such as a DRAM (Dynamic Random Access Memory) and an SRAM (Static Random Access Memory). The RAM 103 is connected to the bus 106 via a RAM interface (I / F) 108. The RAM 103 functions as a work area that is expanded when various programs are executed by the CPU 101.

  The imaging unit 104 is a camera that images an object, and is connected to the image processing unit 105. An image (luminance image) captured by the imaging unit 104 is appropriately subjected to image processing by the image processing unit 105. The imaging unit 104 is connected to the bus 106 via an imaging unit control interface (I / F) 109.

  The image processing unit 105 calculates a parallax based on the luminance image of the object imaged by the imaging unit 104, calculates a distance from the stereo camera 100 to a person or an object based on the calculated parallax, and generates a parallax image. To do. Further, the image processing unit 105 recognizes a person or an object based on the generated parallax image and luminance image. A method of calculating a distance to a person or an object (hereinafter also referred to as “ranging”) by the image processing unit 105 will be described later. In addition, since parallax is a function about distance, it demonstrates as one aspect | mode of distance information.

  The rotation mechanism 200 is an example of a line-of-sight direction changing mechanism. The rotation mechanism 200 is a mechanism that can freely change the viewing direction of the stereo camera 100. In FIG. 1, the rotation mechanism 200 is used as an example of the line-of-sight direction changing mechanism, but is not particularly limited as long as the line-of-sight direction of the stereo camera 100 can be freely changed.

  ECU 300 is an example of a control unit. ECU 300 controls the line-of-sight direction of stereo camera 100 based on information indicating the traveling direction of vehicle 500. As a result, ECU 300 controls the line-of-sight direction of stereo camera 100 in accordance with the change in the traveling direction of vehicle 500. As information indicating the traveling direction of the vehicle 500, information related to steering such as a steering angle, a steering speed, and a wheel inclination angle can be used. Specifically, the ECU 300 acquires (detects) the steering steering angle of the vehicle 500, for example, and controls the line-of-sight direction of the stereo camera 100 based on the acquired change in the steering steering angle. The ECU 300 is communicably connected to the rotating mechanism control unit 400 and other ECUs mounted on the vehicle 500 via an in-vehicle network such as a CAN (Controller Area Network) and a LIN (Local Interconnect Network).

  The rotation mechanism control unit 400 is connected to the in-vehicle network, and can communicate with the ECU 300, for example. The rotation mechanism control unit 400 controls the operation of the rotation mechanism 200 so that the rotation angle of the rotation mechanism 200 becomes an angle designated by the ECU 300.

(Mobile system operation)
An example of the operation of the mobile system will be described.

  As shown in FIG. 2, ECU 300 determines the rotation angle of rotation mechanism 200 with respect to rotation mechanism control unit 400 based on information indicating the traveling direction of vehicle 500, and rotates the rotation mechanism via rotation mechanism control unit 400. 200 operations are controlled. When the rotation mechanism 200 rotates, the stereo camera 100 attached to the rotation mechanism 200 rotates together with the rotation mechanism 200, so that the line-of-sight direction of the stereo camera 100 is changed. As a result, it is possible to shorten the time until the information around the vehicle 500 is acquired. As a result, a collision between the vehicle 500 and a person or an object can be prevented.

  The ECU 300 preferably determines the rotation angle of the rotation mechanism 200 so as to change the angle formed by the line-of-sight direction of the stereo camera 100 and the traveling direction of the vehicle 500 in accordance with the amount of change in the traveling direction of the vehicle 500. In particular, ECU 300 preferably determines the rotation angle of rotation mechanism 200 so that the angle formed by the line-of-sight direction of stereo camera 100 and the traveling direction of vehicle 500 increases as the amount of change in the traveling direction of vehicle 500 increases. . Thereby, even when the driver changes the traveling direction of the vehicle 500 greatly in a short time, the vehicle periphery information can be quickly acquired.

  The ECU 300 preferably determines the rotation angle of the rotation mechanism 200 so as to be in a direction with an angle inside the turning direction of the vehicle 500. As a result, the stereo camera 100 faces inwardly at an extra angle with respect to the traveling direction of the vehicle 500, so that the presence of an object outside the driver's field of view can be imaged and recognized particularly quickly.

  ECU 300 preferably uses vehicle speed information in addition to information indicating the traveling direction of vehicle 500 as information used to change the line-of-sight direction of stereo camera 100. Specifically, ECU 300 determines the rotation angle of rotation mechanism 200 so that the rotation angle of rotation mechanism 200 when the vehicle speed is high is larger than the rotation angle of rotation mechanism 200 when the vehicle speed is low. Is preferred. As a result, when the vehicle speed is high, the amount of angle that needs to be added to the direction of movement can be increased, and when the vehicle speed is low, the amount of movement is excessive for the direction of movement. The amount of angle that needs to be angled can be kept small.

  Next, an example of control performed by the ECU 300 will be described. FIG. 4 is a flowchart showing an example of the operation of the mobile system according to the first embodiment of the present invention.

  As shown in FIG. 4, first, the ECU 300 determines whether or not the direction indicator (blinker) is turned on (flashing) (step S11).

  When it is determined in step S11 that the direction indicator is turned on, ECU 300 acquires the steering angle of vehicle 500 (step S12). If it is determined in step S11 that the direction indicator is not turned on, step S11 is repeated.

  Next, ECU 300 determines whether or not the steering angle obtained in step S12 is greater than or equal to a predetermined angle (step S13).

  If it is determined in step S13 that the steering angle is equal to or greater than the predetermined angle, the ECU 300 calculates the line-of-sight direction (angle) of the stereo camera 100 as a control value using the following equation 1 (step S14).

θ = A × H [Formula 1]
In Equation 1, θ is the viewing direction of the stereo camera 100, H is the steering angle, A is a constant corresponding to the vehicle type, camera installation position, camera angle of view, and the like. If it is determined in step S13 that the steering angle is smaller than the predetermined angle, the process returns to step S11.

  Next, the ECU 300 controls the stereo camera 100 using the control value calculated in step S14 so that the line-of-sight direction of the stereo camera 100 changes by the calculated angle (step S15). Then, it returns to step S11.

  In FIG. 4, the ECU 300 acquires the steering angle of the vehicle 500 when it is determined that the direction indicator is on, but the ECU 300 does not determine whether the direction indicator is on or off. May acquire the steering angle of the vehicle 500. However, it is preferable to determine whether the direction indicator is on or off from the viewpoint that the line-of-sight direction θ of the stereo camera 100 can be suppressed from being frequently changed when the vehicle 500 changes lanes.

  In Equation 1, the line-of-sight direction θ of the stereo camera 100 is a linear function of the steering steering angle H, but is not limited to this. For example, a high-order function such as a quadratic function or a cubic function of the steering steering angle H is used. It may be.

  Further, the ECU 300 controls the stereo camera 100 based on a table indicating a relationship between a steering steering angle H and a line-of-sight direction θ of the stereo camera 100 that are determined in advance according to the vehicle type, camera installation position, camera angle of view, and the like. May be. FIG. 5 is a table illustrating the relationship between the steering angle H and the viewing direction θ of the stereo camera 100.

  In the table shown in FIG. 5A, when the steering angle H is equal to or smaller than a predetermined threshold T, the line-of-sight direction θ of the stereo camera 100 is controlled to 0 degree. When the steering angle H is larger than the predetermined threshold T, the line-of-sight direction θ of the stereo camera 100 is controlled to 90 degrees.

  In the table shown in FIG. 5B, when the steering angle H is equal to or smaller than the first threshold T1, the line-of-sight direction θ of the stereo camera 100 is controlled to 0 degree. When the steering angle H is greater than the first threshold T1 and equal to or less than the second threshold T2, the line-of-sight direction θ of the stereo camera 100 is controlled to an angle that depends on θ. Further, when the steering angle H is larger than the second threshold T2, the line-of-sight direction θ of the stereo camera 100 is controlled to 90 degrees. Note that the angle dependent on θ is, for example, an angle defined by the above-described linear expression or quadratic expression for θ.

  Next, another example of control performed by the ECU 300 will be described. FIG. 6 is a flowchart showing another example of the operation of the mobile system according to the first embodiment of the present invention.

  As shown in FIG. 6, first, the ECU 300 determines whether or not the direction indicator is turned on (step S21).

  If it is determined in step S21 that the direction indicator is turned on, the ECU 300 controls the stereo camera 100 so that the line-of-sight direction of the stereo camera 100 is changed by a predetermined angle in a direction corresponding to the direction indicator. Control (step S22). For example, when the direction indicator points to the left direction, the ECU 300 controls the stereo camera 100 so that the line-of-sight direction of the stereo camera 100 changes by a predetermined angle in the left direction. If it is determined in step S21 that the direction indicator is not turned on, step S21 is repeated.

  Next, the ECU 300 acquires the steering angle of the vehicle 500 (step S23).

  Next, the ECU 300 determines whether or not the steering angle acquired in step S23 is greater than or equal to a predetermined angle (step S24).

  When it is determined in step S24 that the steering angle is equal to or greater than the predetermined angle, the ECU 300 calculates the line-of-sight direction of the stereo camera 100 as a control value using the following equation 2 (step S25).

θ = A × H + W [Formula 2]
In Equation 2, θ is the line-of-sight direction of the stereo camera 100, H is the steering angle, A is a constant corresponding to the vehicle type, camera installation position, camera angle of view, etc. W is a stereo that is changed when the direction indicator is turned on. This is the angle of the camera 100. If it is determined in step S24 that the steering angle is smaller than the predetermined angle, the process returns to step S21.

  Next, the ECU 300 uses the control value calculated in step S25 to control the stereo camera 100 so that the line-of-sight direction of the stereo camera 100 changes by the calculated angle (step S26). Then, it returns to step S21.

  In FIG. 6, when the direction indicator is turned on by the driver, the line-of-sight direction of the stereo camera 100 is controlled so as to face the direction corresponding to the direction indicator. Thereby, before the driver operates the steering (steering wheel), information on the direction in which the vehicle 500 is scheduled to travel can be acquired, so that the involvement of a person, a motorcycle, or the like can be particularly suppressed.

  In Equation 2, the line-of-sight direction θ of the stereo camera 100 is a linear function of the steering steering angle H, but is not limited to this, for example, a high-order function such as a quadratic function or a cubic function of the steering steering angle H. It may be.

  Similarly to the case of FIG. 4, the ECU 300 is based on a table indicating the relationship between the steering angle H determined in advance according to the vehicle type, the camera installation position, the camera angle of view, and the line-of-sight direction θ of the stereo camera 100. Thus, the stereo camera 100 may be controlled.

  Hereinafter, specific control by the ECU 300 will be described by taking as an example a case where the vehicle 500 turns to the left when the traveling direction of the vehicle 500 is the forward direction.

  FIG. 7 is a diagram for explaining the operation of the mobile system when the vehicle 500 is steered leftward when the traveling direction of the vehicle 500 is the forward direction. When the traveling direction of the vehicle 500 is the forward direction and the vehicle 500 turns to the left (indicated by “arrow FL” in FIG. 7), the ECU 300 determines whether the stereo camera 100 changes in accordance with the change in the traveling direction of the vehicle 500. Control gaze direction.

  Hereinafter, the relationship between the traveling direction of the vehicle 500 and the line-of-sight direction of the stereo camera 100 at each point when the traveling direction of the vehicle 500 changes from the front direction to the left direction will be described.

  FIG. 8 is a diagram for explaining the relationship between the traveling direction of the vehicle 500 and the viewing direction of the stereo camera 100. FIG. 8A shows the trajectory of the vehicle 500 when the vehicle 500 turns to the left and points a, b, c, d, and e in the trajectory, and FIG. 8B shows the points a, The line-of-sight directions of the stereo camera 100 at b, c, d, and e are shown. The white arrow in FIG. 8B represents the trajectory of the vehicle 500. In FIG. 8B, solid arrows Ea, Eb, Ec, Ed, and Ee indicate the line-of-sight directions of the stereo camera 100 at points a, b, c, d, and e, and broken lines Tb, Tc, and Td indicate points b, It represents the tangent of the trajectory of the vehicle 500 at c and d.

  At the point a, the driver turns on the direction indicator, but does not operate the steering (not cut). Therefore, the vehicle 500 is traveling straight forward. At this time, since the traveling direction of the vehicle 500 has not changed, the ECU 300 obtains H = 0, which is information that the steering is not turned off, calculates the line-of-sight direction θ of the stereo camera 100 using Equation 1, and The rotation mechanism 200 is controlled via the rotation mechanism control unit 400 so that the line-of-sight direction of the camera 100 is maintained in the same state as the traveling direction of the vehicle 500 (indicated by “arrow Ea” in FIG. 8). Thereby, the line-of-sight direction of the stereo camera 100 is the direction of the arrow Ea.

  At the point b, the driver rotates the steering counterclockwise (turns to the left). Therefore, the vehicle 500 is traveling so as to turn left (indicated by “arrow Tb” in FIG. 8). At this time, the ECU 300 obtains H = h1 which is information that the steering is turned to the left, calculates the line-of-sight direction θ of the stereo camera 100 using Equation 1, and the line-of-sight direction of the stereo camera 100 bends the vehicle 500. The rotation mechanism 200 is controlled via the rotation mechanism control unit 400 so that the angle θb is inside the direction (indicated by “arrow Eb” in FIG. 8). Thereby, the line-of-sight direction of the stereo camera 100 becomes the direction of the arrow Eb.

  At the point c, the driver gradually returns the steering wheel that has been turned to the left to the original state. Therefore, the vehicle 500 is traveling in the left direction (indicated by “arrow Tc” in the figure). At this time, the ECU 300 obtains H = h2 (h2 <h1), which is information that the steering is turned to the left, calculates the line-of-sight direction θ of the stereo camera 100 using Equation 1, and the line-of-sight direction of the stereo camera 100 The rotation mechanism 200 is controlled via the rotation mechanism control unit 400 so that is in a direction with an angle θc on the inner side of the bending direction of the vehicle 500 (indicated by “arrow Ec” in FIG. 8). Thereby, the line-of-sight direction of the stereo camera 100 is the direction of the arrow Ec.

  At the point d, the driver returns the steering, which has been turned to the left, to the original state further than the point c, and brings the steering closer straight. Therefore, the vehicle 500 is traveling so as to turn to the left (indicated by “arrow Td” in FIG. 8). At this time, the ECU 300 obtains H = h3 (h3 <h2), which is information that the steering is turned to the left, calculates the line-of-sight direction θ of the stereo camera 100 according to Equation 1, and the line-of-sight direction of the stereo camera 100 The rotation mechanism 200 is controlled via the rotation mechanism control unit 400 so that the angle θd is inside the turning direction of the vehicle 500 (indicated by an arrow “Ed” in FIG. 8). Thereby, the line-of-sight direction of the stereo camera 100 becomes the direction of the arrow Ed.

  At the point e, the driver returns the steering that had been turned to the left to the original state (straight), and the vehicle 500 is traveling straight in the left direction. At this time, since the traveling direction of the vehicle 500 has not changed, the ECU 300 obtains H = 0, which is information that the steering is not turned off, calculates the line-of-sight direction θ of the stereo camera 100 using Equation 1, and The rotation mechanism 200 is controlled via the rotation mechanism control unit 400 so that the line-of-sight direction of the camera 100 is maintained in the same state as the traveling direction of the vehicle 500 (indicated by “arrow Ee” in FIG. 8). Thereby, the line-of-sight direction of the stereo camera 100 is the direction of the arrow Ee.

  As described above, the ECU 300 acquires the steering steering angle of the vehicle 500 and controls the line-of-sight direction of the stereo camera 100 based on the acquired change in the steering steering angle. That is, ECU 300 controls the line-of-sight direction of stereo camera 100 according to the change in the traveling direction of vehicle 500. For this reason, it is possible to shorten the time until the information around the vehicle 500 is acquired.

  In the above description, the steering angle is used. However, the present invention is not limited to this, and can be realized by information such as the steering speed and the wheel inclination angle.

  As shown in FIG. 9, when the traveling direction of the vehicle 500 is the forward direction, the ECU 300 performs the same control even when the vehicle 500 turns to the right (indicated by “arrow FR” in FIG. 9). It is preferable to carry out.

  Next, a case where the traveling direction of the vehicle 500 is the backward direction will be described as an example. FIG. 10: is a schematic block diagram (the 2) of the mobile body system which concerns on 1st Embodiment of this invention. FIG. 11 is a diagram for explaining the operation of the mobile system when the vehicle 500 is steered leftward when the traveling direction of the vehicle 500 is backward. FIG. 12 is a diagram for explaining the operation of the mobile system when the vehicle 500 is steered rightward when the traveling direction of the vehicle 500 is backward. Note that the arrows in FIG. 10 indicate the traveling direction of the vehicle 500.

  As shown in FIG. 10, when the traveling direction of vehicle 500 is backward, when vehicle 500 turns to the left (indicated by “arrow RL” in FIG. 11), ECU 300 changes the traveling direction of vehicle 500. The line-of-sight direction of the stereo camera 100 is controlled according to the change. For this reason, it is possible to shorten the time until the information around the vehicle 500 is acquired.

  Also, as shown in FIG. 12, when the traveling direction of the vehicle 500 is the backward direction, the ECU 300 also performs the same control when the vehicle 500 turns to the right (indicated by “arrow RR” in FIG. 12). It is preferable to carry out.

  As described above, the mobile system according to the first embodiment of the present invention is a stereo camera attached to a vehicle, a rotating mechanism that changes the viewing direction of the stereo camera, and a change in the traveling direction of the vehicle. And an ECU for controlling the viewing direction of the stereo camera. For this reason, it is possible to shorten the time required to acquire information around the vehicle. As a result, it is possible to suppress accidents involving people, motorcycles, etc. when making a left turn.

[Second Embodiment]
A mobile system according to a second embodiment of the present invention will be described.

  The mobile system according to the second embodiment of the present invention is that the ECU 300 changes at least one of the frame rate and the recognition processing rate of the stereo camera 110 in accordance with a change in the traveling direction of the vehicle 500. Different from the mobile system according to the first embodiment of the invention. Hereinafter, a description will be given focusing on differences from the first embodiment.

  FIG. 13 is a block diagram showing a hardware configuration of a stereo camera 110 according to the second embodiment of the present invention. FIG. 14 is a control block diagram of a mobile system according to the second embodiment of the present invention.

  As shown in FIG. 13, the stereo camera 110 has a vehicle information interface (I / F) 111.

  The vehicle information interface 111 is connected to the ECU 300. For this reason, the ECU 300 can transmit vehicle information such as information on steering and steering of the vehicle 500 and information on vehicle speed to the stereo camera 110 via the vehicle information interface 111. The vehicle information interface 111 is not particularly limited as long as it is an interface that can receive vehicle information. For example, a CAN interface (I / F) or a LIN interface (I / F) can be used.

  As shown in FIG. 14, ECU 300 determines a rotation angle of rotation mechanism 200 with respect to rotation mechanism control unit 400 based on information indicating the traveling direction of vehicle 500 and / or information on vehicle speed, and controls the rotation mechanism. The operation of the rotation mechanism 200 is controlled via the unit 400. As information indicating the traveling direction of the vehicle 500, information related to steering such as a steering angle, a steering speed, and a wheel inclination angle can be used.

  The ECU 300 transmits information indicating the traveling direction of the vehicle 500 and / or vehicle speed information to the stereo camera 110, and changes at least one of the frame rate of the imaging unit of the stereo camera 110 and the recognition processing rate of the stereo camera 110. To do. Specifically, when the line-of-sight direction of the stereo camera 110 is changed, the ECU 300 preferably increases at least one of the frame rate and the recognition processing rate of the stereo camera 110. As a result, the time difference between image frames that affects the recognition process can be shortened. As a result, temporally high-density imaging and recognition can be executed with respect to a range outside the field of view of the driver of the vehicle 500. In particular, since the tracking accuracy of the recognition object captured in the adjacent image frame can be ensured, the recognition accuracy can be improved.

  When the traveling direction of the vehicle 500 is returned to the original state, the ECU 300 preferably returns the changed frame rate or recognition processing rate of the stereo camera 110 to the original state. Thereby, power consumption and heat generation when the traveling direction of the vehicle 500 is not changed can be suppressed.

  As described above, the mobile system according to the second embodiment of the present invention is a stereo camera attached to a vehicle, a rotating mechanism that changes the viewing direction of the stereo camera, and a change in the traveling direction of the vehicle. And an ECU for controlling the viewing direction of the stereo camera. For this reason, it is possible to shorten the time required to acquire information around the vehicle.

  In particular, in the second embodiment, the ECU changes at least one of the frame rate of the stereo camera and the recognition processing rate in accordance with a change in the traveling direction of the vehicle. For this reason, the recognition accuracy of a person or an object by a stereo camera can be improved.

[Third Embodiment]
A mobile system according to a third embodiment of the present invention will be described.

  The moving body system according to the third embodiment of the present invention is that the ECU 300 controls the operation of the stereo camera 110 and / or the operation of the vehicle 500 based on the information of the object recognized by the stereo camera 100. It is different from the mobile system according to the second embodiment. Hereinafter, a description will be given focusing on differences from the second embodiment.

  FIG. 15 is a control block diagram of a mobile system according to the third embodiment of the present invention.

  The ECU 300 changes the frame rate of the stereo camera 110 based on the parallax information acquired by the stereo camera 110. Specifically, the ECU 300 causes the frame rate of the stereo camera 110 when an object with a large parallax exists in the parallax information to be higher than the frame rate of the stereo camera 110 when an object with a large parallax does not exist in the parallax information. In addition, it is preferable to control the operation of the stereo camera 110. This shortens the recognition time interval when there is a person or object that is close to the vehicle 500 and can secure the tracking information of the recognition target imaged in the adjacent image frame, thus improving the recognition accuracy. Can be made.

  When there is no person or object close to the vehicle 500, it is not necessary to shorten the recognition time interval. Therefore, when the traveling direction of the vehicle 500 is not changed, the frame rate and the recognition processing rate are not changed. It is preferable to keep the same setting as. Thereby, power consumption and heat generation when the traveling direction of the vehicle 500 is not changed can be suppressed.

  Moreover, it is preferable that ECU300 acquires the recognition result of the person and object recognized by the stereo camera 110, and performs vehicle control, such as an automatic brake. Thereby, it is possible to prevent the vehicle 500 from colliding with a person or an object.

  As described above, the mobile system according to the third embodiment of the present invention is a stereo camera attached to a vehicle, a rotating mechanism that changes the viewing direction of the stereo camera, and a change in the traveling direction of the vehicle. And an ECU for controlling the viewing direction of the stereo camera. For this reason, it is possible to shorten the time required to acquire information around the vehicle.

  In particular, in the third embodiment, the ECU controls the operation of the stereo camera based on information on the object recognized by the stereo camera. For this reason, the recognition accuracy of a person or an object by a stereo camera can be improved.

  In the third embodiment, the ECU controls the operation of the vehicle based on the information on the object recognized by the stereo camera. For this reason, it is possible to prevent the vehicle from colliding with a person or an object.

[Principles of ranging]
The principle of distance measurement applied in an embodiment of the present invention will be described. Here, for example, the principle of deriving a parallax with respect to an object from a stereo camera and measuring the distance from the stereo camera to the object with a parallax value indicating the parallax will be described. FIG. 16 is an explanatory diagram of the principle of deriving the distance from the imaging device to the object. In the following, in order to simplify the description, the description will be made in units of one pixel instead of a predetermined area.

(Parallax value calculation)
First, the images captured by the imaging device 10a and the imaging device 10b are referred to as a reference image Ia and a comparative image Ib, respectively. In FIG. 16, it is assumed that the imaging device 10a and the imaging device 10b are installed in parallel equiposition. In FIG. 16, the point S on the object E in the three-dimensional space is mapped to a position on the same horizontal line of the imaging device 10a and the imaging device 10b.

  That is, the S point in each image is imaged at a point Sa (x, y) in the reference image Ia and a point Sb (X, y) in the comparative image Ib. At this time, the parallax value Δ is expressed by the following equation using Sa (x, y) at coordinates on the imaging device 10a and Sb (X, y) at coordinates on the imaging device 10b.

Δ = X−x [Formula 3]
Here, in the case as shown in FIG. 16, the distance between the point Sa (x, y) in the reference image Ia and the intersection of the perpendicular line taken from the imaging lens 11a onto the imaging surface is Δa, and in the comparative image Ib. When the distance between the point Sb (X, y) and the intersection of the perpendicular line taken from the imaging lens 11b on the imaging surface is Δb, the parallax value Δ = Δa + Δb.

(Distance calculation)
Further, by using the parallax value Δ, the distance Z between the imaging devices 10a and 10b and the object E can be derived. Specifically, the distance Z is a distance from a plane including the focal position of the imaging lens 11a and the focal position of the imaging lens 11b to the specific point S on the object E.

  As shown in FIG. 16, using the focal length f of the imaging lens 11a and the imaging lens 11b, the baseline length B that is the length between the imaging lens 11a and the imaging lens 11b, and the parallax value Δ, The distance Z can be calculated.

Z = (B × f) / Δ [Formula 4]
From the above equation 4, it can be seen that the larger the parallax value Δ, the smaller the distance Z, and the smaller the parallax value Δ, the larger the distance Z.

(Parallax calculation method)
The parallax calculation method will be described with reference to FIG. FIG. 17 is a diagram for explaining a processing process for generating a parallax image in a stereo camera.

  As shown in FIG. 17, the parallax calculation method uses the reference image Ia and the comparison image Ib to calculate a cost value that is a dissimilarity for each disparity, and the disparity at a position where the dissimilarity is low. calculate. Then, finally, a parallax image indicating a parallax value in all pixels is derived by the parallax image generation unit. The generated parallax image is used for the recognition of a person or an object by the object recognition unit provided at the subsequent stage of the parallax image generation unit together with the luminance image.

(Calculation of cost value)
A method for calculating the cost value C (p, d) will be described with reference to FIGS. 18 and 19.

  FIG. 18 is a diagram for explaining image matching by parallax search. Specifically, FIG. 18A is a schematic diagram showing reference pixels in the reference image Ia, and FIG. 18B shows corresponding pixel candidates in the comparison image Ib with respect to the reference pixels in FIG. It is the schematic at the time of calculating shift amount (shift amount), shifting sequentially (shifting). Here, the corresponding pixel means a pixel in the comparison image Ib that is most similar to the reference pixel in the reference image Ia.

  FIG. 19 is a graph showing the cost value for each shift amount. Specifically, the horizontal axis in FIG. 19 represents the shift amount d, and the vertical axis represents the cost value C.

  As shown in FIG. 18A, a predetermined reference pixel p (x, y) in the reference image Ia and a plurality of correspondences on the epipolar line EL in the comparison image Ib with respect to the reference pixel p (x, y). Based on each luminance value with the pixel candidate q (x + d, y), the cost value C (p, d) of each corresponding pixel candidate q (x + d, y) with respect to the reference pixel p (x, y) is calculated. The d is the shift amount (deviation amount) between the reference pixel p and the corresponding pixel candidate q, and in this embodiment, the shift amount is expressed in units of pixels.

  That is, in FIG. 18, the corresponding pixel candidate q (x + d, y) is sequentially shifted by one pixel within a predetermined range (for example, 0 <d <25). And a cost value C (p, d), which is a dissimilarity between the luminance values of the reference pixel p (x, y) and the reference pixel p (x, y).

  The cost value C (p, d) calculated in this way can be represented by a graph shown for each shift amount d, for example, as shown in FIG. In FIG. 19, since the cost value C is the minimum value when the shift amount d = 17, the parallax value is determined to be 17.

  EXAMPLES Hereinafter, although an Example demonstrates this invention concretely, this invention is limited to these Examples and is not interpreted.

[Example 1]
As an example of the mobile system, a rotation mechanism 200 for changing the viewing direction of the stereo camera 100 is provided on the hood of the vehicle 500, and the stereo camera 100 is attached on the rotation mechanism 200. The rotation mechanism 200 can be controlled using CAN information based on information on the steering angle of the vehicle 500. A simulation was performed with an actual vehicle assuming that a person crosses the road with a distance of 10 m from the left side when the vehicle 500 turns left. As a result, since the stereo camera 100 can capture and recognize a person who is not in the driver's field of view, the vehicle 500 receiving the information automatically brakes to prevent a collision with the person in advance. I was able to.

[Comparative Example 1]
Using the same moving body system as in the first embodiment, the rotation mechanism 200 is turned off and the line-of-sight direction of the stereo camera 100 does not move (a state in which the stereo camera 100 is fixed in the traveling direction of the vehicle 500). A simulation was performed with a real car. As a result, since the presence of a person was not transmitted to the vehicle 500, the vehicle 500 and the person were likely to collide.

[Example 2]
Using a mobile system similar to that of the first embodiment, when a vehicle 500 turns left, a simulation is performed with an actual vehicle assuming that another vehicle crosses the road with a distance of 20 m from the left side from the vehicle 500. went. As a result, since the stereo camera 100 was able to capture and recognize other vehicles that are not in the driver's field of view, the vehicle 500 that received the information automatically brakes and collides with the other vehicles in advance. Was able to prevent.

[Comparative Example 2]
Using the same moving body system as in the second embodiment, the simulation was performed with the same vehicle as in the second embodiment in a state where the rotation mechanism 200 was turned off and the line-of-sight direction of the stereo camera 100 did not move. As a result, since the presence of the other vehicle was not transmitted to the vehicle 500, the vehicle 500 and the other vehicle were likely to collide.

[Example 3]
As an example of the mobile system, a rotation mechanism 200 for changing the viewing direction of the stereo camera 100 is provided on the roof of the vehicle 500, and the stereo camera 100 is attached on the rotation mechanism 200. The rotation mechanism 200 can be controlled using CAN information based on information on the wheel inclination angle of the vehicle 500. A simulation was performed with an actual vehicle assuming that a person crosses the road with a distance of 10 m from the left side when the vehicle 500 turns left. As a result, since the stereo camera 100 can capture and recognize a person who is not in the driver's field of view, the vehicle 500 receiving the information automatically brakes to prevent a collision with the person in advance. I was able to.

[Comparative Example 3]
Using the same moving body system as in the third embodiment, the simulation was performed with the same vehicle as in the third embodiment in a state where the rotation mechanism 200 was turned off and the line-of-sight direction of the stereo camera 100 did not move. As a result, since the presence of a person was not transmitted to the vehicle 500, the vehicle 500 and the person were likely to collide.

[Example 4]
Using a mobile system similar to that of the third embodiment, when the vehicle 500 makes a left turn, a simulation is performed with an actual vehicle assuming that another vehicle crosses the road with a distance of 20 m from the left side from the vehicle 500. went. As a result, since the stereo camera 100 was able to capture and recognize other vehicles that are not in the driver's field of view, the vehicle 500 that received the information automatically brakes and collides with the other vehicles in advance. Was able to prevent.

[Comparative Example 4]
Using a mobile system similar to that of the fourth embodiment, the simulation was performed with the same vehicle as that of the fourth embodiment in a state where the rotation mechanism 200 was turned off and the line-of-sight direction of the stereo camera 100 did not move. As a result, since the presence of the other vehicle was not transmitted to the vehicle 500, the vehicle 500 and the other vehicle were likely to collide.

[Example 5]
In the form of Example 1, since the stereo camera 100 acquired the parallax about the person who is 10 m away, the frame rate of the stereo camera 100 was increased. As a result, since the stereo camera 100 was able to capture and recognize a person who is not in the driver's field of view, the vehicle 500 that received the information automatically brakes to prevent a collision with a margin. I was able to.

  As mentioned above, although the mobile body system was demonstrated by embodiment, this invention is not limited to the said embodiment, A various deformation | transformation and improvement are possible within the scope of the present invention.

  Although the mobile body system which concerns on the above-mentioned embodiment demonstrated the case of a vehicle as an example of a mobile body, this invention is not limited in this point. The moving body may be, for example, a helicopter operated by a control stick, a flying body such as an airplane, or an unmanned flying body remotely operated by a radio pilot. Further, the moving body may be a ship.

  In the case where the moving body is a flying object operated with a control stick, it is preferable that the control unit of the moving body system controls the line-of-sight direction of the imaging device based on a change in the control signal of the control stick.

  When the moving object is a flying object that is remotely operated by a radio pilot, the control unit of the mobile system controls the line-of-sight direction of the imaging device based on the control signal of the radio pilot or the image information captured by the imaging device. It is preferable to do. When controlling the line-of-sight direction of the imaging device based on image information captured by the imaging device, for example, the obstacle is recognized based on the image information, and the moving body is controlled so as to avoid the recognized obstacle. . For this reason, when the movement signal indicating which direction to move in order to avoid the obstacle is calculated, the line-of-sight direction of the imaging apparatus may be controlled based on the movement signal. Note that the configuration in which the line-of-sight direction of the imaging device is changed based on image information is suitable for a flying object, but can also be applied to the above-described vehicle and the like.

  When the moving body is a flying body, the control of the line-of-sight direction of the imaging device is naturally performed not only in the left-right direction but also in the up-down direction.

  A monocular camera may be used instead of the stereo camera. When a monocular camera is used, distance information can be acquired by a laser radar or the like. However, when recognizing a person or an object based on the acquired distance information, it may be difficult to obtain a sufficient recognition accuracy because it is difficult to obtain a spatial resolution using a laser radar. Therefore, it is preferable to use a stereo camera with higher spatial resolution in a system that acquires not only the front of the moving body but also the information around the moving body, such as the above-described moving body system.

100 Stereo camera 200 Rotating mechanism 300 ECU
500 vehicles

JP 2013-93013 A

Claims (15)

An imaging device attached to a moving body;
A line-of-sight direction changing mechanism for changing a line-of-sight direction of the imaging device;
A control unit that controls a line-of-sight direction of the imaging device according to a change in a traveling direction of the moving body,
Mobile body system. The control unit controls the line-of-sight direction of the imaging apparatus so as to change an angle formed by the line-of-sight direction of the imaging apparatus and the movement direction of the mobile body according to the amount of change in the traveling direction of the moving body. ,
The mobile system according to claim 1. The control unit controls the line-of-sight direction of the imaging device based on at least one information of a steering steering angle, a steering steering speed, and a wheel inclination angle of the moving body.
The mobile system according to claim 1 or 2. The control unit controls the line-of-sight direction of the imaging device so that the angle formed by the line-of-sight direction of the imaging device and the travel direction of the mobile body increases as the amount of change in the traveling direction of the moving body increases. ,
The mobile system according to claim 1. The moving body is a vehicle;
The mobile body system as described in any one of Claims 1 thru | or 4. The control unit controls the line-of-sight direction of the imaging device so as to be a direction with an angle inside the direction of bending of the vehicle;
The mobile system according to claim 5. The control unit detects a change amount in a traveling direction of the moving body based on a change in an image captured by a steering steering angle, a wheel inclination angle, or the imaging device.
The mobile system according to claim 5 or 6. The moving body is a flying body;
The mobile body system as described in any one of Claims 1 thru | or 4. The moving body is a flying body operated with a control stick,
The controller is
Detecting a change in the moving direction of the moving body based on a change in a control signal of the control stick, and controlling a line-of-sight direction of the imaging device;
The mobile system according to claim 8. The moving body is a flying body remotely operated by a radio pilot,
The control unit controls a line-of-sight direction of the imaging device based on a steering signal of the wireless pilot or image information captured by the imaging device;
The mobile system according to claim 8. The control unit changes at least one of a frame rate and a recognition processing rate of the imaging device when the line-of-sight direction of the imaging device is changed.
The mobile body system as described in any one of Claims 1 thru | or 10. The control unit increases at least one of the frame rate and the recognition processing rate of the imaging device when the line-of-sight direction of the imaging device is changed.
The mobile system according to claim 11. The control unit returns the changed frame rate or recognition processing rate of the imaging device to the original value when the traveling direction of the moving body returns to straight ahead after the line-of-sight direction of the imaging device is changed.
The mobile system according to claim 11 or 12. The control unit, based on the image information acquired by the imaging device, when an object with a short distance exists around the moving body, than when the object with a short distance does not exist around the moving body, Increasing the frame rate of the imaging device;
The mobile system according to any one of claims 11 to 13. The imaging device is a stereo camera.
The mobile system according to any one of claims 1 to 14.

Images