JP2018088092A - Image processing apparatus, on-vehicle system, on-vehicle camera, moving body, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an accurate height of a subject without burdening a driver.SOLUTION: An image processing apparatus acquires an image from an imaging part (S300). The image processing apparatus acquires distance information on a distance up to an arbitrary subject picked up in the image (S302). The image processing apparatus calculates a height Hof a predetermined subject in the arbitrary subject upon the basis of the image and distance information (S303).SELECTED DRAWING: Figure 6

Description

The present invention relates to an image processing apparatus, an in-vehicle system, an in-vehicle camera, a moving body, and an image processing method.

  There are traveling roads such as tunnels, pedestrian bridges, bridges, and other roads that have obstacles that can hinder driving. The driver needs to know the height of the vehicle he / she is driving and determine whether or not he / she can pass under these obstacles. In order to assist discrimination, it has been proposed to support a camera on the surface of a vehicle or a load loaded on the vehicle and display a photographed image of the camera (see Patent Document 1).

Japanese Patent Laid-Open No. 2001-239883

  However, since the driver determines the collision based on the image captured by the camera, the driver is still burdened. Further, depending on the support posture in which the camera is inclined with respect to the horizontal plane, it is difficult to accurately determine the collision.

  Accordingly, an object of the present disclosure made in view of the problems of the conventional technology as described above is to provide an image processing device, an in-vehicle system, an in-vehicle camera, and an image processing device that provide an accurate subject height without placing a burden on the driver. A moving object and an image processing method are provided.

In order to solve the above-described problems, the image processing apparatus according to the first aspect is
Obtain an image from the imaging unit,
Obtain distance information to any subject imaged in the image,
The height of a predetermined subject among the arbitrary subjects is calculated based on the image and the distance information.

The in-vehicle system according to the second aspect is
An image is acquired from an imaging unit, distance information to an arbitrary subject captured in the image is acquired, and a height of a predetermined subject among the arbitrary subjects is calculated based on the image and the distance information. The predetermined subject includes a moving body imaged in the image, and an image processing device that creates notification information including height information of the moving body to be calculated;
A communication device for notifying the mobile body of the notification information.

Moreover, the vehicle-mounted camera by the 3rd viewpoint is
An imaging unit;
The distance information to an arbitrary subject imaged by the imaging unit is acquired, and the height of a predetermined subject among the arbitrary subjects is calculated based on the image generated by the imaging of the imaging unit and the distance information. And an image processing device.

In addition, the mobile object according to the fourth aspect is
An imaging unit;
The distance information to an arbitrary subject imaged by the imaging unit is acquired, and the height of a predetermined subject among the arbitrary subjects is calculated based on the image generated by the imaging of the imaging unit and the distance information. And an image processing device.

  As described above, the solution of the present invention has been described as an apparatus. However, the present disclosure can also be realized as a method, a program, and a storage medium storing the program, which are substantially equivalent to these, and the scope of the present invention. It should be understood that these are also included.

For example, an image processing method that realizes the fifth aspect as a method is as follows:
Obtain an image from the imaging unit,
Obtain distance information to any subject imaged in the image,
The height of a predetermined subject among the arbitrary subjects is calculated based on the image and the distance information.

  According to the image processing apparatus, the in-vehicle system, the in-vehicle camera, the moving body, and the image processing method according to the present disclosure configured as described above, an accurate subject height is provided without burdening the driver. obtain.

1 is a functional block diagram illustrating a schematic configuration of an in-vehicle system including an image processing apparatus according to an embodiment of the present disclosure. It is a figure which shows the example of arrangement | positioning in the one part mobile body of the vehicle-mounted system of FIG. FIG. 2 is a diagram for describing a position of a subject and an angle of the subject with respect to an optical axis in an image captured by the imaging unit of FIG. 1. 3 is a flowchart for explaining a correction process performed by the image processing apparatus of FIG. 1. 3 is a flowchart for explaining request processing performed by the image processing apparatus of FIG. 1. It is a flowchart for demonstrating the collision discrimination | determination process which the image processing apparatus of FIG. 1 performs. 3 is a flowchart for explaining height calculation processing performed by the image processing apparatus in FIG. 1.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 shows an in-vehicle system 11 including an image processing apparatus 10 according to the present embodiment. The in-vehicle system 11 includes an imaging unit 12, a tilt sensor 13, a stack sensor 14, a distance detection device 15, an image processing device 10, a communication device 16, and a display device 17. The imaging unit 12, the tilt sensor 13, the stack sensor 14, the distance detection device 15, the image processing device 10, the communication device 16, and the display device 17 are connected to a bus 18. The imaging unit 12, the tilt sensor 13, the loading sensor 14, the distance detection device 15, the image processing device 10, the communication device 16, and the display device 17 can communicate various information, signals, commands, and the like via the bus 18. It is.

  The imaging unit 12 images a subject and generates an image. For example, the imaging unit 12 includes an optical system and an imaging element. The image sensor captures an object image formed by the optical system by photoelectric conversion, and generates an image of the object as a signal. The imaging unit 12 notifies the image processing apparatus 10 of the generated image as a signal.

  As shown in FIG. 2, the imaging unit 12 is attached, for example, in the vicinity of the center above the front shield 20 so that a subject in front of the moving body 19 can be imaged. The height of the mounting position of the imaging unit 12 on the moving body 19 from the ground is determined by design, and the imaging unit 12 is mounted on the moving body 19 so as to match the height. The direction of the optical axis of the imaging unit 12 in the moving body 19 is determined by design so as to be a specific angle with respect to the horizontal plane of the moving body 19 in the direction from the rear to the front of the moving body 19. The imaging unit 12 is attached to the moving body 19 so as to match the direction. The specific angle is, for example, zero degrees in the present embodiment. Furthermore, in this embodiment, the imaging unit 12 is attached so that the vertical direction and the horizontal direction of the image to be captured correspond to the vertical vertical direction and the horizontal direction of the moving body 19.

  The mobile body 19 may include a vehicle, a ship, an aircraft, and the like, for example. The vehicle may include, for example, an automobile, an industrial vehicle, a railway vehicle, a living vehicle, a fixed wing aircraft that runs on a runway, and the like. Automobiles may include, for example, passenger cars, trucks, buses, motorcycles, trolley buses, and the like. Industrial vehicles may include, for example, industrial vehicles for agriculture and construction. Industrial vehicles may include, for example, forklifts and golf carts. Industrial vehicles for agriculture may include, for example, tractors, cultivators, transplanters, binders, combiners, lawn mowers, and the like. Industrial vehicles for construction may include, for example, bulldozers, scrapers, excavators, crane trucks, dump trucks, road rollers, and the like. The vehicle may include a vehicle that travels manually. The classification of the vehicle is not limited to the example described above. For example, an automobile may include an industrial vehicle capable of traveling on a road. The same vehicle may be included in multiple classifications. Ships may include, for example, marine jets, boats, tankers, and the like. Aircraft may include, for example, fixed wing aircraft and rotary wing aircraft.

  The tilt sensor 13 detects the pitch angle of the moving body 19, that is, the tilt angle with respect to the horizontal plane from the reference posture with the horizontal direction of the moving body 19 as an axis. The tilt sensor 13 notifies the detected tilt angle to the image processing apparatus 10 as a signal.

  The loading sensor 14 detects whether or not cargo is loaded on the upper part of the moving body 19. The loading sensor 14 is, for example, a piezoelectric element provided on the top of the moving body 19 and detects a weight exceeding a weight threshold value as a detection result. The stack sensor 14 notifies the image processing apparatus 10 of the detection result as information.

  The distance detection device 15 detects the distance of an arbitrary subject included in the imaging range substantially the same as the imaging range of the imaging unit 12. The distance detection device 15 is, for example, a TOF (Time Of Flight) distance image camera. Alternatively, the distance detection device 15 is a laser range finder, for example. The distance detection device 15 notifies the image processing device 10 of the detected arbitrary distance as distance information. The distance detection device 15 is attached in the vicinity of the imaging unit 12 in the vicinity of the center above the front shield 20 so that the distance of the subject in front of the moving body 19 can be detected.

  The image processing apparatus 10 includes one or more processors and a memory. The processor may include a general-purpose processor that reads a specific program and executes a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include an FPGA (Field-Programmable Gate Array). The image processing apparatus 10 may be one of SoC (System-on-a-Chip) in which one or a plurality of processors cooperate and SiP (System In a Package).

  The image processing device 10 determines whether or not a predetermined subject exists among arbitrary subjects captured in the image acquired from the imaging unit 12. The predetermined subject is, for example, an obstacle over the traveling path of the moving body 19 on which the image processing apparatus 10 is mounted. The obstacle is, for example, a tunnel, a bridge such as a pedestrian bridge, or a signboard. The image processing apparatus 10 determines the presence or absence of an obstacle by pattern matching, for example. As will be described later, the predetermined subject is the moving body 19 in the image other than the own moving body 19 on which the image processing apparatus 10 is mounted.

  When there is a predetermined subject, the image processing apparatus 10 determines the height of the lower end of the predetermined subject based on the images and distance information acquired from the imaging unit 12 and the distance detection device 15 as described below. calculate.

As shown in FIG. 3, an arbitrary position _{P A} in the vertical direction relative to the position _{P OX} corresponding on the image IM on the optical axis OX is the straight line _{L 1} and the optical axis OX connecting the lower end _{BE sub} mobile 19 and the object This corresponds to the first angle θ ₁ formed. Based on the position of the lower end BE _sub of the predetermined subject in the image IM, the image processing apparatus 10 calculates the first angle θ ₁ formed by the optical axis OX and the straight line L ₁ connecting the moving body 19 and the lower end BE _sub. To do.

Based on a predetermined angle (in this embodiment, zero degree) with respect to the horizontal plane of the optical axis OX of the imaging unit 12 and the first angle θ ₁ , the image processing apparatus 10 and the moving body 19 and A second angle θ ₂ formed by the straight line L ₁ connecting the subject and the horizontal plane is calculated. The specific angle is determined in advance as described above, and is stored in advance in the memory of the image processing apparatus 10 when the image processing apparatus 10 is mounted on the moving body 19. Alternatively, the specific angle may be stored in an ECU (Electronic Control Unit) of the moving body 19 and may be read by the image processing apparatus 10 from the ECU as necessary.

The image processing apparatus 10 corrects the second angle when the inclination angle acquired from the inclination sensor 13 exceeds the angle threshold. The angle threshold is, for example, a value that allows an error in height based on the second angle θ ₂ of the subject at a predetermined distance from the moving body 19 to fall within an allowable range.

The image processing apparatus 10 detects the first distance d _s from the moving body 19 to the lower end BE _sub of the predetermined subject from the distance information acquired from the distance detection device 15.

Based on the calculated second angle θ ₂ and the detected first distance d _s , the image processing apparatus 10 uses the following equation (1) as a reference for the installation height H _IG of the imaging unit 12 in the moving body 19. and the calculated height _{H SI} of the lower end _{bE sub} of predetermined subject.

H _SI = d _s × sin θ ₂ (1)

In the equation (1), θ ₂ is the second angle, that is, the angle formed by the straight line L ₁ connecting the moving body 19 and the lower end BE _sub of the subject and the horizontal plane.

The image processing apparatus 10, calculated to the height _{H SI,} by adding the height _{H IG} from the ground G of the mounting position of the imaging unit 12 in the mobile body 19, the lower end _{BE sub} predetermined subject with respect to the ground G High H _SG is calculated. As described above, the height H _IG from the ground G at the mounting position of the imaging unit 12 in the moving body 19 is determined in advance as described above, and image processing is performed when the image processing apparatus 10 is mounted on the moving body 19. Pre-stored in the memory of the device 10. Alternatively, the height H _IG may be stored in the ECU of the moving body 19 and read by the image processing apparatus 10 from the ECU as necessary.

When the image processing apparatus 10 calculates the height H _SG of the lower end BE _sub of a predetermined subject, the calculated height H _SG is used as a server for managing traffic information and other movements that do not have a height measurement function. The communication device 16 is controlled to notify the body. Further, when the height H _SG of the lower end BE _sub of the predetermined subject is calculated, the image processing apparatus 10 acquires the height information of the moving body 19 as described below.

Corresponds to the height information, itself the height H _OM mobile 19, is predetermined by the design, for example, mounted upon the memory in the memory of the image processing apparatus 10 to the mobile 19 of the image processing apparatus 10 Has been. Or, the height H _OM is stored in the ECU of the moving body 19, the image processing apparatus 10 may be configured to read from the ECU as needed. The height H _OM determined in advance by design can be corrected by the image processing apparatus 10 by the method described below.

For correction, the image processing apparatus 10 determines whether or not there is a subject on the traveling path of its own mobile body 19 such as a center line and a stop line in the image IM acquired from the imaging unit 12. The image processing apparatus 10, when detecting an object traveling path, the height H _SI of the subject relative to the height H _IG installation of the imaging unit 12 in the mobile 19, the height of the predetermined subject in the above Similar to the calculation, calculation is performed based on position and distance information in the image IM.

Since the height H _SG of the subject on the traveling road with respect to the ground G is zero, the calculated height H _SI corresponds to the installation height H _IG of the imaging unit 12 in the moving body 19 at the time of calculation. The image processing apparatus 10 calculates the difference between the installation height H _IG of the imaging unit 12 in the moving body 19 in the design and the installation height H _SI of the imaging unit 12 in the moving body 19 at the time of calculation. The image processing apparatus 10 corrects the height information corresponding to the height _HOM of the moving body 19 using the calculated difference.

  In addition, as will be described later, the image processing apparatus 10 can acquire height information from another moving body 19 via the communication apparatus 16. The height corresponding to the height information acquired from the other mobile body 19 via the communication device 16 may include the height of the cargo loaded on the top of the mobile body 19.

Further, the image processing apparatus 10 obtains the height _{H OM} own mobile body 19, or the height _{H OM} smaller than whether the height _{H SG} its own mobile 19 of the lower end _{BE sub} predetermined subject Is determined. The image processing apparatus 10 generates a warning signal when the height H _SG of the lower end BE _sub of a predetermined subject is smaller than the height H _OM of its own mobile body 19. The warning signal may be in the form of warning by sound or in the form of warning by an image. The image processing apparatus 10 transmits the generated warning signal to the ECU and the display device 17.

Further, as described above, the image processing apparatus 10 can calculate the height H _AM of the moving body 19 in the image IM as a predetermined subject in the image IM acquired from the imaging unit 12 as described below. It is. When the image processing apparatus 10 obtains a notification request from another moving body 19 other than its own moving body 19, the image processing apparatus 10 calculates the height of the moving body 19 in the image IM. Notification request, the notification information including the height information corresponding to the height H _AM mobile 19 in the image IM, a request to create the moving body 19 relative to the imaging-enabled mobile body 19.

When the image processing apparatus 10 acquires the notification request and detects the moving body 19 in the image IM, the image processing apparatus 10 calculates the height H _AM of the moving body 19 in the image IM in the same manner as described above. Incidentally, when the moving body 19 in the image IM in the calculation of the height H _AM which was predetermined object, when the moving body 19 in the image IM is loaded with cargo on top, the height of the upper end of the loading the cargo Is calculated.

The image processing apparatus 10, when calculating the height H _AM mobile 19 in the image IM, create a notification information including height information corresponding to the height H _AM mobile 19 in the image IM. The image processing device 10 causes the communication device 16 to notify the moving body 19 in the image IM of the notification information.

When the image processing apparatus 10 acquires a notification request and detects the presence of an obstacle on the traveling road as well as other moving bodies 19 in the image IM, the image processing apparatus 10 further increases the height of the obstacle based on the ground G. H _SG is calculated. The image processing apparatus 10 compares the height H _AM of the moving body 19 and the height H _SG of the obstacle in the image IM.

The image processing apparatus 10 generates a warning signal when the height H _SG of the obstacle is smaller than the height H _AM of the moving body 19 in the image IM. The image processing apparatus 10 causes the communication device 16 to transmit the generated warning signal to the moving body 19 in the image IM. Alternatively, the image processing apparatus 10 adds the height comparison result to the notification information including the height information of the moving body 19 in the image IM. The image processing apparatus 10 causes the communication apparatus 16 to notify the created notification information to the mobile body 19 that has transmitted the notification request.

  In addition, the image processing apparatus 10 creates a notification request for requesting the mobile body 19 other than the mobile body 19 to notify the height information of the mobile body 19. The image processing apparatus 10 automatically creates a notification request when a specific condition is satisfied, for example, when the loading sensor 14 detects cargo loading. The image processing apparatus 10 may create a notification request based on an input to an input device that detects a user operation, for example.

  The image processing apparatus 10 causes the communication device 16 to communicate the created notification request. As described above, the image processing apparatus 10 of its own moving body 19 acquires notification information including the height information of its own moving body 19 calculated by the image processing apparatus 10 of the other moving body 19.

  The communication device 16 communicates with another mobile body 19 other than its own mobile body 19 on which the communication device 16 is mounted. For example, in this embodiment, the communication device 16 notifies the other mobile body 19 of the notification information created by the image processing apparatus 10 mounted on the mobile body 19 thereof. In addition, the communication device 16 acquires notification information created by the image processing device 10 mounted on the other mobile body 19 from the other mobile body 19. In addition, the communication device 16 notifies the other mobile body 19 of a notification request created by the image processing apparatus 10 mounted on its own mobile body 19. In addition, the communication device 16 acquires a notification request created by the image processing apparatus 10 mounted on the other mobile body 19 from the other mobile body 19.

  The display device 17 displays various images. For example, in the present embodiment, when the display device 17 acquires a warning signal from the image processing device 10, the display device 17 displays an image that notifies that there is an obstacle with a high possibility of collision. As shown in FIG. 2, the display device 17 is attached to a position where the driver can visually recognize, for example, near the center of the dashboard.

  Next, correction processing executed by the image processing apparatus 10 in the present embodiment will be described with reference to the flowchart of FIG. The correction process starts when a drive source such as an engine and a motor of the moving body 19 is activated.

  In step S100, the image processing apparatus 10 causes the imaging unit 12 to start capturing a moving image. When imaging is started, the process proceeds to step S101.

  In step S101, the image processing apparatus 10 determines whether or not there is a subject on the road in the image IM of the frame that constitutes the moving image started to be captured in step S100. When there is no subject on the travel path, the process returns to step S101, and step S101 is repeated for the image IM of the frame after the frame used for determination. When there is a subject on the travel path, the process proceeds to step S102.

  In step S <b> 102, the image processing device 10 acquires distance information by requesting distance information from the distance detection device 15. When the distance information is acquired, the process proceeds to step S103.

In step S103, the image processing apparatus 10 uses the installation height _HIG of the imaging unit 12 as a reference based on the image IM in which the presence of the subject on the travel path in step S101 is confirmed and the distance information acquired in step S102. of the subject height _{H SI,} it calculated. After calculating the height _{H SI,} the process proceeds to step S104.

In step S104, the image processing apparatus 10 uses the height H _SI calculated in step S103, to correct the height H _OM of the moving body 19 itself stored in the memory of the image processing apparatus 10 is mounted . After correction, the process proceeds to step S105.

In step S105, the image processing apparatus 10 determines to use the height _HOM corrected in step S104 as the height of the moving body 19 used for subsequent height determination and the like. After the determination, the correction process ends.

  Next, the height request process of the mobile unit 19 executed by the image processing apparatus 10 in the present embodiment will be described with reference to the flowchart of FIG. The request process starts when a drive source such as an engine and a motor of the moving body 19 is activated.

  In step S <b> 200, the image processing apparatus 10 determines whether the loading sensor 14 has detected cargo loading or whether there is an input to the input device for an operation in which the user requests height information. If there is neither loading detection nor input of the operation, the process repeats step S200. When there is a load detection or an operation input, the process proceeds to step S201.

  In step S201, the image processing apparatus 10 creates a notification request. Further, the image processing apparatus 10 causes the communication device 16 to communicate the created notification request to another mobile body 19 other than the mobile body 19 itself. When communicating, the process proceeds to step S202.

  In step S <b> 202, the image processing apparatus 10 determines whether or not height information is acquired from another moving body 19. When the height information has not been acquired, the process returns to step S201, and steps S201 and S202 are repeated. When obtaining height information, the process proceeds to step S203.

In step S203, the image processing apparatus 10, the height H _AM corresponding to the height information confirming the acquired in step S202, be used as the height H _OM own mobile body 19 for use in such subsequent height determination To decide. After the determination, the request process ends.

  Next, the obstacle collision determination process executed by the image processing apparatus 10 in the present embodiment will be described with reference to the flowchart of FIG. The collision determination process starts when the moving body 19 moves forward or backward.

  In step S300, the image processing apparatus 10 causes the imaging unit 12 to start capturing a moving image. When imaging is started, the process proceeds to step S301.

  In step S301, the image processing apparatus 10 determines whether there is an obstacle on the traveling path of the moving body 19 in the image IM of an arbitrary frame constituting the moving image started to be imaged in step S300. When there is no obstacle, the process returns to step S301, and step S301 is repeated for the image IM of the frame after the frame used for determination. If there is an obstacle, the process proceeds to step S302.

  In step S <b> 302, the image processing apparatus 10 acquires distance information by requesting distance information from the distance detection device 15. When the distance information is acquired, the process proceeds to step S303.

In step S303, the image processing apparatus 10 calculates the height H _SG of the lower end BE _sub of the obstacle based on the image IM in which the presence of the obstacle is confirmed in step S301 and the distance information acquired in step S302. After calculating the height H _SG , the process proceeds to step S304.

In step S304, the image processing apparatus 10, the height H _SG obstacles calculated in step S303 is, correction processing (see FIG. 4) or request processing height of their mobile body 19 as determined in (see FIG. 5) It is determined whether it is smaller than _HOM . When the height H _SG of the obstacle is _{equal to} or higher than the height H _OM of the moving body 19, the process returns to step S301, and for the image IM of the frame after the frame used for calculating the height in step S303, Steps S301 to S304 are repeated. When the height H _SG of the obstacle is smaller than the height H _OM of the moving body 19 of the obstacle, the process proceeds to step S305.

  In step S305, the image processing apparatus 10 generates a warning signal. Further, the image processing apparatus 10 transmits the generated warning signal to the ECU and the display device 17. After the transmission, the collision determination process ends.

  Next, height calculation processing of another moving body 19 executed by the image processing apparatus 10 in the present embodiment will be described with reference to the flowchart of FIG. The height calculation process starts when a notification request is acquired.

  In step S400, the image processing apparatus 10 interrupts the collision determination process when the collision determination process is being executed. When the collision determination process is not executed or after the interruption, the process proceeds to step S401.

  In step S401, the image processing apparatus 10 causes the imaging unit 12 to start capturing a moving image. When imaging is started, the process proceeds to step S402.

  In step S402, the image processing apparatus 10 determines whether or not another moving body 19 that has transmitted the notification request exists in the image IM of an arbitrary frame constituting the moving image that has started imaging in step S401. . When there is no other moving body 19, the process returns to Step S402, and Step S402 is repeated for the image IM of the frame after the frame used for the determination. When another mobile 19 exists, the process proceeds to step S403.

  In step S403, the image processing apparatus 10 acquires distance information by requesting distance information from the distance detection apparatus 15. When the distance information is acquired, the process proceeds to step S404.

In step S404, the image processing apparatus 10 calculates the height H _AM of the upper end of the other moving body 19 based on the image IM in which the existence of the other moving body 19 is confirmed in step S402 and the distance information acquired in step S403. calculate. Once the height is calculated, the process proceeds to step S405.

In step S405, the image processing apparatus 10 is calculated in step S404, creates a notification form report including height information corresponding to the height H _AM of other mobile 19. After creating the notification letter, the process proceeds to step S406.

  In step S406, the image processing apparatus 10 determines whether there is an obstacle in the image IM in which the presence of the other moving body 19 is confirmed in step S402. If there is an obstacle, the process proceeds to step S407. If there are no obstacles, the process proceeds to step S412.

In step S407, the image processing apparatus 10 calculates the height H _SG of the lower end BE _sub of the obstacle based on the image IM in which the presence of the obstacle is confirmed in step S406 and the distance information acquired in step S403. Once the height H _SG is calculated, the process proceeds to step S408.

In step S408, the image processing apparatus 10 determines that the height H _SG of the obstacle calculated in step S407 is the height of the moving body 19 determined in the correction process (see FIG. 4) or the request process (see FIG. 5). It is determined whether it is smaller than _HOM . When the height of the obstacle is smaller than the height of the moving body 19, the process proceeds to step S409. When the height H _SG of the obstacle is _{equal to} or higher than the height H _OM of the moving body 19, the process proceeds to step S410.

  In step S409, the image processing apparatus 10 generates a warning signal. Further, the image processing apparatus 10 transmits the generated warning signal to the ECU and the display device 17. After transmission, the process proceeds to step S410.

In step S410, the image processing apparatus 10 determines whether the obstacle height H _SG calculated in step S407 is smaller than the height H _{AM of} the other moving body 19 calculated in step S404. When the height H _SG of the obstacle is smaller than the height H _AM of the other mobile body 19, the process proceeds to step S411. When the height of the obstacle is equal to or higher than the height of the moving body 19, the process proceeds to step S412.

  In step S411, the image processing apparatus 10 adds the comparison result in step S410 to the notification information created in step S405. When the comparison result is added, the process proceeds to step S412.

  In step S412, the image processing apparatus 10 sends a notification letter to the other mobile body 19 that has sent the notification request, which is created in step S405 and added with the comparison result when the comparison result is created in step S411. . After sending the notification information, the process proceeds to step S413.

  In step S413, the image processing apparatus 10 resumes the collision determination process interrupted in step S400. With the restart of the collision determination process, the height calculation process ends.

The image processing apparatus 10 according to the present embodiment configured as described above calculates a predetermined subject height H _SG based on the image IM and the distance information. Therefore, the image processing apparatus 10 can provide an accurate predetermined subject height H _SG without imposing a burden such as visual observation on a driver or the like.

Further, the image processing apparatus 10 according to the present embodiment calculates the height H _SG of the obstacle over the traveling road, and compares the height H _OM of the mobile body 19 on which the vehicle is mounted and the height H _SG of the obstacle. Based on the above, a warning signal is generated. Therefore, the image processing apparatus 10 can provide a driver or the like with a direct warning as to whether an obstacle in the traveling direction of the moving body 19 on which the image processing apparatus 10 is mounted can collide with the moving body 19.

  Further, the image processing apparatus 10 according to the present embodiment makes a request for notification of the height information of the mobile body 19 to another mobile body 19 other than the mobile body 19 on which the image processing apparatus 10 is mounted. Get height information. Therefore, the image processing apparatus 10 can provide the height of the mobile unit 19 even when cargo is loaded on the upper surface of the mobile unit 19.

  Further, the image processing apparatus 10 according to the present embodiment corrects the height information of the moving body 19 based on the position and distance information of the subject on the traveling path of the moving body 19 in the image IM. Therefore, the image processing apparatus 10 can adjust the weight of the moving body 19 even when the height varies depending on the weight of the passenger on the moving body 19 and the amount of cargo loaded in the moving body 19. Height accuracy can be maintained.

The image processing apparatus 10 of the present embodiment, based on the image IM and distance information, calculates the height H _AM mobile 19 in the image IM, including height information of the movable body 19 in the image IM Create notification information. Therefore, the image processing apparatus 10 can provide the height of the moving body 19 in the image IM other than the moving body 19 on which the image processing apparatus 10 is mounted.

In addition, the image processing apparatus 10 according to the present embodiment compares the height H _AM of the moving body 19 and the height H _SG of the obstacle in the image IM, and adds the comparison result to the notification information. Therefore, the image processing apparatus 10 can provide information that can determine whether an obstacle in the traveling direction of the other moving body 19 can collide with the other moving body 19.

  Further, the image processing apparatus 10 according to the present embodiment creates notification information when a notification request is received. Therefore, the image processing apparatus 10 can generate the height information of the other moving body 19 only when necessary.

The image processing apparatus 10 of this embodiment, the lateral direction of the moving body 19 and the axis, from the reference position, with an inclination angle relative to the horizontal plane, between the straight line L ₁ and the horizontal plane connecting the moving body 19 and the object The formed second angle θ ₂ is corrected. Therefore, the image processing apparatus 10 can maintain the accuracy of the calculated height even when the moving body 19 is inclined by loading cargo in front of or behind the moving body 19.

  Although the present invention has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various changes and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention.

  For example, in the present embodiment, the display device 17 or the like is configured to notify the driver or the like that there is an obstacle with a high possibility of collision based on a warning signal transmitted by the image processing device 10. For example, risk avoidance such as braking control may be performed.

  Moreover, in this embodiment, although the imaging part 12 and the image processing apparatus 10 are comprised as a different body, you may comprise as a vehicle-mounted camera which united both.

  In the present embodiment, the image processing device 10 acquires distance information from the distance detection device 15 to an arbitrary subject imaged by the imaging unit 12, but based on the image IM acquired from the imaging unit 12. The distance to an arbitrary subject may be calculated. For example, the distance to an arbitrary subject can be measured by photogrammetry using an image IM taken from two cameras provided with parallax by the imaging unit 12, or photogrammetry using a motion stereo method.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 In-vehicle system 12 Image pick-up part 13 Inclination sensor 14 Loading sensor 15 Distance detection apparatus 16 Communication apparatus 17 Display apparatus 18 Bus 19 Mobile body 20 Front shield BE _sub lower end of subject G Ground IM image OX Optical axis PA In _A image Position in the vertical direction relative to the position corresponding to the optical axis of P P Position corresponding to the optical axis in the _OX image

Claims (14)

Obtain an image from the imaging unit,
Obtain distance information to any subject imaged in the image,
An image processing apparatus, comprising: calculating a height of a predetermined subject among the arbitrary subjects based on the image and the distance information. The image processing apparatus according to claim 1.
The predetermined subject is an obstacle over a traveling path of a moving body equipped with the image processing device,
Obtaining height information of the moving object;
An image processing apparatus, wherein a warning signal is generated when a height corresponding to the height information is greater than a height of the predetermined subject. The image processing apparatus according to claim 2,
Making a request for notification of height information of the mobile body to another mobile body other than the mobile body on which the image processing apparatus is mounted, and acquiring the height information from the other mobile body. A featured image processing apparatus. The image processing apparatus according to claim 2,
The height information is corrected based on the position and distance information in the image of a subject on a traveling path of a moving body on which the image processing device is mounted. In the image processing device according to any one of claims 1 to 4,
The predetermined subject includes a moving body in the image,
An image processing apparatus that creates notification information including height information of a moving object in the image to be calculated. The image processing apparatus according to claim 5.
The predetermined subject includes an obstacle over the traveling path of the moving body in the image,
The height of the moving body and the obstacle in the image is compared, and the comparison result is added to the notification information. The image processing apparatus according to claim 6.
An image processing apparatus that generates a warning signal when the height of the obstacle is smaller than the height of the moving object in the image by comparing the height of the moving object and the obstacle in the image. The image processing apparatus according to claim 6.
The image processing apparatus, wherein the notification information is notified to a moving body in the image. The image processing apparatus according to any one of claims 5 to 8,
The image processing apparatus, wherein the notification information is generated when a notification request is issued from a mobile body other than the mobile body on which the image processing apparatus is mounted. The image processing apparatus according to any one of claims 1 to 9,
The distance information is obtained by calculating a distance to the arbitrary subject based on the image. An image is acquired from an imaging unit, distance information to an arbitrary subject captured in the image is acquired, and a height of a predetermined subject among the arbitrary subjects is calculated based on the image and the distance information. The predetermined subject includes a moving body imaged in the image, and an image processing device that creates notification information including height information of the moving body to be calculated;
And a communication device that notifies the mobile body of the notification information. An imaging unit;
The distance information to an arbitrary subject imaged by the imaging unit is acquired, and the height of a predetermined subject among the arbitrary subjects is calculated based on the image generated by the imaging of the imaging unit and the distance information. An in-vehicle camera comprising: an image processing device. An imaging unit;
The distance information to an arbitrary subject imaged by the imaging unit is acquired, and the height of a predetermined subject among the arbitrary subjects is calculated based on the image generated by the imaging of the imaging unit and the distance information. An image processing apparatus. Obtain an image from the imaging unit,
Obtain distance information to any subject imaged in the image,
An image processing method, comprising: calculating a height of a predetermined subject among the arbitrary subjects based on the image and the distance information.

Images