WO2015045904A1 - Vehicle-periphery-moving-object detection system - Google Patents

Vehicle-periphery-moving-object detection system Download PDF

Info

Publication number
WO2015045904A1
WO2015045904A1 PCT/JP2014/074212 JP2014074212W WO2015045904A1 WO 2015045904 A1 WO2015045904 A1 WO 2015045904A1 JP 2014074212 W JP2014074212 W JP 2014074212W WO 2015045904 A1 WO2015045904 A1 WO 2015045904A1
Authority
WO
WIPO (PCT)
Prior art keywords
moving object
object detection
vehicle
video
surrounding
Prior art date
Application number
PCT/JP2014/074212
Other languages
French (fr)
Japanese (ja)
Inventor
善文 福田
小沼　知恵子
守飛 太田
石本　英史
Original Assignee
日立建機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日立建機株式会社 filed Critical 日立建機株式会社
Publication of WO2015045904A1 publication Critical patent/WO2015045904A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to a vehicle surrounding moving object detection system, and more particularly, to a vehicle surrounding moving object detection for avoiding contact with an object moving in the vicinity of a vehicle in a construction machine such as a large dump truck working in a mine. About the system.
  • Patent Document 1 JP 2010-147523 A (Patent Document 1). This publication describes that “a composite bird's-eye view image that can easily grasp a three-dimensional object around a vehicle to be watched by a supervisor while avoiding the disappearance of the three-dimensional object in the common area” is described.
  • the conventional vehicle surrounding detection system a part of the image around the host vehicle captured by a plurality of cameras is extracted, and the shape is converted and synthesized to generate a composite overhead image, and the user is notified of the surroundings of the host vehicle. In addition to showing the situation, if an object (three-dimensional object) that the user should pay attention to is present in the video, the user is warned by highlighting it.
  • a composite overhead image is generated to display the situation around the host vehicle, and an object (a three-dimensional object) is detected at the displayed location.
  • an object a three-dimensional object
  • the three-dimensional object is not detected and the user cannot be alerted. For this reason, there existed a possibility of impairing operational safety.
  • the present invention has been made based on the above-mentioned matters, and its purpose is to detect a moving object even in a place that is not displayed as a composite bird's-eye view image such as a lower part of a vehicle body of a large construction machine.
  • a vehicle surrounding moving object detection system capable of notifying the user of the result is provided.
  • a first invention provides a surrounding image input unit that captures a plurality of images around the host vehicle, and a composite image configuration area from the plurality of surrounding images captured by the surrounding image input unit.
  • Vehicle surrounding movement provided with a composite video construction unit that constructs a composite overhead view video by combining a plurality of extracted composite video composition areas and an output unit that presents the composite overhead view video to a user
  • the object detection system includes a moving object detection processing unit that performs a detection process of the moving object with respect to the plurality of surrounding images, and the moving object detection processing unit detects a moving object in a region other than the region shown in the composite overhead image. Even in this case, the detection result is output to the synthesized bird's-eye view image to notify the user.
  • a region information holding unit that holds information of a detection result display non-target region that is a region where a detection result indicating that a moving object has been detected is not displayed, and the region information holding A moving object detection processing unit that takes in the information of the detection result display non-target region from the unit, and performs a moving object detection process on the video that excludes the detection result display non-target region from the plurality of surrounding images,
  • the moving object detection processing unit outputs a detection result to the synthesized bird's-eye video and notifies the user even when a moving object is detected outside the area shown in the synthesized bird's-eye video.
  • the third invention is characterized in that, in the second invention, the detection result display non-target area is a part of the own vehicle reflected in the plurality of surrounding images.
  • the moving object detection processing unit when the moving object detection processing unit detects a moving object in a lower part of the host vehicle, the detection result is output as a message to the synthesized overhead view image and used. It is characterized by notifying a person.
  • the moving object detection processing unit when the moving object detection processing unit detects a moving object in a lower portion of the host vehicle, the detection result is set to a position indicating the host vehicle in the synthesized overhead view image. The drawing is output and notified to the user.
  • the information processing apparatus further includes an emphasis information holding unit that holds color information indicating a degree of emphasis display according to a distance between the host vehicle and the detected moving object.
  • the moving object detection processing unit takes in color information indicating the degree of emphasis in the emphasis display from the emphasis information holding unit, and sets an emphasis display color from a distance between the host vehicle and the detected moving object, The moving object that is a detection result is displayed on the composite bird's-eye view image in the highlighted display color that is set, and is notified to the user.
  • the detection result is notified to the user. Therefore, the user can quickly grasp the existence and the positional relationship of the object to be noted in the work of the own vehicle. As a result, it is possible to improve the efficiency of confirmation work for ensuring work safety, and to avoid contact with an object such as a person or another vehicle. As a result, the operational efficiency of the entire work can be improved.
  • FIG. 1 is a side view showing a dump truck provided with a first embodiment of a vehicle surrounding moving object detection system of the present invention.
  • a dump truck (vehicle) 1 shown in FIG. 1 includes a vehicle body 2 formed with a sturdy frame structure, a vessel (loading platform) 3 mounted on the vehicle body 2 so as to be able to undulate, and a left front wheel 4A mounted on the vehicle body 2. (L) and left rear wheel 4B (L) are mainly provided.
  • the vehicle body 2 is provided with an engine (not shown) that drives the rear wheels 4B.
  • the engine has, for example, an engine control device (hereinafter referred to as ECU), and the number of revolutions thereof is controlled by controlling the flow rate of fuel supplied by a command signal from the ECU.
  • ECU engine control device
  • the vessel 3 is a container provided to load a load such as a crushed stone, and is connected to the vehicle body 2 via a pin coupling portion 5 so as to be raised and lowered.
  • Two undulation cylinders 6 are installed at a lower portion of the vessel 3 with a predetermined interval in the width direction of the vehicle. When pressure oil is supplied to and discharged from the undulation cylinder 6, the undulation cylinder 6 extends and contracts, and the vessel 3 is undulated.
  • a collar portion 7 is provided on the upper front side of the vessel 3.
  • the heel portion 7 has a function of protecting the cab 8 installed on the lower side thereof (that is, the front portion of the vehicle body 2) from scattered objects such as rocks and protecting the cab 8 when the vehicle falls. Yes.
  • a control device 100 constituting a vehicle surrounding moving object detection system, a steering handle (not shown), an accelerator pedal, a brake pedal and the like (not shown). Is installed.
  • FIG. 2 is a conceptual diagram for explaining the arrangement of cameras constituting the surrounding image input unit in the first embodiment of the vehicle surrounding moving object detection system of the present invention
  • FIG. 3 is a diagram of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram which shows the image
  • a front camera 301 that captures the front of the dump truck 1 with a wide-angle lens and a rear camera 303 that captures the rear of the dump truck 1 with a wide-angle lens are provided on the front and rear sides of the vehicle body 2 of the dump truck 1. And are provided. Further, on the left side surface and the right side surface of the vehicle body 2, a left direction camera 302 that captures the left direction of the dump truck 1 with a wide angle lens and a right direction camera 304 that captures the right direction of the dump truck 1 with a wide angle lens are provided. Is provided.
  • the vehicle body 2 of the dump truck 1 is equipped with a right front wheel 4A (R) and a left rear wheel 4B (R).
  • FIG. 3 shows an example of surrounding images taken by these cameras 301-303.
  • Reference numeral 401 denotes an example of a front image captured by the front camera 301.
  • Reference numeral 402 denotes an example of a left direction image captured by the left direction camera 302.
  • Reference numeral 403 denotes an example of a rear image taken by the rear camera 303.
  • Reference numeral 404 denotes an example of a right direction image captured by the right direction camera 304.
  • the surrounding images 401 to 404 are each photographed with a wide-angle lens, a distant horizon located above each image is reflected in a curved state.
  • a part of the vehicle body 2 on the front side, a part of the left and right front wheels 4A (R) and 4A (L), and a part of the left and right rear wheels 4B (R) and 4B (L) are shown below each image. ing.
  • a synthesized overhead view video is generated based on these surrounding videos.
  • FIG. 4 is a block diagram showing the configuration of the first embodiment of the vehicle surrounding moving object detection system of the present invention.
  • the first embodiment of the vehicle surrounding moving object detection system includes a vehicle surrounding moving object detection device 100 and a surrounding image input unit 101.
  • the ambient video input unit 101 includes a plurality of cameras 301 to 304 that respectively capture the situation around the dump truck 1.
  • the surrounding image input unit 101 transmits a plurality of surrounding images 401 to 404, which are photographing results, to the vehicle surrounding moving object detection device 100.
  • the vehicle surrounding moving object detection device 100 includes a moving object detection unit 102, a region information holding unit 103, a composite video construction unit 104, and an output unit 105.
  • the moving object detection unit 102 includes a plurality of surrounding images 401 to 404 transmitted by the surrounding image input unit 101 and a detection result display non-display region that is a non-detection / non-display region from the region information holding unit 103 (details will be described later). ) And the moving object detection process is performed in a region excluding the detection result display non-target region from the surrounding video.
  • the moving object detection unit 102 outputs an information request signal to the area information holding unit 103 and outputs a detection position coordinate signal indicating the detected position to the composite video construction unit 104 when a moving object is detected.
  • the area information holding unit 103 holds an information signal of a detection result display non-target area, and outputs the held information signal to the moving object detection unit 102 in response to a request from the moving object detection unit 102.
  • the detection result display non-target area refers to an area in which a detection result indicating that a moving object is detected is not displayed even if a moving object is detected in the surrounding images 401 to 404.
  • a part of the host vehicle 1 such as the front wheel 4A and the rear wheel 4B of the vehicle body 2 is set.
  • the composite video constructing unit 104 receives a plurality of surrounding images 401 to 404 transmitted from the surrounding image input unit 101 and a detection position coordinate signal indicating the position of the moving object from the moving object detection unit 102.
  • the composite video constructing unit 104 cuts out and transforms a necessary part from the surrounding video input to generate the composite overhead video, combines them, and based on the input detection position coordinate signal, the detection position of the moving object A composite bird's-eye view image highlighting the region is generated and output to the output unit 105.
  • the output unit 105 includes a display or the like, and displays the synthesized overhead view video input from the synthesized video construction unit 104 on the display or the like.
  • FIG. 5 is a flowchart showing the processing contents in the first embodiment of the vehicle surrounding moving object detection system of the present invention
  • FIG. 6 shows the detection result display in the first embodiment of the vehicle surrounding moving object detection system of the present invention
  • FIG. 7 is a conceptual diagram showing an example of an extraction region from a surrounding image in the first embodiment of the vehicle surrounding moving object detection system of the present invention
  • FIG. 8 is a vehicle of the present invention
  • FIG. 9 is a conceptual diagram showing an example of a composite bird's-eye view image and a moving object detection result display in the first embodiment of the surrounding moving object detection system
  • FIG. 9 is a conceptual diagram showing an example of a composite bird's-eye view image and a moving object detection result display in the first embodiment of the surrounding moving object detection system
  • FIG. 9 is a diagram in the first embodiment of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram explaining a moving object detection. 5 to 9, the same reference numerals as those shown in FIGS. 1 to 4 are the same parts, and detailed description thereof is omitted.
  • the vehicle surrounding moving object detection device 100 performs surrounding image input (step S101). Specifically, ambient images 401 to 404 photographed by the ambient image input unit 101 are input.
  • the vehicle surrounding moving object detection device 100 performs the exclusion process on the non-target region (step S102). Specifically, based on the information on the detection result display non-target area held by the area information holding unit 103 shown in FIG. 4, even if a moving object is detected, an area where the detection result is not displayed is set.
  • a portion surrounded by a broken line is set as a detection result display non-target area.
  • 501 indicates a non-target area for display of the front detection result
  • 502 in the left direction video 402 indicates a non-target area for display of the detection result of the left direction
  • 503 in the rear video 403
  • Reference numeral 504 denotes a right direction detection result display non-target area.
  • the vehicle surrounding moving object detection apparatus 100 executes construction of a composite image (step S103). Specifically, the composite video construction unit 104 shown in FIG. 4 cuts a part from the front video 401, the left video 402, the rear video 403, and the right video 404 and arranges them so as to touch the boundary, thereby providing the dump truck 1 Display the surroundings of the front, back, left and right.
  • a front extraction region 601, a left direction extraction region 602, a rear extraction region 603, and a right direction extraction region 604 are provided inside each of the surrounding images 401 to 404. Based on the characteristics of the wide-angle camera, these extraction areas are reflected from the area starting from just below the dump truck 1 to a portion separated by a certain distance, and the extraction area in the video input from the adjacent camera and the camera. The range is set based on the fact that the extracted area in the input video is adjacent without overlapping.
  • a synthesized overhead image 605 shown in FIG. 8 is constructed.
  • the forward extraction area 601 is converted into a forward converted image 606.
  • the left direction extraction region 602 is converted into a left direction conversion image 607
  • the rear extraction region 603 is converted into a rear conversion image 608, and the right direction extraction region 604 is converted into a right direction conversion image 609.
  • these converted images 606 to 609 are sequentially arranged so as to be adjacent to each other, and in the present embodiment, an image of the bird's-eye view of the periphery of the dump truck 1 is represented.
  • the own vehicle icon 610 is inserted and synthesized in the central portion where the video does not exist to indicate the region where the vehicle body 2 of the dump truck 1 exists.
  • the vehicle surrounding moving object detection apparatus 100 executes a detection process (step S104). Specifically, the moving object detection unit 102 shown in FIG. 4 performs a moving object detection process.
  • the images to be detected are the forward video 401, the left video 402, the rear video 403, and the right video 404 input from the surrounding video input unit 101. Further, in each video, the front detection result display non-display area 501, the detection results in the left direction detection result display non-target area 502, the rear detection result display non-target area 503, and the right direction detection result display non-target area 504 are excluded.
  • FIG. 9 shows detection of a moving object in the left-direction image 402, and in order from the top, (a) an image at a time t 0 ⁇ 1 that is a time before a certain specific time t 0 , and (b) a certain specific An image at time t 0 , (c) an image showing a moving object initial detection result, and (d) an image showing a corrected detection result are shown.
  • (a) shows a state in which the moving object 702 at time t 0 ⁇ 1 is photographed on the left direction image 402, and (b) shows the moving object 701 at the specific time t 0 in the left direction. A state of being photographed on the video 402 is shown.
  • These moving objects 701 and 702 are the same, and the state of moving with the passage of time is captured by the left camera 303.
  • the moving object 701 at a particular time t 0, the moving object 702 at time t 0 -1 is a time before the specific time t 0, is present in the interior of the left extraction region 602
  • it is a target for moving object detection, and in the case of a moving object, it is a target for display.
  • the moving object detection unit 102 shown in FIG. 4 compares these images input from the surrounding image input unit 101, and identifies a site where a change has occurred in the image with time. Specifically, a part is specified like a detection result 703 in the left direction indicated by a square in FIG.
  • the moving object detection unit 102 calculates and holds coordinate information on the left direction image 402 of the detection result 703 in the left direction. It should be noted that such a detection result cannot be obtained when there is no change in the image.
  • the vehicle surrounding moving object detection apparatus 100 determines whether or not a moving object is detected by the detection processing in (Step S104) (Step S105). If a moving object is detected, the process proceeds to (Step S106). Otherwise, the process proceeds to (Step S108).
  • the vehicle surrounding moving object detection device 100 calculates the coordinates of the detection location (step S106). Specifically, the moving object detection unit 102 shown in FIG. 4 converts the part of the moving object obtained in the front video 401, the left video 402, the rear video 403, and the right video 404 into coordinate information, and combines it. Coordinate information in the overhead view image 605 is calculated. The conversion added to the coordinate information is the same as that used when constructing the composite bird's-eye view image 605 in (Step S103).
  • the moving object 701 at the specific time t 0 exists outside the left direction extraction region 602 although it is inside the left direction image 402.
  • the coordinate information of the moving object 701 after conversion indicates the outside of the left-direction converted image 607 shown in FIG. Therefore, the moving object detection unit 102 calculates a part inside the left direction extraction region 602 closest to the left direction detection result 703 at the specific time t 0 shown in FIG. 9C, and the calculated part is shown in FIG.
  • the corrected detection result 704 in the left direction at the specific time t 0 is used.
  • the vehicle surrounding moving object detection apparatus 100 performs enhancement setting of the detection location (step S ⁇ b> 107). Specifically, the composite video constructing unit 104 sets highlighting in order to indicate to the user that the moving object is detected.
  • the detection result 801 after coordinate conversion at the specific time t 0 is emphasized by adding a conversion to the corrected detection result 704 in the left direction at the specific time t 0 shown in FIG. Display is set.
  • the left direction of the corrected detection results 704 at a specific time t 0 is, in the interior of the left extraction region 602, and the detection result of the left at a particular time t 0 It exists at a position closest to 703. Then, the detection result 801 after the coordinate conversion at the specific time t 0 shown in FIG. 8 becomes a moving object indicated by the detection result 703 in the left direction at the specific time t 0 on the left-direction converted video 607 on the synthesized overhead image 605. It exists in the nearest position.
  • the vehicle surrounding moving object detection apparatus 100 outputs a composite image (step S ⁇ b> 108). Specifically, the composite video construction unit 104 outputs the composite overhead video 605 to the output unit 105. In the above-described processing, when a moving object is detected, the output unit 105 outputs a composite bird's-eye view image 605 that is highlighted as in the detection result 801 after coordinate conversion at the specific time t 0 . .
  • the vehicle surrounding moving object detection apparatus 100 determines whether the detection and output of the moving object is completed (step S109). If these processes are completed, the process ends. Otherwise, the process returns to (Step S101).
  • the detection and display processing of the moving object in the present embodiment is performed, it is possible to detect the presence or absence of the moving object in the part that is not displayed on the composite overhead view image 605, and its existence. By highlighting, it is possible to notify the user.
  • the vehicle surrounding moving object detection system of the present invention it is included in the portion displayed as the synthesized bird's-eye view image 605 for the user in the situation around the host vehicle 1. Even when a moving object is detected at a non-existing location, the detection result is notified to the user, so that the user can quickly grasp the presence and positional relationship of the object to be noted in the operation of the host vehicle 1. As a result, it is possible to improve the efficiency of confirmation work for ensuring work safety, and to avoid contact with an object such as a person or another vehicle. As a result, the operational efficiency of the entire work can be improved.
  • FIG. 10 is a conceptual diagram showing an example of a composite bird's-eye view image and a moving object detection result display in the second embodiment of the vehicle surrounding moving object detection system of the present invention.
  • the same reference numerals as those shown in FIG. 1 to FIG. 9 are the same parts, and detailed description thereof will be omitted.
  • the vehicle surrounding moving object detection system in the present embodiment has a configuration and an operation method that are substantially the same as those in the first embodiment.
  • the second embodiment when a moving object that is located below the host vehicle 1 and is not displayed on the composite bird's-eye view image 605 is detected, the user is notified by displaying a message to that effect. Different from the first embodiment.
  • Step S106 the processing of (Step S106) and (Step S107) in the flowchart showing the processing contents shown in FIG. 5 in the first embodiment, and the composite overhead view video shown in FIG. 8 in the first embodiment. Since the detection result display of the moving object in is different, this part will be described.
  • the coordinates of the detection location are calculated in the same manner as in the first embodiment, and whether or not the moving object detected based on this information is below the host vehicle 1. Determine whether. Specifically, when all of the following conditions are satisfied in the moving object detection unit 102 shown in FIG. 4, it is determined that the moving object exists below the host vehicle 1. (1) A moving object is included in any of the front video 401, the left video 402, the rear video 403, and the right video 404. (2) A moving object exists below the lower side indicating each of the front extraction area 601, the left direction extraction area 602, the rear extraction area 603, and the right direction extraction area 604 shown in FIG.
  • the detection location is emphasized.
  • the composite video constructing unit 104 performs settings for indicating the detection of the moving object to the user.
  • the message character string “A moving object has been detected in the lower part of the own vehicle body” is output to the output unit 105.
  • the message character string is set not to be output to the output unit 105.
  • FIG. 10 is a display example on the output unit 105 made by these processes, and shows a state in which a moving object is detected in the lower part of the host vehicle 1.
  • a detection message 901 indicating that a moving object has been detected is displayed on the lower part of the host vehicle 1 on the synthesized overhead image 605 displayed on the output unit 105. Accordingly, it is possible to notify the user of the state of the moving object existing below the host vehicle 1 that is not displayed in any of the forward converted video 606, the left converted video 607, the backward converted video 608, and the right converted video 609. it can.
  • FIG. 11 is a conceptual diagram showing an example of a composite overhead view image and a moving object detection result display in the third embodiment of the vehicle surrounding moving object detection system of the present invention.
  • the same reference numerals as those shown in FIGS. 1 to 10 are the same parts, and detailed description thereof is omitted.
  • the vehicle surrounding moving object detection system in the present embodiment has a configuration and an operation method that are substantially the same as those in the first embodiment.
  • a moving object that is below the host vehicle 1 and is not displayed on the composite overhead view image 605 is detected, it is detected at a position on the host vehicle icon 610 corresponding to the detected position.
  • the point which notifies a user by displaying a result differs from a 1st embodiment.
  • Step S106 the processing of (Step S106) and (Step S107) in the flowchart showing the processing contents shown in FIG. 5 in the first embodiment, and the composite overhead view video shown in FIG. 8 in the first embodiment. Since the detection result display of the moving object in is different, this part will be described.
  • step S106 but calculates the coordinates of the same detected position of the first embodiment, this time, without correcting the detection result, the left direction at a particular time t 0
  • the detection result 703 is used in subsequent processing. Note that the detection result 703 in the left direction at the specific time t 0 is present at the lower part of the host vehicle 1 and behind the front wheel 4A as shown in FIG. 7C.
  • Step S107 the detection location is emphasized. Specifically, the composite video constructing unit 104 sets highlighting in order to indicate to the user that the moving object is detected.
  • the detection result 703 in the left direction at the specific time t 0 exists outside the left direction extraction region 602. Therefore, when coordinate conversion is performed, the detection result 703 exists in the display region of the vehicle icon 610 in FIG. Will do. For this reason, in FIG. 10, the detection result 1001 after the coordinate conversion at the specific time t 0 is displayed so as to be superimposed on the vehicle icon 610. As a result, the state of the moving object existing below the host vehicle 1 that is not displayed in any of the forward converted video 606, the left converted video 607, the backward converted video 608, and the right converted video 609, including the approximate position. The user can be notified.
  • FIG. 12 is a block diagram showing the configuration of the fourth embodiment of the vehicle surrounding moving object detection system of the present invention
  • FIG. 13 is a table showing an example of the table in the emphasis information holding unit shown in FIG. 12, and FIG. FIG.
  • FIG. 15 is a flowchart showing the processing contents in the fourth embodiment of the vehicle surrounding moving object detection system of the invention, and FIG. 15 is an image of the moving object reflected in the fourth embodiment of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram which shows an example. 12 to 15, the same reference numerals as those shown in FIGS. 1 to 11 are the same parts, and detailed description thereof is omitted.
  • the vehicle surrounding moving object detection system in the present embodiment has a configuration and an operation method that are substantially the same as those in the first embodiment.
  • the fourth embodiment when the moving object is present at a position closer to the host vehicle 1, the first point is that the detection result of the moving object is emphasized and displayed to the user. Different from the embodiment.
  • the vehicle surrounding moving object detection device 100 ⁇ / b> A further includes an emphasis information holding unit 106.
  • the emphasis information holding unit 106 holds, as a table, for example, a color information signal indicating the degree of emphasis in the emphasis display according to the distance between the host vehicle 1 and the moving object in the video displayed by the output unit 105. Output to the video construction unit 104.
  • FIG. 1 An example of the table held by the emphasis information holding unit 106 is shown in FIG.
  • the distance L between the moving object and the vehicle 1 in the surrounding images 401 to 404 is 300 pixels or more
  • green is used as the highlight color.
  • the highlight color is yellow
  • the highlight color is red.
  • the emphasis information holding unit 106 can store other types of tables.
  • step S206 is added between (step S106) and (step S107). Others are the same.
  • the vehicle surrounding moving object detection device 100A calculates the coordinates of the detection location in (Step S106) in the same manner as in the first embodiment, and then the region contents based on the coordinates of the detection location. Is set (step S206). Specifically, the composite video construction unit 104 shown in FIG. 12 calculates the distance L between the moving object and the host vehicle 1 in the surrounding videos 401 to 404, and highlights the color in light of the table held by the highlight information holding unit 106. Set.
  • the composite video construction unit 104 sets the highlighted color to yellow in (Step S206). Set to.
  • the vehicle surrounding moving object detection device 100 ⁇ / b> A performs enhancement setting of the detection location (step S ⁇ b> 107). Specifically, the composite video construction unit 104 sets the highlight color to yellow. Thereafter, in the composite video output in (Step S108), the moving object 1001 is highlighted in yellow with respect to the composite overhead image 605.
  • the presence of the moving object 1001 is highlighted and the highlighted color is changed according to the distance.
  • the person can quickly and intuitively grasp the distance L between the moving object 1001 and the host vehicle 1 together with the presence of the moving object 1001. As a result, more safety can be ensured.
  • FIG. 16 is a table showing an example of a table in the emphasis information holding unit in the fifth embodiment of the vehicle surrounding moving object detection system of the present invention
  • FIG. 17 shows the fifth embodiment of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram which shows an example of the image
  • the vehicle surrounding moving object detection system in the present embodiment has substantially the same configuration and operation method as in the fourth embodiment.
  • the table held by the emphasis information holding unit 106 is different from that in the fourth embodiment.
  • FIG. 16 shows an example of a table held by the emphasis information holding unit 106 of the present embodiment
  • FIG. 17 shows an example of an image showing a moving object.
  • an object photographed above the surrounding image is located farther from the vehicle body 2 of the host vehicle 1 and is photographed below the surrounding image. The closer the object is, the closer to the body 2 of the host vehicle 1 is.
  • the y-coordinate which is the vertical coordinate in each surrounding image is defined as 0, with the upper end of each surrounding image being 0, and the numerical value is increased as it moves downward. For this reason, it shows that it is closer to the own vehicle 1 so that the numerical value of y coordinate is large.
  • the vertical coordinate is 0.
  • the highlighted color is green.
  • the highlight color is yellow, and when the coordinates are 300 pixels or more, the highlight color is red.
  • the vehicle surrounding moving object detection device 100A calculates the coordinates of the detection location in (step S106) as in the first embodiment, and these coordinates are the same as those of the moving object 1001 described above. It has been converted to a y-coordinate value.
  • the vehicle surrounding moving object detection device 100A performs region content enhancement setting based on the y-coordinate of this detection location (step S206). Specifically, the composite video construction unit 104 calculates the y-coordinate v of the moving object in the surrounding videos 401 to 404, and sets the highlight display color against the table held by the highlight information holding unit 106.
  • the composite image constructing unit 104 sets the highlight color to yellow in (Step S206). .
  • the vehicle surrounding moving object detection device 100 ⁇ / b> A performs enhancement setting of the detection location (step S ⁇ b> 107). Specifically, the composite video construction unit 104 sets the highlight color to yellow. Thereafter, in the composite video output in (Step S108), the moving object 1001 is highlighted in yellow with respect to the composite overhead image 605.
  • a table is used to set the highlight color from the position of the moving object 1001, but the present invention is not limited to this. For example, you may set using a numerical formula etc.
  • this invention is not limited to the above-mentioned Example, Various modifications are included.
  • the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described.
  • Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit.
  • Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)

Abstract

Provided is a vehicle-periphery-moving-object detection system for detecting a moving object in a location not displayed in a synthesized overhead video such as underneath the chassis of a large construction vehicle, and capable of reporting the results thereof to a user. A vehicle-periphery-moving-object detection system equipped with a peripheral-video input unit for capturing a plurality of videos around a vehicle, a synthesized-video-building unit for extracting regions for synthesized-video building from the plurality of peripheral videos captured by the peripheral-video input unit, and building a synthesized overhead video by synthesizing the plurality of extracted synthesized-video-building regions, and an output unit for presenting the synthesized overhead video to the user, the vehicle-periphery-moving-object detection system also having a moving-object-detection processing unit for performing the moving-object detection processing on the plurality of peripheral videos, and reporting the detection results to the user by outputting the same to the synthesized overhead video, even when a moving object is detected somewhere other than in the regions expressed in the synthesized overhead video.

Description

車両周囲移動物体検知システムMoving object detection system around the vehicle
　本発明は、車両周囲移動物体検知システムに係り、さらに詳しくは、鉱山で作業を行う大型ダンプトラック等の建設機械において、車両近傍を移動する物体との接触を回避するための車両周囲移動物体検知システムに関する。 The present invention relates to a vehicle surrounding moving object detection system, and more particularly, to a vehicle surrounding moving object detection for avoiding contact with an object moving in the vicinity of a vehicle in a construction machine such as a large dump truck working in a mine. About the system.
　本技術分野の背景技術として、特開２０１０－１４７５２３号公報（特許文献１）がある。この公報には、“共通領域における立体物の消失を回避しつつ、監視者が注意すべき車両周囲の立体物を把握しやすい合成俯瞰画像を生成すること。”が記載されている。 As a background art in this technical field, there is JP 2010-147523 A (Patent Document 1). This publication describes that “a composite bird's-eye view image that can easily grasp a three-dimensional object around a vehicle to be watched by a supervisor while avoiding the disappearance of the three-dimensional object in the common area” is described.
特開２０１０－１４７５２３号公報JP 2010-147523 A
　従来の車両周囲検知システムでは、複数のカメラが撮影する自車両周囲の映像から一部を抽出し、形状を変換、合成することにより、合成俯瞰画像を生成して、利用者に自車両周囲の状況を示すとともに、利用者が注目すべき物体（立体物）が映像中に存在する場合はこれを強調表示することで利用者に注意を促していた。 In the conventional vehicle surrounding detection system, a part of the image around the host vehicle captured by a plurality of cameras is extracted, and the shape is converted and synthesized to generate a composite overhead image, and the user is notified of the surroundings of the host vehicle. In addition to showing the situation, if an object (three-dimensional object) that the user should pay attention to is present in the video, the user is warned by highlighting it.
　一方、鉱山向けのダンプトラックなど大型の建設機械は、自車両の車体下部にも広い空間が存在する。したがって、建設機械の安全な作業を確保するためには、車体下部の空間に人物や車両が立ち入っていないことを確認することが重要となる。 On the other hand, large construction machines such as dump trucks for mines have a large space under the body of the vehicle. Therefore, in order to ensure safe work of the construction machine, it is important to confirm that no person or vehicle enters the space below the vehicle body.
　しかしながら、上述した従来のシステムでは、合成俯瞰画像を生成して自車両周囲の状況を表示すると共に、表示した箇所における物体（立体物）の検知を行うため、自車両の車体下部などの合成俯瞰画像に表示されない箇所に人物や他車両が立ち入った場合、立体物の検知がなされず利用者に注意を促すことができない。このため、作業上の安全を損なう虞があった。 However, in the above-described conventional system, a composite overhead image is generated to display the situation around the host vehicle, and an object (a three-dimensional object) is detected at the displayed location. When a person or another vehicle enters a place that is not displayed in the image, the three-dimensional object is not detected and the user cannot be alerted. For this reason, there existed a possibility of impairing operational safety.
　本発明は上述の事柄に基づいてなされたものであって、その目的は、大型の建設機械の自車両の車体下部などの合成俯瞰映像として表示されない箇所においても移動物体の検知を行うと共に、その結果を利用者に報知できる車両周囲移動物体検知システムを提供するものである。 The present invention has been made based on the above-mentioned matters, and its purpose is to detect a moving object even in a place that is not displayed as a composite bird's-eye view image such as a lower part of a vehicle body of a large construction machine. A vehicle surrounding moving object detection system capable of notifying the user of the result is provided.
　上記の目的を達成するために、第１の発明は、自車両の周囲の複数の映像を撮影する周囲映像入力部と、前記周囲映像入力部が撮影した複数の周囲映像から合成映像構成用領域を抽出して、抽出した複数の合成映像構成用領域を合成することで合成俯瞰映像を構築する合成映像構築部と、前記合成俯瞰映像を利用者に提示する出力部とを備えた車両周囲移動物体検知システムにおいて、前記複数の周囲映像に関して移動物体の検知処理を行う移動物体検知処理部とを有し、前記移動物体検知処理部は、前記合成俯瞰映像に示す領域以外で移動物体を検知した場合にも検知結果を前記合成俯瞰映像に出力して前記利用者に報知するものとする。 In order to achieve the above object, a first invention provides a surrounding image input unit that captures a plurality of images around the host vehicle, and a composite image configuration area from the plurality of surrounding images captured by the surrounding image input unit. Vehicle surrounding movement provided with a composite video construction unit that constructs a composite overhead view video by combining a plurality of extracted composite video composition areas and an output unit that presents the composite overhead view video to a user The object detection system includes a moving object detection processing unit that performs a detection process of the moving object with respect to the plurality of surrounding images, and the moving object detection processing unit detects a moving object in a region other than the region shown in the composite overhead image. Even in this case, the detection result is output to the synthesized bird's-eye view image to notify the user.
　また、第２の発明は、第１の発明において、移動物体を検知したとの検知結果を表示させない領域である検知結果表示対象外領域の情報を保持する領域情報保持部と、前記領域情報保持部から前記検知結果表示対象外領域の情報を取込み、前記複数の周囲映像から前記検知結果表示対象外領域を除外した映像に関して移動物体の検知処理を行う移動物体検知処理部とを更に備え、前記移動物体検知処理部は、前記合成俯瞰映像に示す領域以外で移動物体を検知した場合にも検知結果を前記合成俯瞰映像に出力して前記利用者に報知することを特徴とする。 According to a second aspect of the present invention, in the first aspect of the present invention, a region information holding unit that holds information of a detection result display non-target region that is a region where a detection result indicating that a moving object has been detected is not displayed, and the region information holding A moving object detection processing unit that takes in the information of the detection result display non-target region from the unit, and performs a moving object detection process on the video that excludes the detection result display non-target region from the plurality of surrounding images, The moving object detection processing unit outputs a detection result to the synthesized bird's-eye video and notifies the user even when a moving object is detected outside the area shown in the synthesized bird's-eye video.
　更に、第３の発明は、第２の発明において、前記検知結果表示対象外領域は、前記複数の周囲映像に映りこんだ前記自車両の一部であることを特徴とする。 Furthermore, the third invention is characterized in that, in the second invention, the detection result display non-target area is a part of the own vehicle reflected in the plurality of surrounding images.
　また、第４の発明は、第２の発明において、前記移動物体検知処理部は、前記自車両の下部で移動物体を検知した場合に、検知結果を前記合成俯瞰映像にメッセージ出力して前記利用者に報知することを特徴とする。 In a fourth aspect based on the second aspect, when the moving object detection processing unit detects a moving object in a lower part of the host vehicle, the detection result is output as a message to the synthesized overhead view image and used. It is characterized by notifying a person.
　更に、第５の発明は、第２の発明において、前記移動物体検知処理部は、前記自車両の下部で移動物体を検知した場合に、検知結果を前記合成俯瞰映像における自車両を示す位置に描画出力して前記利用者に報知することを特徴とする。 Further, in a fifth aspect based on the second aspect, when the moving object detection processing unit detects a moving object in a lower portion of the host vehicle, the detection result is set to a position indicating the host vehicle in the synthesized overhead view image. The drawing is output and notified to the user.
　また、第６の発明は、第２の発明において、前記自車両と検知した移動物体との距離に応じて、強調表示の強調の度合いを示す色の情報を保持する強調情報保持部を更に備え、前記移動物体検知処理部は、前記強調情報保持部から前記強調表示での強調の度合いを示す色の情報を取込み、前記自車両と検知した移動物体との距離から強調表示色を設定し、検知結果である前記移動物体を設定した前記強調表示色で前記合成俯瞰映像上に表示して前記利用者に報知することを特徴とする。 According to a sixth aspect, in the second aspect, the information processing apparatus further includes an emphasis information holding unit that holds color information indicating a degree of emphasis display according to a distance between the host vehicle and the detected moving object. The moving object detection processing unit takes in color information indicating the degree of emphasis in the emphasis display from the emphasis information holding unit, and sets an emphasis display color from a distance between the host vehicle and the detected moving object, The moving object that is a detection result is displayed on the composite bird's-eye view image in the highlighted display color that is set, and is notified to the user.
　本発明によれば、自車両周囲の状況において、利用者に対して合成俯瞰映像として表示している部分には含まれない箇所で移動物体を検知した場合でも、検知結果を利用者に報知するので、利用者は、自車両の作業上注意すべき対象物の存在と位置関係を迅速に把握できる。これにより、作業上の安全を確保するための確認作業を効率化できると共に、人物や他車両といった対象物との接触を回避することができる。この結果、作業全体の運用効率を向上することができる。 According to the present invention, even when a moving object is detected at a location that is not included in the portion displayed as a composite bird's-eye view image for the user in a situation around the host vehicle, the detection result is notified to the user. Therefore, the user can quickly grasp the existence and the positional relationship of the object to be noted in the work of the own vehicle. As a result, it is possible to improve the efficiency of confirmation work for ensuring work safety, and to avoid contact with an object such as a person or another vehicle. As a result, the operational efficiency of the entire work can be improved.
本発明の車両周囲移動物体検知システムの第１の実施の形態を備えたダンプトラックを示す側面図である。It is a side view showing a dump truck provided with a 1st embodiment of a vehicle circumference moving object detection system of the present invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態における周囲映像入力部を構成するカメラの配置を説明する概念図である。It is a conceptual diagram explaining arrangement | positioning of the camera which comprises the surrounding image input part in 1st Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態を構成するカメラが撮影した映像を示す概念図である。It is a conceptual diagram which shows the image | video which the camera which comprises 1st Embodiment of the vehicle surrounding moving object detection system of this invention image | photographed. 本発明の車両周囲移動物体検知システムの第１の実施の形態の構成を示すブロック図である。It is a block diagram which shows the structure of 1st Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態における処理内容を示すフローチャート図である。It is a flowchart figure which shows the processing content in 1st Embodiment of the vehicle periphery moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態における検知結果表示対象外領域の例を示す概念図である。It is a conceptual diagram which shows the example of the detection result display non-target area | region in 1st Embodiment of the vehicle periphery moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態における周囲映像からの抽出領域の例を示す概念図である。It is a conceptual diagram which shows the example of the extraction area | region from the surrounding image | video in 1st Embodiment of the vehicle periphery moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態における合成俯瞰映像と移動物体の検知結果表示の一例を示す概念図である。It is a conceptual diagram which shows an example of the synthetic | combination bird's-eye view image and the detection result display of a moving object in 1st Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第１の実施の形態における移動物体検知を説明する概念図である。It is a conceptual diagram explaining the moving object detection in 1st Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第２の実施の形態における合成俯瞰映像と移動物体の検知結果表示の一例を示す概念図である。It is a conceptual diagram which shows an example of the synthetic | combination bird's-eye view image and the detection result display of a moving object in 2nd Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第３の実施の形態における合成俯瞰映像と移動物体の検知結果表示の一例を示す概念図である。It is a conceptual diagram which shows an example of the synthetic | combination bird's-eye view image and the detection result display of a moving object in 3rd Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第４の実施の形態の構成を示すブロック図である。It is a block diagram which shows the structure of 4th Embodiment of the vehicle surrounding moving object detection system of this invention. 図１２に示す強調情報保持部におけるテーブルの一例を示す表図である。It is a table | surface figure which shows an example of the table in the emphasis information holding | maintenance part shown in FIG. 本発明の車両周囲移動物体検知システムの第４の実施の形態における処理内容を示すフローチャート図である。It is a flowchart figure which shows the processing content in 4th Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第４の実施の形態において移動物体が映った映像の一例を示す概念図である。It is a conceptual diagram which shows an example of the image | video which showed the moving object in 4th Embodiment of the vehicle surrounding moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第５の実施の形態における強調情報保持部におけるテーブルの一例を示す表図である。It is a table | surface figure which shows an example of the table in the emphasis information holding | maintenance part in 5th Embodiment of the vehicle periphery moving object detection system of this invention. 本発明の車両周囲移動物体検知システムの第５の実施の形態において移動物体が映った映像の一例を示す概念図である。It is a conceptual diagram which shows an example of the image | video which showed the moving object in 5th Embodiment of the vehicle surrounding moving object detection system of this invention.
　以下、鉱山等で採掘した砕石や鉱物等を運搬する大型の運搬車両であるダンプトラックに適用した場合を例にとって本発明の実施の形態を図面を用いて説明する。なお、本発明の適用はダンプトラックに限定されるものではない。
＜実施例１＞
　図１は本発明の車両周囲移動物体検知システムの第１の実施の形態を備えたダンプトラックを示す側面図である。　
　図１に示すダンプトラック（車両）１は、頑丈なフレーム構造で形成された車体２と、車体２上に起伏可能に搭載されたベッセル（荷台）３と、車体２に装着された左前輪４Ａ（L）及び左後輪４Ｂ（L）を主に備えている。 Embodiments of the present invention will be described below with reference to the drawings, taking as an example a case where the present invention is applied to a dump truck that is a large transport vehicle that transports crushed stones, minerals, and the like mined at a mine or the like. The application of the present invention is not limited to the dump truck.
<Example 1>
FIG. 1 is a side view showing a dump truck provided with a first embodiment of a vehicle surrounding moving object detection system of the present invention.
A dump truck (vehicle) 1 shown in FIG. 1 includes a vehicle body 2 formed with a sturdy frame structure, a vessel (loading platform) 3 mounted on the vehicle body 2 so as to be able to undulate, and a left front wheel 4A mounted on the vehicle body 2. (L) and left rear wheel 4B (L) are mainly provided.
　車体２には、後輪４Ｂを駆動するエンジン（図示せず）が配設されている。エンジンは、例えば、エンジン制御装置（以下ＥＣＵという）を有し、ＥＣＵからの指令信号によって、供給される燃料流量が制御されることで、その回転数が制御されている。 The vehicle body 2 is provided with an engine (not shown) that drives the rear wheels 4B. The engine has, for example, an engine control device (hereinafter referred to as ECU), and the number of revolutions thereof is controlled by controlling the flow rate of fuel supplied by a command signal from the ECU.
　ベッセル３は、砕石物等の荷物を積載するために設けられた容器であり、ピン結合部５等を介して車体２に対して起伏可能に連結されている。ベッセル３の下部には、車両の幅方向に所定の間隔を介して２つの起伏シリンダ６が設置されている。起伏シリンダ６に圧油が供給・排出されると、起伏シリンダ６が伸長・縮短してベッセル３が起伏される。また、ベッセル３の前側上部には庇部７が設けられている。 The vessel 3 is a container provided to load a load such as a crushed stone, and is connected to the vehicle body 2 via a pin coupling portion 5 so as to be raised and lowered. Two undulation cylinders 6 are installed at a lower portion of the vessel 3 with a predetermined interval in the width direction of the vehicle. When pressure oil is supplied to and discharged from the undulation cylinder 6, the undulation cylinder 6 extends and contracts, and the vessel 3 is undulated. In addition, a collar portion 7 is provided on the upper front side of the vessel 3.
　庇部７は、その下側（すなわち車体２の前部）に設置された運転室８を岩石等の飛散物から保護するとともに、車両転倒時等に運転室８を保護する機能を有している。運転室８の内部には、車両周囲移動物体検知システムを構成する制御装置１００（図４参照）と、操舵用のハンドル（図示せず）と、アクセルペダル及びブレーキペダル等（図示せず）とが設置されている。 The heel portion 7 has a function of protecting the cab 8 installed on the lower side thereof (that is, the front portion of the vehicle body 2) from scattered objects such as rocks and protecting the cab 8 when the vehicle falls. Yes. Inside the cab 8 is a control device 100 (see FIG. 4) constituting a vehicle surrounding moving object detection system, a steering handle (not shown), an accelerator pedal, a brake pedal and the like (not shown). Is installed.
　図２は本発明の車両周囲移動物体検知システムの第１の実施の形態における周囲映像入力部を構成するカメラの配置を説明する概念図、図３は本発明の車両周囲移動物体検知システムの第１の実施の形態を構成するカメラが撮影した映像を示す概念図である。 FIG. 2 is a conceptual diagram for explaining the arrangement of cameras constituting the surrounding image input unit in the first embodiment of the vehicle surrounding moving object detection system of the present invention, and FIG. 3 is a diagram of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram which shows the image | video which the camera which comprises 1 embodiment image | photographed.
　図２において、ダンプトラック１の車体２の前側面と後側面とには、ダンプトラック１の前方を広角レンズで撮影する前方カメラ３０１と、ダンプトラック１の後方を広角レンズで撮影する後方カメラ３０３とが設けられている。また、車体２の左側面と右側面とには、ダンプトラック１の左方向を広角レンズで撮影する左方向カメラ３０２と、ダンプトラック１の右方向を広角レンズで撮影する右方向カメラ３０４とが設けられている。また、ダンプトラック１の車体２には、右前輪４Ａ（R）と左後輪４Ｂ（R）とが装着されている。 In FIG. 2, a front camera 301 that captures the front of the dump truck 1 with a wide-angle lens and a rear camera 303 that captures the rear of the dump truck 1 with a wide-angle lens are provided on the front and rear sides of the vehicle body 2 of the dump truck 1. And are provided. Further, on the left side surface and the right side surface of the vehicle body 2, a left direction camera 302 that captures the left direction of the dump truck 1 with a wide angle lens and a right direction camera 304 that captures the right direction of the dump truck 1 with a wide angle lens are provided. Is provided. The vehicle body 2 of the dump truck 1 is equipped with a right front wheel 4A (R) and a left rear wheel 4B (R).
　図３はこれらのカメラ３０１～３０３が撮影した周囲映像の例を示す。４０１は前方カメラ３０１が撮影した前方映像の例を示す。４０２は左方向カメラ３０２が撮影した左方向映像の例を示す。４０３は後方カメラ３０３が撮影した後方映像の例を示す。４０４は右方向カメラ３０４が撮影した右方向映像の例を示す。 FIG. 3 shows an example of surrounding images taken by these cameras 301-303. Reference numeral 401 denotes an example of a front image captured by the front camera 301. Reference numeral 402 denotes an example of a left direction image captured by the left direction camera 302. Reference numeral 403 denotes an example of a rear image taken by the rear camera 303. Reference numeral 404 denotes an example of a right direction image captured by the right direction camera 304.
　周囲映像４０１～４０４は、それぞれ広角レンズで撮影されるため、各映像の上方に位置する遠方の地平線が湾曲状態に映っている。また、各映像の各下方には手前側の車体２の一部や左右前輪４Ａ（R），４Ａ（L）の一部や左右後輪４Ｂ（R），４Ｂ（L）の一部が映っている。これらの周囲映像を基に合成俯瞰映像が生成される。 Since the surrounding images 401 to 404 are each photographed with a wide-angle lens, a distant horizon located above each image is reflected in a curved state. In addition, a part of the vehicle body 2 on the front side, a part of the left and right front wheels 4A (R) and 4A (L), and a part of the left and right rear wheels 4B (R) and 4B (L) are shown below each image. ing. A synthesized overhead view video is generated based on these surrounding videos.
　図４は本発明の車両周囲移動物体検知システムの第１の実施の形態の構成を示すブロック図である。図４において、車両周囲移動物体検知システムの第１の実施の形態は、車両周囲移動物体検知装置１００と、周囲映像入力部１０１とを備えている。 FIG. 4 is a block diagram showing the configuration of the first embodiment of the vehicle surrounding moving object detection system of the present invention. In FIG. 4, the first embodiment of the vehicle surrounding moving object detection system includes a vehicle surrounding moving object detection device 100 and a surrounding image input unit 101.
　周囲映像入力部１０１は、ダンプトラック１の周囲の様子をそれぞれ撮影する複数のカメラ３０１～３０４を備えている。周囲映像入力部１０１は、撮影結果である複数の周囲映像４０１～４０４を車両周囲移動物体検知装置１００へ送信する。 The ambient video input unit 101 includes a plurality of cameras 301 to 304 that respectively capture the situation around the dump truck 1. The surrounding image input unit 101 transmits a plurality of surrounding images 401 to 404, which are photographing results, to the vehicle surrounding moving object detection device 100.
　車両周囲移動物体検知装置１００は、移動物体検知部１０２と領域情報保持部１０３と合成映像構築部１０４と出力部１０５とを備えている。 The vehicle surrounding moving object detection device 100 includes a moving object detection unit 102, a region information holding unit 103, a composite video construction unit 104, and an output unit 105.
　移動物体検知部１０２は、周囲映像入力部１０１が送信した複数の周囲映像４０１～４０４と、領域情報保持部１０３からの非検知非表示の領域である検知結果表示対象外領域（詳細は後述する）の情報信号とを入力し、周囲映像から検知結果表示対象外領域を除いた領域において移動物体の検知処理を行う。移動物体検知部１０２は、領域情報保持部１０３へ情報要求信号を出力すると共に、移動物体を検知した場合には、検知した位置を示す検知位置座標信号を合成映像構築部１０４へ出力する。 The moving object detection unit 102 includes a plurality of surrounding images 401 to 404 transmitted by the surrounding image input unit 101 and a detection result display non-display region that is a non-detection / non-display region from the region information holding unit 103 (details will be described later). ) And the moving object detection process is performed in a region excluding the detection result display non-target region from the surrounding video. The moving object detection unit 102 outputs an information request signal to the area information holding unit 103 and outputs a detection position coordinate signal indicating the detected position to the composite video construction unit 104 when a moving object is detected.
　領域情報保持部１０３は、検知結果表示対象外領域の情報信号を保持し、移動物体検知部１０２からの要求に応じて保持した情報信号を移動物体検知部１０２に出力する。ここで、検知結果表示対象外領域とは、周囲映像４０１～４０４の中で、移動する物体を検知した場合であっても、移動物体を検知したとの検知結果を表示させない領域をいい、例えば、車体２の前輪４Ａや後輪４Ｂ等の自車両１の一部が設定される。 The area information holding unit 103 holds an information signal of a detection result display non-target area, and outputs the held information signal to the moving object detection unit 102 in response to a request from the moving object detection unit 102. Here, the detection result display non-target area refers to an area in which a detection result indicating that a moving object is detected is not displayed even if a moving object is detected in the surrounding images 401 to 404. A part of the host vehicle 1 such as the front wheel 4A and the rear wheel 4B of the vehicle body 2 is set.
　合成映像構築部１０４は、周囲映像入力部１０１の送信した複数の周囲映像４０１～４０４と、移動物体検知部１０２から移動物体の位置を示す検知位置座標信号とを入力する。合成映像構築部１０４は、合成俯瞰映像を生成するために入力した周囲映像の中から必要な部分を切り出し変形して合成を行い、入力した検知位置の座標信号を基に移動物体の検知位置の部位を強調表示した合成俯瞰映像を生成し、出力部１０５へ出力する。 The composite video constructing unit 104 receives a plurality of surrounding images 401 to 404 transmitted from the surrounding image input unit 101 and a detection position coordinate signal indicating the position of the moving object from the moving object detection unit 102. The composite video constructing unit 104 cuts out and transforms a necessary part from the surrounding video input to generate the composite overhead video, combines them, and based on the input detection position coordinate signal, the detection position of the moving object A composite bird's-eye view image highlighting the region is generated and output to the output unit 105.
　出力部１０５は、ディスプレイ等を備えていて、合成映像構築部１０４から入力した合成俯瞰映像をディスプレイ等に表示する。 The output unit 105 includes a display or the like, and displays the synthesized overhead view video input from the synthesized video construction unit 104 on the display or the like.
　次に、本発明の車両周囲移動物体検知システムの第１の実施の形態における移動物体の検知と表示の処理内容について図５乃至図９を用いて説明する。図５は本発明の車両周囲移動物体検知システムの第１の実施の形態における処理内容を示すフローチャート図、図６は本発明の車両周囲移動物体検知システムの第１の実施の形態における検知結果表示対象外領域の例を示す概念図、図７は本発明の車両周囲移動物体検知システムの第１の実施の形態における周囲映像からの抽出領域の例を示す概念図、図８は本発明の車両周囲移動物体検知システムの第１の実施の形態における合成俯瞰映像と移動物体の検知結果表示の一例を示す概念図、図９は本発明の車両周囲移動物体検知システムの第１の実施の形態における移動物体検知を説明する概念図である。図５乃至図９において、図１乃至図４に示す符号と同符号のものは同一部分であるので、その詳細な説明は省略する。 Next, moving object detection and display processing contents in the first embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to FIGS. FIG. 5 is a flowchart showing the processing contents in the first embodiment of the vehicle surrounding moving object detection system of the present invention, and FIG. 6 shows the detection result display in the first embodiment of the vehicle surrounding moving object detection system of the present invention. FIG. 7 is a conceptual diagram showing an example of an extraction region from a surrounding image in the first embodiment of the vehicle surrounding moving object detection system of the present invention, and FIG. 8 is a vehicle of the present invention. FIG. 9 is a conceptual diagram showing an example of a composite bird's-eye view image and a moving object detection result display in the first embodiment of the surrounding moving object detection system, and FIG. 9 is a diagram in the first embodiment of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram explaining a moving object detection. 5 to 9, the same reference numerals as those shown in FIGS. 1 to 4 are the same parts, and detailed description thereof is omitted.
　図５において、車両周囲移動物体検知装置１００は、周囲映像入力を行う（ステップＳ１０１）。具体的には、周囲映像入力部１０１で撮影した周囲映像４０１～４０４を入力する。 In FIG. 5, the vehicle surrounding moving object detection device 100 performs surrounding image input (step S101). Specifically, ambient images 401 to 404 photographed by the ambient image input unit 101 are input.
　車両周囲移動物体検知装置１００は、対象外領域を除外処理する（ステップＳ１０２）。具体的には、図４に示す領域情報保持部１０３の保持する検知結果表示対象外領域の情報を基に、移動物体を検知した場合であっても、検知結果を表示させない領域を設定する。 The vehicle surrounding moving object detection device 100 performs the exclusion process on the non-target region (step S102). Specifically, based on the information on the detection result display non-target area held by the area information holding unit 103 shown in FIG. 4, even if a moving object is detected, an area where the detection result is not displayed is set.
　この設定例を図６を用いて説明する。図６の周囲映像４０１～４０４において、破線で囲んでいる部分を検知結果表示対象外領域と設定する。前方映像４０１において５０１は前方検知結果表示対象外領域を、左方向映像４０２において５０２は左方向検知結果表示対象外領域を、後方映像４０３において５０３は後方検知結果表示対象外領域を、右方向映像４０４において５０４は右方向検知結果表示対象外領域をそれぞれ示している。 This setting example will be described with reference to FIG. In the surrounding images 401 to 404 in FIG. 6, a portion surrounded by a broken line is set as a detection result display non-target area. In the front image 401, 501 indicates a non-target area for display of the front detection result, 502 in the left direction video 402 indicates a non-target area for display of the detection result of the left direction, 503 in the rear video 403, Reference numeral 504 denotes a right direction detection result display non-target area.
　これらの検知結果表示対象外領域においては、この領域で動作する物体を検知した場合であっても、移動物体を検知したとの表示を行わない。例えば、左方向映像４０２または右方向映像４０４において車体２の前輪４Ａや後輪４Ｂの動作を検知しても、出力部１０５に対して移動物体の検知信号が出力されないので、合成俯瞰映像には移動物体が検知されたとの表示はされない。 In these detection result display non-target areas, even if an object operating in this area is detected, a display indicating that a moving object has been detected is not performed. For example, even if the movement of the front wheel 4A or the rear wheel 4B of the vehicle body 2 is detected in the left direction image 402 or the right direction image 404, the detection signal of the moving object is not output to the output unit 105. There is no indication that a moving object has been detected.
　図５にもどり、車両周囲移動物体検知装置１００は、合成映像の構築を実行する（ステップＳ１０３）。具体的には、図４に示す合成映像構築部１０４において前方映像４０１、左方向映像４０２、後方映像４０３、右方向映像４０４から一部を切り出して境界を接するように並べることにより、ダンプトラック１の前後左右にわたる周囲の状況を表示させる。 Returning to FIG. 5, the vehicle surrounding moving object detection apparatus 100 executes construction of a composite image (step S103). Specifically, the composite video construction unit 104 shown in FIG. 4 cuts a part from the front video 401, the left video 402, the rear video 403, and the right video 404 and arranges them so as to touch the boundary, thereby providing the dump truck 1 Display the surroundings of the front, back, left and right.
　この例を図７及び図８を用いて説明する。図７の周囲映像４０１～４０４において、それぞれの内部に前方抽出領域６０１、左方向抽出領域６０２、後方抽出領域６０３、右方向抽出領域６０４を設ける。これらの抽出領域は、広角カメラの特性を基に、ダンプトラック１の真下から始まって一定距離だけ離れた部分までの間が映ることと、隣接するカメラから入力する映像における抽出領域と当該カメラから入力する映像における抽出領域とが重ならずに隣接することを基に範囲が設定されている。 This example will be described with reference to FIGS. In the surrounding images 401 to 404 in FIG. 7, a front extraction region 601, a left direction extraction region 602, a rear extraction region 603, and a right direction extraction region 604 are provided inside each of the surrounding images 401 to 404. Based on the characteristics of the wide-angle camera, these extraction areas are reflected from the area starting from just below the dump truck 1 to a portion separated by a certain distance, and the extraction area in the video input from the adjacent camera and the camera. The range is set based on the fact that the extracted area in the input video is adjacent without overlapping.
　次に、これら抽出領域６０１～６０４に対して座標変換及び合成を行うことで、図８に示す合成俯瞰映像６０５を構築する。図８の合成俯瞰映像６０５において、前方抽出領域６０１は前方変換映像６０６へと変換されている。同様に、左方向抽出領域６０２は左方向変換映像６０７に、後方抽出領域６０３は後方変換映像６０８に、右方向抽出領域６０４は右方向変換映像６０９へと変換されている。 Next, by performing coordinate conversion and synthesis on these extraction areas 601 to 604, a synthesized overhead image 605 shown in FIG. 8 is constructed. In the synthesized bird's-eye view image 605 in FIG. 8, the forward extraction area 601 is converted into a forward converted image 606. Similarly, the left direction extraction region 602 is converted into a left direction conversion image 607, the rear extraction region 603 is converted into a rear conversion image 608, and the right direction extraction region 604 is converted into a right direction conversion image 609.
　合成俯瞰映像６０５において、これらの変換映像６０６～６０９は順次隣接するように配置されていて、本実施の形態においては、ダンプトラック１の周囲を俯瞰した映像を表現している。なお、変換映像の配置の結果映像が存在しない中央の部分にはダンプトラック１の車体２の存在領域を示すために、自車アイコン６１０を嵌め込み合成している。 In the composite bird's-eye view image 605, these converted images 606 to 609 are sequentially arranged so as to be adjacent to each other, and in the present embodiment, an image of the bird's-eye view of the periphery of the dump truck 1 is represented. As a result of the arrangement of the converted video, the own vehicle icon 610 is inserted and synthesized in the central portion where the video does not exist to indicate the region where the vehicle body 2 of the dump truck 1 exists.
　図５にもどり、車両周囲移動物体検知装置１００は、検知処理を実行する（ステップＳ１０４）。具体的には、図４に示す移動物体検知部１０２において、移動物体の検知処理を実行する。検知の対象とする映像は、周囲映像入力部１０１から入力する前方映像４０１、左方向映像４０２、後方映像４０３、右方向映像４０４であり、さらにそれぞれの映像内において、前方検知結果表示対象外領域５０１、左方向検知結果表示対象外領域５０２、後方検知結果表示対象外領域５０３、右方向検知結果表示対象外領域５０４での検知結果は対象外としている。 Returning to FIG. 5, the vehicle surrounding moving object detection apparatus 100 executes a detection process (step S104). Specifically, the moving object detection unit 102 shown in FIG. 4 performs a moving object detection process. The images to be detected are the forward video 401, the left video 402, the rear video 403, and the right video 404 input from the surrounding video input unit 101. Further, in each video, the front detection result display non-display area 501, the detection results in the left direction detection result display non-target area 502, the rear detection result display non-target area 503, and the right direction detection result display non-target area 504 are excluded.
　この例を図９を用いて説明する。図９は左方向映像４０２における移動物体の検知を示すものであって、上から順に、（ａ）ある特定時刻ｔ0 より前の時刻である時刻ｔ0－１における映像、（ｂ）ある特定時刻ｔ0 における映像、（ｃ）移動物体初期検知結果を示す映像、（ｄ）補正済み検知結果を示す映像を示している。 This example will be described with reference to FIG. FIG. 9 shows detection of a moving object in the left-direction image 402, and in order from the top, (a) an image at a time t 0 −1 that is a time before a certain specific time t 0 , and (b) a certain specific An image at time t 0 , (c) an image showing a moving object initial detection result, and (d) an image showing a corrected detection result are shown.
　ここで、（ａ）には、時刻ｔ0－１における移動物体７０２が左方向映像４０２上に撮影されている様子が示され、（ｂ）には特定時刻ｔ0 における移動物体７０１が左方向映像４０２上に撮影されている様子が示されている。これらの移動物体７０１と７０２は同一のものであり、時間の推移に伴って移動している様子が左方向カメラ３０３によって撮影されている。本実施の形態において、特定時刻ｔ0 における移動物体７０１と、特定時刻ｔ0 より前の時刻である時刻ｔ0－１における移動物体７０２とは、左方向抽出領域６０２の内部には存在していないが、左方向検知結果表示対象外領域５０２からも外れているので、移動物体検知の対象となると共に、移動物体の場合には、その表示がなされる対象となる。 Here, (a) shows a state in which the moving object 702 at time t 0 −1 is photographed on the left direction image 402, and (b) shows the moving object 701 at the specific time t 0 in the left direction. A state of being photographed on the video 402 is shown. These moving objects 701 and 702 are the same, and the state of moving with the passage of time is captured by the left camera 303. In the present embodiment, the moving object 701 at a particular time t 0, the moving object 702 at time t 0 -1 is a time before the specific time t 0, is present in the interior of the left extraction region 602 Although not included in the left direction detection result display non-target area 502, it is a target for moving object detection, and in the case of a moving object, it is a target for display.
　図４に示す移動物体検知部１０２は、周囲映像入力部１０１から入力するこれらの映像を比較し、時間の推移に伴って映像に変化が発生した部位を特定する。具体的には、図９（ｃ）の四角形で示す左方向の検知結果７０３のように、部位を特定する。移動物体検知部１０２は、この左方向の検知結果７０３の左方向映像４０２上での座標情報を算出して保持する。なお、映像に変化が発生していない場合はこのような検知結果は得られない。 The moving object detection unit 102 shown in FIG. 4 compares these images input from the surrounding image input unit 101, and identifies a site where a change has occurred in the image with time. Specifically, a part is specified like a detection result 703 in the left direction indicated by a square in FIG. The moving object detection unit 102 calculates and holds coordinate information on the left direction image 402 of the detection result 703 in the left direction. It should be noted that such a detection result cannot be obtained when there is no change in the image.
　図５にもどり、車両周囲移動物体検知装置１００は、（ステップＳ１０４）における検知処理による移動物体の検知の有無を判断する（ステップＳ１０５）。移動物体を検知した場合には、（ステップＳ１０６）へ進み、それ以外の場合には、（ステップＳ１０８）へ進む。 Returning to FIG. 5, the vehicle surrounding moving object detection apparatus 100 determines whether or not a moving object is detected by the detection processing in (Step S104) (Step S105). If a moving object is detected, the process proceeds to (Step S106). Otherwise, the process proceeds to (Step S108).
　車両周囲移動物体検知装置１００は、検知箇所の座標を算出する（ステップＳ１０６）。具体的には、図４に示す移動物体検知部１０２において前方映像４０１、左方向映像４０２、後方映像４０３、右方向映像４０４の中で得られた移動物体の部位を座標情報に変換し、合成俯瞰映像６０５における座標情報を算出する。座標情報に加える変換は、（ステップＳ１０３）にて合成俯瞰映像６０５を構築する際に用いるものと同等である。 The vehicle surrounding moving object detection device 100 calculates the coordinates of the detection location (step S106). Specifically, the moving object detection unit 102 shown in FIG. 4 converts the part of the moving object obtained in the front video 401, the left video 402, the rear video 403, and the right video 404 into coordinate information, and combines it. Coordinate information in the overhead view image 605 is calculated. The conversion added to the coordinate information is the same as that used when constructing the composite bird's-eye view image 605 in (Step S103).
　しかし、本ステップにおいては、図９（ｂ）に示すように、特定時刻ｔ0 における移動物体７０１は、左方向映像４０２の内部にはあるものの左方向抽出領域６０２の外部に存在している。このため、この座標情報に変換を加えた場合、変換後の移動物体７０１の座標情報は、図８に示す左方向変換映像６０７の外部を指すことになる。そこで、移動物体検知部１０２は、図９（ｃ）で示す特定時刻ｔ0 における左方向の検知結果７０３に最も近い左方向抽出領域６０２の内部の部位を算出し、この算出した部位を図９（ｄ）に示すように特定時刻ｔ0 における左方向の補正済み検知結果７０４としている。 However, in this step, as shown in FIG. 9B, the moving object 701 at the specific time t 0 exists outside the left direction extraction region 602 although it is inside the left direction image 402. For this reason, when conversion is applied to this coordinate information, the coordinate information of the moving object 701 after conversion indicates the outside of the left-direction converted image 607 shown in FIG. Therefore, the moving object detection unit 102 calculates a part inside the left direction extraction region 602 closest to the left direction detection result 703 at the specific time t 0 shown in FIG. 9C, and the calculated part is shown in FIG. As shown in (d), the corrected detection result 704 in the left direction at the specific time t 0 is used.
　図５にもどり、車両周囲移動物体検知装置１００は、検知箇所の強調設定を行う（ステップＳ１０７）。具体的には、合成映像構築部１０４にて、移動物体の検知を利用者に示すために、強調表示の設定が行われる。 Returning to FIG. 5, the vehicle surrounding moving object detection apparatus 100 performs enhancement setting of the detection location (step S <b> 107). Specifically, the composite video constructing unit 104 sets highlighting in order to indicate to the user that the moving object is detected.
　この例を図８と図９を用いて説明する。図８の合成俯瞰映像６０５において、図９（ｄ）に示す特定時刻ｔ0 における左方向の補正済み検知結果７０４に変換を加えることにより、特定時刻ｔ0 における座標変換後の検知結果８０１に強調表示が設定されている。 This example will be described with reference to FIGS. In the composite bird's-eye view image 605 in FIG. 8, the detection result 801 after coordinate conversion at the specific time t 0 is emphasized by adding a conversion to the corrected detection result 704 in the left direction at the specific time t 0 shown in FIG. Display is set.
　図９（ｃ）と図９（ｄ）において、特定時刻ｔ0 における左方向の補正済み検知結果７０４は、左方向抽出領域６０２の内部にあって、かつ特定時刻ｔ0 における左方向の検知結果７０３に最も近い位置に存在している。そして、図８に示す特定時刻ｔ0 における座標変換後の検知結果８０１は、合成俯瞰映像６０５上における左方向変換映像６０７上で、特定時刻ｔ0 における左方向の検知結果７０３の示す移動物体に最も近い位置に存在している。 Figure 9 (c) and in FIG. 9 (d), the left direction of the corrected detection results 704 at a specific time t 0 is, in the interior of the left extraction region 602, and the detection result of the left at a particular time t 0 It exists at a position closest to 703. Then, the detection result 801 after the coordinate conversion at the specific time t 0 shown in FIG. 8 becomes a moving object indicated by the detection result 703 in the left direction at the specific time t 0 on the left-direction converted video 607 on the synthesized overhead image 605. It exists in the nearest position.
　図５にもどり、車両周囲移動物体検知装置１００は、合成映像を出力する（ステップＳ１０８）。具体的には、合成映像構築部１０４が合成俯瞰映像６０５を出力部１０５へ出力する。上述した処理において、移動物体を検知している場合には、出力部１０５には特定時刻ｔ0 における座標変換後の検知結果８０１のような強調表示が行われた合成俯瞰映像６０５が出力される。 Returning to FIG. 5, the vehicle surrounding moving object detection apparatus 100 outputs a composite image (step S <b> 108). Specifically, the composite video construction unit 104 outputs the composite overhead video 605 to the output unit 105. In the above-described processing, when a moving object is detected, the output unit 105 outputs a composite bird's-eye view image 605 that is highlighted as in the detection result 801 after coordinate conversion at the specific time t 0 . .
　なお、本実施の形態においては、特定時刻ｔ0 における左方向の検知結果７０３に補正を加えた特定時刻ｔ0 における左方向の補正済み検知結果７０４を表示に利用すると共に、図８に示すように三角形の形状で強調表示している。また、上述の処理で移動物体を検知しない場合には、強調表示は行われない。 In the present embodiment, while utilizing the left of the corrected detection results 704 at a particular time t 0 plus correction in the left direction of the detection result 703 at a particular time t 0 on the display, as shown in FIG. 8 Is highlighted with a triangular shape. Further, when a moving object is not detected in the above-described processing, highlighting is not performed.
　図５にもどり、車両周囲移動物体検知装置１００は、移動物体の検知及び出力が完了したかどうかを判定する（ステップＳ１０９）。これらの処理が完了した場合は終了し、それ以外の場合は（ステップＳ１０１）に戻る。 Returning to FIG. 5, the vehicle surrounding moving object detection apparatus 100 determines whether the detection and output of the moving object is completed (step S109). If these processes are completed, the process ends. Otherwise, the process returns to (Step S101).
　上述したように、本実施の形態における移動物体の検知と表示の処理がなされるので、合成俯瞰映像６０５上では表示されていない部位における移動物体の有無に関しても、検知することができ、その存在を強調表示することで、利用者に報知できる。 As described above, since the detection and display processing of the moving object in the present embodiment is performed, it is possible to detect the presence or absence of the moving object in the part that is not displayed on the composite overhead view image 605, and its existence. By highlighting, it is possible to notify the user.
　上述した本発明の車両周囲移動物体検知システムの第１の実施の形態によれば、自車両１の周囲の状況において、利用者に対して合成俯瞰映像６０５として表示している部分には含まれない箇所で移動物体を検知した場合でも、検知結果を利用者に報知するので、利用者は、自車両１の作業上注意すべき対象物の存在と位置関係を迅速に把握できる。これにより、作業上の安全を確保するための確認作業を効率化できると共に、人物や他車両といった対象物との接触を回避することができる。この結果、作業全体の運用効率を向上することができる。
＜実施例２＞
　以下、本発明の車両周囲移動物体検知システムの第２の実施の形態を図面を用いて説明する。図１０は本発明の車両周囲移動物体検知システムの第２の実施の形態における合成俯瞰映像と移動物体の検知結果表示の一例を示す概念図である。図１０において、図１乃至図９に示す符号と同符号のものは同一部分であるので、その詳細な説明は省略する。 According to the first embodiment of the vehicle surrounding moving object detection system of the present invention described above, it is included in the portion displayed as the synthesized bird's-eye view image 605 for the user in the situation around the host vehicle 1. Even when a moving object is detected at a non-existing location, the detection result is notified to the user, so that the user can quickly grasp the presence and positional relationship of the object to be noted in the operation of the host vehicle 1. As a result, it is possible to improve the efficiency of confirmation work for ensuring work safety, and to avoid contact with an object such as a person or another vehicle. As a result, the operational efficiency of the entire work can be improved.
<Example 2>
Hereinafter, a second embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to the drawings. FIG. 10 is a conceptual diagram showing an example of a composite bird's-eye view image and a moving object detection result display in the second embodiment of the vehicle surrounding moving object detection system of the present invention. In FIG. 10, the same reference numerals as those shown in FIG. 1 to FIG. 9 are the same parts, and detailed description thereof will be omitted.
　本実施の形態における車両周囲移動物体検知システムは、その構成及び運用方法は第１の実施の形態と大略同様である。第２の実施の形態においては、自車両１の下部にあって合成俯瞰映像６０５上では表示されていない移動物体を検知した場合に、その旨をメッセージ表示することで利用者に報知する点が第１の実施の形態と異なる。 The vehicle surrounding moving object detection system in the present embodiment has a configuration and an operation method that are substantially the same as those in the first embodiment. In the second embodiment, when a moving object that is located below the host vehicle 1 and is not displayed on the composite bird's-eye view image 605 is detected, the user is notified by displaying a message to that effect. Different from the first embodiment.
　具体的には、第１の実施の形態において図５に示す処理内容を示すフローチャート図の（ステップＳ１０６）と（ステップＳ１０７）の処理と、第１の実施の形態において図８に示す合成俯瞰映像における移動物体の検知結果表示が異なるので、この部分について説明する。 Specifically, the processing of (Step S106) and (Step S107) in the flowchart showing the processing contents shown in FIG. 5 in the first embodiment, and the composite overhead view video shown in FIG. 8 in the first embodiment. Since the detection result display of the moving object in is different, this part will be described.
　本実施の形態においては、（ステップＳ１０６）において、第１の実施の形態と同様に検知箇所の座標を算出すると共に、この情報を基に検知した移動物体が自車両１の下部にあるか否かを判断する。具体的には、図４に示す移動物体検知部１０２において以下の条件が全て充足されたときに、この移動物体は自車両１の下部に存在すると判断する。
（１）前方映像４０１、左方向映像４０２、後方映像４０３、右方向映像４０４の中のいずれかに移動物体が含まれていること。
（２）図７に示す前方抽出領域６０１、左方向抽出領域６０２、後方抽出領域６０３、右方向抽出領域６０４のそれぞれを示す下辺よりも下側に移動物体が存在すること。
（３）図６に示す前方検知結果表示対象外領域５０１、左方向検知結果表示対象外領域５０２、後方検知結果表示対象外領域５０３、右方向検知結果表示対象外領域５０４のいずれも含まない領域に移動物体が存在すること。 In the present embodiment, in (Step S106), the coordinates of the detection location are calculated in the same manner as in the first embodiment, and whether or not the moving object detected based on this information is below the host vehicle 1. Determine whether. Specifically, when all of the following conditions are satisfied in the moving object detection unit 102 shown in FIG. 4, it is determined that the moving object exists below the host vehicle 1.
(1) A moving object is included in any of the front video 401, the left video 402, the rear video 403, and the right video 404.
(2) A moving object exists below the lower side indicating each of the front extraction area 601, the left direction extraction area 602, the rear extraction area 603, and the right direction extraction area 604 shown in FIG.
(3) Area that does not include any of the forward detection result display non-target area 501, the left direction detection result display non-target area 502, the rear detection result display non-target area 503, and the right direction detection result display non-target area 504 illustrated in FIG. 6. There must be a moving object.
　次に（ステップＳ１０７）において、検知箇所の強調設定がなされる。具体的には、合成映像構築部１０４にて、移動物体の検知を利用者に示すための設定が行われる。検知した移動物体が自車両１の下部に存在する場合、“自車体の下部に移動物体を検知しました。”とのメッセージ文字列を出力部１０５に出力するように設定される。同様に、検知した移動物体が自車両１の下部に存在しない場合は、メッセージ文字列を出力部１０５に出力しないように設定される。 Next, in (Step S107), the detection location is emphasized. Specifically, the composite video constructing unit 104 performs settings for indicating the detection of the moving object to the user. When the detected moving object exists in the lower part of the own vehicle 1, the message character string “A moving object has been detected in the lower part of the own vehicle body” is output to the output unit 105. Similarly, when the detected moving object does not exist below the host vehicle 1, the message character string is set not to be output to the output unit 105.
　図１０は、これらの処理によりなされた出力部１０５上の表示例であって、自車両１の下部に移動物体を検知した場合の様子を示している。ステップ１０６及びステップ１０７の処理により、出力部１０５に表示する合成俯瞰映像６０５上には、自車両１の下部に移動物体を検知したことを示す検知メッセージ９０１が表示されている。これにより、前方変換映像６０６、左方向変換映像６０７、後方変換映像６０８、右方向変換映像６０９のいずれにも表示されない自車両１の下部に存在する移動物体の状態を利用者に報知することができる。 FIG. 10 is a display example on the output unit 105 made by these processes, and shows a state in which a moving object is detected in the lower part of the host vehicle 1. As a result of the processing of Step 106 and Step 107, a detection message 901 indicating that a moving object has been detected is displayed on the lower part of the host vehicle 1 on the synthesized overhead image 605 displayed on the output unit 105. Accordingly, it is possible to notify the user of the state of the moving object existing below the host vehicle 1 that is not displayed in any of the forward converted video 606, the left converted video 607, the backward converted video 608, and the right converted video 609. it can.
　上述した本発明の車両周囲移動物体検知システムの第２の実施の形態によれば、上述した第１の実施の形態と同様な効果を得ることができる。 According to the second embodiment of the vehicle surrounding moving object detection system of the present invention described above, the same effect as that of the first embodiment described above can be obtained.
　また、上述した本発明の車両周囲移動物体検知システムの第２の実施の形態によれば、利用者は、具体的に自車両１の下部に移動物体が存在することを迅速に把握できるので、作業上の安全を確保するための確認作業を効率化できる。この結果、より安全性を確保できる。
＜実施例３＞
　以下、本発明の車両周囲移動物体検知システムの第３の実施の形態を図面を用いて説明する。図１１は本発明の車両周囲移動物体検知システムの第３の実施の形態における合成俯瞰映像と移動物体の検知結果表示の一例を示す概念図である。図１１において、図１乃至図１０に示す符号と同符号のものは同一部分であるので、その詳細な説明は省略する。 In addition, according to the second embodiment of the vehicle surrounding moving object detection system of the present invention described above, the user can quickly grasp that there is a moving object in the lower part of the own vehicle 1 specifically. It is possible to improve the efficiency of confirmation work to ensure work safety. As a result, more safety can be ensured.
<Example 3>
Hereinafter, a third embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to the drawings. FIG. 11 is a conceptual diagram showing an example of a composite overhead view image and a moving object detection result display in the third embodiment of the vehicle surrounding moving object detection system of the present invention. In FIG. 11, the same reference numerals as those shown in FIGS. 1 to 10 are the same parts, and detailed description thereof is omitted.
　本実施の形態における車両周囲移動物体検知システムは、その構成及び運用方法は第１の実施の形態と大略同様である。第３の実施の形態においては、自車両１の下部にあって合成俯瞰映像６０５上では表示されていない移動物体を検知した場合に、検知した位置に相当する自車アイコン６１０上の位置に検知結果を表示することで利用者に報知する点が第１の実施の形態と異なる。 The vehicle surrounding moving object detection system in the present embodiment has a configuration and an operation method that are substantially the same as those in the first embodiment. In the third embodiment, when a moving object that is below the host vehicle 1 and is not displayed on the composite overhead view image 605 is detected, it is detected at a position on the host vehicle icon 610 corresponding to the detected position. The point which notifies a user by displaying a result differs from a 1st embodiment.
　具体的には、第１の実施の形態において図５に示す処理内容を示すフローチャート図の（ステップＳ１０６）と（ステップＳ１０７）の処理と、第１の実施の形態において図８に示す合成俯瞰映像における移動物体の検知結果表示が異なるので、この部分について説明する。 Specifically, the processing of (Step S106) and (Step S107) in the flowchart showing the processing contents shown in FIG. 5 in the first embodiment, and the composite overhead view video shown in FIG. 8 in the first embodiment. Since the detection result display of the moving object in is different, this part will be described.
　本実施の形態においては、（ステップＳ１０６）において、第１の実施の形態と同様に検知箇所の座標を算出するが、このとき、検知結果の補正を行わず、特定時刻ｔ0 における左方向の検知結果７０３をその後の処理で使用する。なお、特定時刻ｔ0 における左方向の検知結果７０３は、図７（ｃ）に示すように、自車両１の下部、前輪４Ａの後ろ側に存在している。 In the present embodiment, in (step S106), but calculates the coordinates of the same detected position of the first embodiment, this time, without correcting the detection result, the left direction at a particular time t 0 The detection result 703 is used in subsequent processing. Note that the detection result 703 in the left direction at the specific time t 0 is present at the lower part of the host vehicle 1 and behind the front wheel 4A as shown in FIG. 7C.
　次に（ステップＳ１０７）において、検知箇所の強調設定がなされる。具体的には、合成映像構築部１０４にて、移動物体の検知を利用者に示すために、強調表示の設定が行われる。 Next, in (Step S107), the detection location is emphasized. Specifically, the composite video constructing unit 104 sets highlighting in order to indicate to the user that the moving object is detected.
　この例を１１を用いて説明する。図１１の合成俯瞰映像６０５において、図９（ｃ）に示す特定時刻ｔ0 における左方向の検知結果７０３に変換を加えることにより、特定時刻ｔ0 における座標変換後の検知結果１００１に強調表示が設定されている。 This example will be described with reference to 11. In the composite bird's-eye view image 605 of FIG. 11, by adding a conversion to the detection result 703 in the left direction at the specific time t 0 shown in FIG. 9C, the detection result 1001 after the coordinate conversion at the specific time t 0 is highlighted. Is set.
　図９（ｃ）において、特定時刻ｔ0 における左方向の検知結果７０３は、左方向抽出領域６０２の外部に存在するため、座標変換を行うと図１０の自車アイコン６１０の表示領域内部に存在することになる。このため、図１０において、特定時刻ｔ0 における座標変換後の検知結果１００１が自車アイコン６１０の上に重ねるように表示されている。これにより、前方変換映像６０６、左方向変換映像６０７、後方変換映像６０８、右方向変換映像６０９のいずれにも表示されない自車両１の下部に存在する移動物体の状態を、大略の位置を含めて利用者に報知することができる。 In FIG. 9C, the detection result 703 in the left direction at the specific time t 0 exists outside the left direction extraction region 602. Therefore, when coordinate conversion is performed, the detection result 703 exists in the display region of the vehicle icon 610 in FIG. Will do. For this reason, in FIG. 10, the detection result 1001 after the coordinate conversion at the specific time t 0 is displayed so as to be superimposed on the vehicle icon 610. As a result, the state of the moving object existing below the host vehicle 1 that is not displayed in any of the forward converted video 606, the left converted video 607, the backward converted video 608, and the right converted video 609, including the approximate position. The user can be notified.
　上述した本発明の車両周囲移動物体検知システムの第３の実施の形態によれば、上述した第１の実施の形態と同様な効果を得ることができる。 According to the third embodiment of the vehicle surrounding moving object detection system of the present invention described above, the same effects as those of the first embodiment described above can be obtained.
　また、上述した本発明の車両周囲移動物体検知システムの第３の実施の形態によれば、利用者は、具体的に自車両１の下部に移動物体が存在することと、大略の位置を迅速に把握できるので、作業上の安全を確保するための確認作業を効率化できる。この結果、より安全性を確保できる。
＜実施例４＞
　以下、本発明の車両周囲移動物体検知システムの第４の実施の形態を図面を用いて説明する。図１２は本発明の車両周囲移動物体検知システムの第４の実施の形態の構成を示すブロック図、図１３は図１２に示す強調情報保持部におけるテーブルの一例を示す表図、図１４は本発明の車両周囲移動物体検知システムの第４の実施の形態における処理内容を示すフローチャート図、図１５は本発明の車両周囲移動物体検知システムの第４の実施の形態において移動物体が映った映像の一例を示す概念図である。図１２乃至図１５において、図１乃至図１１に示す符号と同符号のものは同一部分であるので、その詳細な説明は省略する。 Further, according to the third embodiment of the vehicle surrounding moving object detection system of the present invention described above, the user can quickly determine that there is a moving object in the lower part of the own vehicle 1 and the approximate position. Therefore, it is possible to improve the efficiency of confirmation work to ensure work safety. As a result, more safety can be ensured.
<Example 4>
Hereinafter, a fourth embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to the drawings. FIG. 12 is a block diagram showing the configuration of the fourth embodiment of the vehicle surrounding moving object detection system of the present invention, FIG. 13 is a table showing an example of the table in the emphasis information holding unit shown in FIG. 12, and FIG. FIG. 15 is a flowchart showing the processing contents in the fourth embodiment of the vehicle surrounding moving object detection system of the invention, and FIG. 15 is an image of the moving object reflected in the fourth embodiment of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram which shows an example. 12 to 15, the same reference numerals as those shown in FIGS. 1 to 11 are the same parts, and detailed description thereof is omitted.
　本実施の形態における車両周囲移動物体検知システムは、その構成及び運用方法は第１の実施の形態と大略同様である。第４の実施の形態においては、移動物体が自車両１に対してより近い位置に存在する場合に、移動物体の検知結果をより強調して表示して利用者に報知する点が第１の実施の形態と異なる。 The vehicle surrounding moving object detection system in the present embodiment has a configuration and an operation method that are substantially the same as those in the first embodiment. In the fourth embodiment, when the moving object is present at a position closer to the host vehicle 1, the first point is that the detection result of the moving object is emphasized and displayed to the user. Different from the embodiment.
　具体的には、図１２に示すように車両周囲移動物体検知装置１００Ａにおいて、強調情報保持部１０６をさらに備えている。　
　強調情報保持部１０６は、出力部１０５で表示する映像において、自車両１と移動物体との距離に応じて、強調表示での強調の度合いを示す色の情報信号を例えばテーブルとして保持し、合成映像構築部１０４に出力する。 Specifically, as shown in FIG. 12, the vehicle surrounding moving object detection device 100 </ b> A further includes an emphasis information holding unit 106.
The emphasis information holding unit 106 holds, as a table, for example, a color information signal indicating the degree of emphasis in the emphasis display according to the distance between the host vehicle 1 and the moving object in the video displayed by the output unit 105. Output to the video construction unit 104.
　強調情報保持部１０６が保持するテーブルの一例を図１３に示す。本実施の形態においては、周囲映像４０１～４０４における移動物体と自車両１との距離Ｌが３００ピクセル以上離れている場合には、強調表示色として緑色を使用する。同様に距離Ｌが１５０ピクセル以上３００ピクセル未満の場合は強調表示色を黄色とし、距離が０ピクセル以上１５０ピクセル未満の場合は強調表示色を赤色としている。なお、強調情報保持部１０６には、この他の種類のテーブルを格納することも可能である。 An example of the table held by the emphasis information holding unit 106 is shown in FIG. In the present embodiment, when the distance L between the moving object and the vehicle 1 in the surrounding images 401 to 404 is 300 pixels or more, green is used as the highlight color. Similarly, when the distance L is 150 pixels or more and less than 300 pixels, the highlight color is yellow, and when the distance is 0 pixel or more and less than 150 pixels, the highlight color is red. The emphasis information holding unit 106 can store other types of tables.
　次に、本発明の車両周囲移動物体検知システムの第４の実施の形態における移動物体の検知と表示の処理内容について図１４と図１５を用いて説明する。本実施の形態における処理内容は、図５に示す第１の実施の形態における処理内容とは、（ステップＳ１０６）と（ステップＳ１０７）との間に（ステップＳ２０６）が加わっている点が異なるが、その他は同様である。 Next, moving object detection and display processing contents in the fourth embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to FIGS. The processing contents in this embodiment are different from the processing contents in the first embodiment shown in FIG. 5 in that (step S206) is added between (step S106) and (step S107). Others are the same.
　本実施の形態においては、車両周囲移動物体検知装置１００Ａは、（ステップＳ１０６）において、第１の実施の形態と同様に検知箇所の座標を算出した後、この検知箇所の座標を基に領域内容の強調設定を行う（ステップＳ２０６）。具体的には、図１２に示す合成映像構築部１０４が周囲映像４０１～４０４における移動物体と自車両１との距離Ｌを算出し、強調情報保持部１０６の保持するテーブルと照らしあわて強調表示色を設定する。 In the present embodiment, the vehicle surrounding moving object detection device 100A calculates the coordinates of the detection location in (Step S106) in the same manner as in the first embodiment, and then the region contents based on the coordinates of the detection location. Is set (step S206). Specifically, the composite video construction unit 104 shown in FIG. 12 calculates the distance L between the moving object and the host vehicle 1 in the surrounding videos 401 to 404, and highlights the color in light of the table held by the highlight information holding unit 106. Set.
　図１５に示すように、例えば、前方映像４０１において、移動物体１００１と自車両１との距離Ｌが２００ピクセルであった場合、（ステップＳ２０６）において、合成映像構築部１０４が強調表示色を黄色に設定する。 As shown in FIG. 15, for example, when the distance L between the moving object 1001 and the host vehicle 1 is 200 pixels in the forward video 401, the composite video construction unit 104 sets the highlighted color to yellow in (Step S206). Set to.
　図１４にもどり、車両周囲移動物体検知装置１００Ａは、検知箇所の強調設定を行う（ステップＳ１０７）。具体的には、合成映像構築部１０４にて、強調表示色を黄色に設定する。この後、（ステップＳ１０８）での合成映像出力において、合成俯瞰映像６０５に対して移動物体１００１を黄色で強調表示される。 Returning to FIG. 14, the vehicle surrounding moving object detection device 100 </ b> A performs enhancement setting of the detection location (step S <b> 107). Specifically, the composite video construction unit 104 sets the highlight color to yellow. Thereafter, in the composite video output in (Step S108), the moving object 1001 is highlighted in yellow with respect to the composite overhead image 605.
　これらの処理が実行されることより、例えば、移動物体１００１が自車両１に近づいてくる場合、その接近距離につれて、緑色から黄色、黄色から赤色へと強調表示色の強調度が増大する。このことにより、運転の安全性確保にとって重要となる自車両１の近傍の移動物体１００１の存在を利用者により強調することができる。 By executing these processes, for example, when the moving object 1001 approaches the host vehicle 1, the degree of emphasis of the highlighted color increases from green to yellow and from yellow to red with the approach distance. Thus, the presence of the moving object 1001 in the vicinity of the host vehicle 1 that is important for ensuring driving safety can be emphasized by the user.
　上述した本発明の車両周囲移動物体検知システムの第４の実施の形態によれば、上述した第１の実施の形態と同様な効果を得ることができる。 According to the fourth embodiment of the vehicle surrounding moving object detection system of the present invention described above, the same effects as those of the first embodiment described above can be obtained.
　また、上述した本発明の車両周囲移動物体検知システムの第４の実施の形態によれば、移動物体１００１の存在を強調表示すると共に、距離に応じて強調表示色を変化させているので、利用者は、移動物体１００１の存在と共に、移動物体１００１と自車両１との距離Ｌとを感覚的に迅速に把握できる。この結果、より安全性を確保できる。 Further, according to the fourth embodiment of the vehicle surrounding moving object detection system of the present invention described above, the presence of the moving object 1001 is highlighted and the highlighted color is changed according to the distance. The person can quickly and intuitively grasp the distance L between the moving object 1001 and the host vehicle 1 together with the presence of the moving object 1001. As a result, more safety can be ensured.
　なお、本実施の形態においては、移動物体１００１の強調表示を色の切り替えで行っているが、これに限るものではない。例えば、表示する線の太さや、表示を点滅させる速度といった情報に対応づけることも可能である。
＜実施例５＞
　以下、本発明の車両周囲移動物体検知システムの第５の実施の形態を図面を用いて説明する。図１６は本発明の車両周囲移動物体検知システムの第５の実施の形態における強調情報保持部におけるテーブルの一例を示す表図、図１７は本発明の車両周囲移動物体検知システムの第５の実施の形態において移動物体が映った映像の一例を示す概念図である。図１６及び図１７において、図１乃至図１５に示す符号と同符号のものは同一部分であるので、その詳細な説明は省略する。 In this embodiment, highlighting of the moving object 1001 is performed by switching colors, but the present invention is not limited to this. For example, information such as the thickness of the line to be displayed and the speed at which the display is blinked can be associated.
<Example 5>
Hereinafter, a fifth embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to the drawings. FIG. 16 is a table showing an example of a table in the emphasis information holding unit in the fifth embodiment of the vehicle surrounding moving object detection system of the present invention, and FIG. 17 shows the fifth embodiment of the vehicle surrounding moving object detection system of the present invention. It is a conceptual diagram which shows an example of the image | video which showed the moving object in the form of. In FIG. 16 and FIG. 17, the same reference numerals as those shown in FIG. 1 to FIG.
　本実施の形態における車両周囲移動物体検知システムは、その構成及び運用方法は第４の実施の形態と大略同様である。第５の実施の形態においては、強調情報保持部１０６の保持するテーブルが第４の実施の形態と異なる。 The vehicle surrounding moving object detection system in the present embodiment has substantially the same configuration and operation method as in the fourth embodiment. In the fifth embodiment, the table held by the emphasis information holding unit 106 is different from that in the fourth embodiment.
　本実施の形態の強調情報保持部１０６が保持するテーブルの一例を図１６に、また、移動物体が映った映像の一例を図１７に示す。上述した周囲映像４０１～４０４に示すように、撮影条件にもとづき、周囲映像のより上方に撮影されている物体ほど自車両１の車体２からより遠い位置に存在し、周囲映像のより下方に撮影されている物体ほど自車両１の車体２からより近い位置に存在する。 FIG. 16 shows an example of a table held by the emphasis information holding unit 106 of the present embodiment, and FIG. 17 shows an example of an image showing a moving object. As shown in the surrounding images 401 to 404 described above, based on the shooting conditions, an object photographed above the surrounding image is located farther from the vehicle body 2 of the host vehicle 1 and is photographed below the surrounding image. The closer the object is, the closer to the body 2 of the host vehicle 1 is.
　そこで、本実施の形態においては、各周囲映像における垂直方向の座標であるｙ座標を、各周囲映像の上端を０として、下方への移動に伴って数値を増大するものと規定している。このため、ｙ座標の数値が大きいほど自車両１に近いことを示している。 Therefore, in the present embodiment, the y-coordinate which is the vertical coordinate in each surrounding image is defined as 0, with the upper end of each surrounding image being 0, and the numerical value is increased as it moves downward. For this reason, it shows that it is closer to the own vehicle 1 so that the numerical value of y coordinate is large.
　図１６に示すように、本実施の形態においては、周囲映像４０１～４０４における垂直方向の最も上部のピクセル位置を０とし、下に向かって座標値が増大するとした場合、垂直方向の座標が０ピクセル以上１５０ピクセル未満の場合は強調表示色を緑色としている。同様に、座標が１５０ピクセル以上３００ピクセル未満の場合は強調表示色を黄色とし、座標が３００ピクセル以上の場合は強調表示色を赤色としている。 As shown in FIG. 16, in the present embodiment, when the uppermost pixel position in the vertical direction in the surrounding images 401 to 404 is 0 and the coordinate value increases downward, the vertical coordinate is 0. When the pixel is greater than or equal to 150 pixels, the highlighted color is green. Similarly, when the coordinates are 150 pixels or more and less than 300 pixels, the highlight color is yellow, and when the coordinates are 300 pixels or more, the highlight color is red.
　次に、本発明の車両周囲移動物体検知システムの第５の実施の形態における移動物体の検知と表示の処理内容について図１４と図１７を用いて説明する。 Next, moving object detection and display processing contents in the fifth embodiment of the vehicle surrounding moving object detection system of the present invention will be described with reference to FIGS.
　本実施の形態においては、車両周囲移動物体検知装置１００Ａは、（ステップＳ１０６）において、第１の実施の形態と同様に検知箇所の座標を算出するが、この座標は、上述した移動物体１００１のｙ座標値に変換されている。 In the present embodiment, the vehicle surrounding moving object detection device 100A calculates the coordinates of the detection location in (step S106) as in the first embodiment, and these coordinates are the same as those of the moving object 1001 described above. It has been converted to a y-coordinate value.
　車両周囲移動物体検知装置１００Ａは、この検知箇所のｙ座標を基に領域内容の強調設定を行う（ステップＳ２０６）。具体的には、合成映像構築部１０４が周囲映像４０１～４０４における移動物体のｙ座標ｖを算出し、強調情報保持部１０６の保持するテーブルと照らしあわて強調表示色を設定する。 The vehicle surrounding moving object detection device 100A performs region content enhancement setting based on the y-coordinate of this detection location (step S206). Specifically, the composite video construction unit 104 calculates the y-coordinate v of the moving object in the surrounding videos 401 to 404, and sets the highlight display color against the table held by the highlight information holding unit 106.
　図１７に示すように、例えば、前方映像４０１において、移動物体１００１のｙ座標値ｖが２５０ピクセルであった場合、（ステップＳ２０６）において、合成映像構築部１０４が強調表示色を黄色に設定する。 As illustrated in FIG. 17, for example, when the y coordinate value v of the moving object 1001 is 250 pixels in the front image 401, the composite image constructing unit 104 sets the highlight color to yellow in (Step S206). .
　図１４にもどり、車両周囲移動物体検知装置１００Ａは、検知箇所の強調設定を行う（ステップＳ１０７）。具体的には、合成映像構築部１０４にて、強調表示色を黄色に設定する。この後、（ステップＳ１０８）での合成映像出力において、合成俯瞰映像６０５に対して移動物体１００１を黄色で強調表示される。 Returning to FIG. 14, the vehicle surrounding moving object detection device 100 </ b> A performs enhancement setting of the detection location (step S <b> 107). Specifically, the composite video construction unit 104 sets the highlight color to yellow. Thereafter, in the composite video output in (Step S108), the moving object 1001 is highlighted in yellow with respect to the composite overhead image 605.
　上述した本発明の車両周囲移動物体検知システムの第５の実施の形態によれば、上述した第１の実施の形態及び第４の実施の形態と同様な効果を得ることができる。 According to the fifth embodiment of the vehicle surrounding moving object detection system of the present invention described above, the same effects as those of the first embodiment and the fourth embodiment described above can be obtained.
　なお、本実施の形態においては、移動物体１００１の位置から強調表示色を設定するに当たりテーブルを用いているが、これに限るものではない。例えば、数式等を用いて設定しても良い。 In the present embodiment, a table is used to set the highlight color from the position of the moving object 1001, but the present invention is not limited to this. For example, you may set using a numerical formula etc.
　なお、本発明は上記した実施例に限定されるものではなく、様々な変形例が含まれる。例えば、上記した実施例は本発明を分かりやすく説明するために詳細に説明したものであり、必ずしも説明した全ての構成を備えるものに限定されるものではない。また、上記の各構成、機能、処理部、処理手段等は、それらの一部又は全部を、例えば集積回路で設計する等によりハードウェアで実現してもよい。また、上記の各構成、機能等は、プロセッサがそれぞれの機能を実現するプログラムを解釈し、実行することによりソフトウェアで実現してもよい。 In addition, this invention is not limited to the above-mentioned Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor.
１　　　　ダンプトラック（車両）
２　　　　車体
３　　　　ベッセル（荷台）
４Ａ　　　前輪
４Ｂ　　　後輪
８　　　　運転室
１００　　車両周囲移動物体検知装置
１０１　　周囲映像入力部
１０２　　移動物体検知部
１０３　　領域情報保持部
１０４　　合成映像構築部
１０５　　出力部
３０１　　前方カメラ
３０２　　左方向カメラ
３０３　　後方カメラ
３０４　　右方向カメラ
４０１　　前方映像
４０２　　左方向映像
４０３　　後方映像
４０４　　右方向映像
５０１　　前方検知結果表示対象外領域
５０２　　左方向検知結果表示対象外領域
５０３　　後方検知結果表示対象外領域
５０４　　右方向検知結果表示対象外領域
６０１　　前方抽出領域
６０２　　左方向抽出領域
６０３　　後方抽出領域
６０４　　右方向抽出領域
６０５　　合成俯瞰映像
６０６　　前方変換映像
６０７　　左方向変換映像
６０８　　後方変換映像
６０９　　右方向変換映像
６１０　　自車アイコン
７０１　　時刻ｔ0 における移動物体
７０２　　時刻ｔ0－１における移動物体
７０３　　時刻ｔ0 における左方向の検知結果
７０４　　時刻ｔ0 における左方向の補正済み検知結果
８０１　　時刻ｔ0 における座標変換後の検知結果
９０１　　検知メッセージ
１００１　移動物体 1 Dump truck (vehicle)
2 Body 3 Vessel
4A Front wheel 4B Rear wheel 8 Driver's cab 100 Vehicle surrounding moving object detection device 101 Ambient image input unit 102 Moving object detection unit 103 Area information holding unit 104 Composite image construction unit 105 Output unit 301 Front camera 302 Left camera 303 Rear camera 304 Right Direction camera 401 Front video 402 Left video 403 Rear video 404 Right video 501 Front detection result display non-target area 502 Left detection result display non-target area 503 Rear detection result display non-target area 504 Right direction detection result non-target display area 601 Forward extraction area 602 Left extraction area 603 Rear extraction area 604 Right extraction area 605 Composite overhead image 606 Forward conversion video 607 Left conversion video 608 Rear conversion video 609 Right conversion video 610 Car icon 701 Movement at time t 0 object Body 702 time t 0 detection result 901 detection message 1001 moving object after the coordinate conversion in the corrected detection results 801 time t 0 in the left direction in the left direction in the detection result 704 time t 0 in the moving object 703 time t 0 at -1

Claims (6)

  1. 　自車両の周囲の複数の映像を撮影する周囲映像入力部と、前記周囲映像入力部が撮影した複数の周囲映像から合成映像構成用領域を抽出して、抽出した複数の合成映像構成用領域を合成することで合成俯瞰映像を構築する合成映像構築部と、前記合成俯瞰映像を利用者に提示する出力部とを備えた車両周囲移動物体検知システムにおいて、
    　前記複数の周囲映像に関して移動物体の検知処理を行う移動物体検知処理部とを有し、
    　前記移動物体検知処理部は、前記合成俯瞰映像に示す領域以外で移動物体を検知した場合にも検知結果を前記合成俯瞰映像に出力して前記利用者に報知する
    　ことを特徴とする車両周囲移動物体検知システム。     A surrounding image input unit that captures a plurality of images around the host vehicle, and a plurality of synthesized image configuration areas extracted from the plurality of surrounding images captured by the surrounding image input unit. In a vehicle surrounding moving object detection system comprising a composite video construction unit that constructs a composite overhead view video by combining, and an output unit that presents the composite overhead view video to a user,
    A moving object detection processing unit that performs detection processing of a moving object with respect to the plurality of surrounding images
    The moving object detection processing unit outputs a detection result to the synthesized bird's-eye image and notifies the user even when a moving object is detected outside the region shown in the synthesized bird's-eye view image. Object detection system.
  2. 　請求項１に記載の車両周囲移動物体検知システムにおいて、
    　移動物体を検知したとの検知結果を表示させない領域である検知結果表示対象外領域の情報を保持する領域情報保持部と、
    　前記領域情報保持部から前記検知結果表示対象外領域の情報を取込み、前記複数の周囲映像から前記検知結果表示対象外領域を除外した映像に関して移動物体の検知処理を行う移動物体検知処理部とを更に備え、
    　前記移動物体検知処理部は、前記合成俯瞰映像に示す領域以外で移動物体を検知した場合にも検知結果を前記合成俯瞰映像に出力して前記利用者に報知する
    　ことを特徴とする車両周囲移動物体検知システム。     The vehicle surrounding moving object detection system according to claim 1,
    An area information holding unit that holds information of a detection result display non-target area that is an area in which a detection result that a moving object is detected is not displayed;
    A moving object detection processing unit that takes in information on the detection result non-target region from the region information holding unit, and performs a moving object detection process on a video that excludes the detection result display non-target region from the plurality of surrounding images; In addition,
    The moving object detection processing unit outputs a detection result to the synthesized bird's-eye image and notifies the user even when a moving object is detected outside the region shown in the synthesized bird's-eye view image. Object detection system.
  3. 　請求項２に記載の車両周囲移動物体検知システムにおいて、
    　前記検知結果表示対象外領域は、前記複数の周囲映像に映りこんだ前記自車両の一部である
    　ことを特徴とする車両周囲移動物体検知システム。     The vehicle surrounding moving object detection system according to claim 2,
    The detection result display non-target area is a part of the host vehicle that is reflected in the plurality of surrounding images. The vehicle surrounding moving object detection system.
  4. 　請求項２に記載の車両周囲移動物体検知システムにおいて、
    　前記移動物体検知処理部は、前記自車両の下部で移動物体を検知した場合に、検知結果を前記合成俯瞰映像にメッセージ出力して前記利用者に報知する
    　ことを特徴とする車両周囲移動物体検知システム。     The vehicle surrounding moving object detection system according to claim 2,
    When the moving object detection processing unit detects a moving object in a lower part of the host vehicle, the moving object detection processing unit outputs a detection result to the synthesized bird's-eye view message to notify the user. system.
  5. 　請求項２に記載の車両周囲移動物体検知システムにおいて、
    　前記移動物体検知処理部は、前記自車両の下部で移動物体を検知した場合に、検知結果を前記合成俯瞰映像における自車両を示す位置に描画出力して前記利用者に報知する
    　ことを特徴とする車両周囲移動物体検知システム。     The vehicle surrounding moving object detection system according to claim 2,
    When the moving object detection processing unit detects a moving object at a lower part of the host vehicle, the moving object detection processing unit draws and outputs a detection result at a position indicating the host vehicle in the synthesized bird's-eye view image and notifies the user. Moving object detection system around the vehicle.
  6. 　請求項２に記載の車両周囲移動物体検知システムにおいて、
    　前記自車両と検知した移動物体との距離に応じて、強調表示の強調の度合いを示す色の情報を保持する強調情報保持部を更に備え、
    　前記移動物体検知処理部は、前記強調情報保持部から前記強調表示での強調の度合いを示す色の情報を取込み、前記自車両と検知した移動物体との距離から強調表示色を設定し、検知結果である前記移動物体を設定した前記強調表示色で前記合成俯瞰映像上に表示して前記利用者に報知する
    　ことを特徴とする車両周囲移動物体検知システム。     The vehicle surrounding moving object detection system according to claim 2,
    An emphasis information holding unit that holds color information indicating the degree of emphasis according to the distance between the host vehicle and the detected moving object;
    The moving object detection processing unit fetches color information indicating the degree of emphasis in the emphasis display from the emphasis information holding unit, sets an emphasis display color from the distance between the host vehicle and the detected moving object, and detects The moving object detection system for vehicles is characterized in that the moving object as a result is displayed on the composite bird's-eye view image with the highlighted display color set and notified to the user.
PCT/JP2014/074212 2013-09-24 2014-09-12 Vehicle-periphery-moving-object detection system WO2015045904A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013-197091 2013-09-24
JP2013197091A JP6339337B2 (en) 2013-09-24 2013-09-24 Moving object detection system around the vehicle

Publications (1)

Publication Number Publication Date
WO2015045904A1 true WO2015045904A1 (en) 2015-04-02

Family

ID=52743046

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/074212 WO2015045904A1 (en) 2013-09-24 2014-09-12 Vehicle-periphery-moving-object detection system

Country Status (2)

Country Link
JP (1) JP6339337B2 (en)
WO (1) WO2015045904A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019121250A (en) * 2018-01-09 2019-07-22 日立建機株式会社 Transport vehicle

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20160136779A (en) * 2015-05-21 2016-11-30 주식회사 케이에스에스이미지넥스트 System and Method for Displaying State of Vehicle
JP6581519B2 (en) * 2016-02-09 2019-09-25 日立建機株式会社 Obstacle detection system for construction machinery
WO2018055873A1 (en) * 2016-09-20 2018-03-29 株式会社Ｊｖｃケンウッド Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program
JP6644264B2 (en) * 2016-09-21 2020-02-12 株式会社Ｊｖｃケンウッド Overhead video generation device, overhead video generation system, overhead video generation method and program
JP6730617B2 (en) * 2016-09-20 2020-07-29 株式会社Ｊｖｃケンウッド Overhead video generation device, overhead video generation system, overhead video generation method and program
JP6700644B2 (en) * 2017-03-29 2020-05-27 日立建機株式会社 Excavation condition monitoring device
JP7374602B2 (en) * 2019-03-29 2023-11-07 日立建機株式会社 work vehicle

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007230319A (en) * 2006-02-28 2007-09-13 Hitachi Ltd Start security device
JP2009071790A (en) * 2007-09-18 2009-04-02 Denso Corp Vehicle surroundings monitoring apparatus
WO2012169359A1 (en) * 2011-06-07 2012-12-13 株式会社小松製作所 Dump truck
JP2012254660A (en) * 2011-06-07 2012-12-27 Komatsu Ltd Periphery monitoring device of work vehicle

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4601505B2 (en) * 2005-07-20 2010-12-22 アルパイン株式会社 Top-view image generation apparatus and top-view image display method
JP4687411B2 (en) * 2005-11-15 2011-05-25 株式会社デンソー Vehicle peripheral image processing apparatus and program
JP4846426B2 (en) * 2006-04-20 2011-12-28 パナソニック株式会社 Vehicle perimeter monitoring device
JP4951639B2 (en) * 2009-03-02 2012-06-13 日立建機株式会社 Work machine with ambient monitoring device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007230319A (en) * 2006-02-28 2007-09-13 Hitachi Ltd Start security device
JP2009071790A (en) * 2007-09-18 2009-04-02 Denso Corp Vehicle surroundings monitoring apparatus
WO2012169359A1 (en) * 2011-06-07 2012-12-13 株式会社小松製作所 Dump truck
JP2012254660A (en) * 2011-06-07 2012-12-27 Komatsu Ltd Periphery monitoring device of work vehicle

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019121250A (en) * 2018-01-09 2019-07-22 日立建機株式会社 Transport vehicle

Also Published As

Publication number Publication date
JP6339337B2 (en) 2018-06-06
JP2015065515A (en) 2015-04-09

Similar Documents

Publication Publication Date Title
JP6339337B2 (en) Moving object detection system around the vehicle
JP6262068B2 (en) Near-body obstacle notification system
CN106797450B (en) Vehicle body external moving object detection device
JP5529943B2 (en) Work vehicle periphery monitoring system and work vehicle
US10781572B2 (en) Working machine
JP5456123B1 (en) Work vehicle periphery monitoring system and work vehicle
CN109691088B (en) Image processing apparatus, image processing method, and program
JP5550695B2 (en) Work vehicle periphery monitoring system and work vehicle
JP5926315B2 (en) Work vehicle periphery monitoring system and work vehicle
KR20170039615A (en) Periphery monitoring device for crawler-type working machine
US10017112B2 (en) Surroundings monitoring device of vehicle
WO2017038123A1 (en) Device for monitoring area around vehicle
WO2018159019A1 (en) Bird&#39;s-eye-view video image generation device, bird&#39;s-eye-view video image generation system, bird&#39;s-eye-view video image generation method, and program
JP6401141B2 (en) Vehicle obstacle detection device
JP2017074871A (en) Device for detecting obstacle around vehicle
JP6796518B2 (en) Moving object detection system
JP5788048B2 (en) Work vehicle periphery monitoring system and work vehicle
JP6802196B2 (en) Transport vehicle
JP7180172B2 (en) OVERALL VIEW IMAGE GENERATING DEVICE, OVERALL VIEW IMAGE GENERATING METHOD AND PROGRAM
JP2018142883A (en) Bird&#39;s eye video creation device, bird&#39;s eye video creation system, bird&#39;s eye video creation method, and program
JP7145137B2 (en) Working machine controller
JP2015029357A (en) Periphery monitoring system for dump truck and dump truck
JP2014143713A (en) Periphery monitoring system for work vehicle and work vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14849436

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14849436

Country of ref document: EP

Kind code of ref document: A1