EP3048012A1 - Sichtsystem und verfahren zur überwachung der umgebung einer maschine - Google Patents
Sichtsystem und verfahren zur überwachung der umgebung einer maschine Download PDFInfo
- Publication number
- EP3048012A1 EP3048012A1 EP15003618.4A EP15003618A EP3048012A1 EP 3048012 A1 EP3048012 A1 EP 3048012A1 EP 15003618 A EP15003618 A EP 15003618A EP 3048012 A1 EP3048012 A1 EP 3048012A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- cameras
- section
- machine
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 27
- 238000012544 monitoring process Methods 0.000 title claims description 14
- 230000009466 transformation Effects 0.000 claims abstract description 33
- 238000003702 image correction Methods 0.000 claims description 15
- 238000004891 communication Methods 0.000 claims description 10
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 4
- 239000002131 composite material Substances 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
- B60R2300/602—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective with an adjustable viewpoint
Definitions
- the present disclosure relates to a vision system for a machine, and more particularly relates to a vision system and a method of monitoring surroundings of a machine.
- Machines such as an articulated truck and a motor grader, include a vision system for monitoring surroundings of the machine.
- the machine includes a front section and a rear section.
- the front section is pivotally coupled to the rear section via an articulation joint.
- One or more cameras may be disposed on the front section and the rear section of the machine to monitor surroundings of the machine. Images captured by the cameras may be processed by a controller and displayed in a monitor for an operator to determine whether any personnel and/or object are around the machine. When multiple cameras are used, images of two or more cameras may overlap. Further, when the front section moves relative to the rear section, images of the cameras mounted on the front section and the rear section may overlap each other. To avoid such overlapping of the images, additional control modules may be required, and hence processing of the images may become complex.
- US Patent Publication Number 2014/0088824 discloses a surrounding monitoring system for an articulated machine having a front section and a rear section.
- the surrounding monitoring system includes a plurality of cameras mounted on the front section and a plurality of cameras mounted on the rear section. An image of the surrounding of the machine is displayed in a monitor.
- the surrounding monitoring system includes a camera position identifying unit to determine position of cameras based on angle of articulation between the front section and the rear section.
- the surrounding monitoring system further includes an image transformation device for converting the camera image into a bird's eye view image and an image composing means for converting the individual bird's eye view image to a composite bird's eye view image.
- the surrounding monitoring system further includes a display image forming device for displaying the composite bird's eye view image along with an image of the machine.
- a vision system for a machine includes a first section, a second section and an articulation joint pivotally connecting the first section to the second section.
- the vision system includes a first set of cameras mounted on the first section of the machine, a second set of cameras mounted on the second section of the machine, and a controller communicably coupled to the first set of cameras and the second set of cameras.
- the controller includes a first image transformation module configured to convert a first image captured by the first set of cameras into a first transformed image based at least on a view type.
- the controller further includes a second image transformation module configured to convert a second image captured by the second set of cameras into a second transformed image based at least on a view type.
- the controller further includes an image combination module configured to receive the first transformed image from the first image transformation module and the second transformed image from the second image transformation module.
- the image combination module is further configured to combine the first transformed image and the second transformed image to obtain a combined image with reference to the articulation joint.
- the combined image is indicative of surroundings of the first section and the second section of the machine.
- the vision system further includes a display device communicably coupled to the controller. The display device is configured to display the combined image.
- a machine in another aspect of the present disclosure, includes a first section including a first set of ground engaging members, a second section including a second set of ground engaging members, and an articulation joint configured to pivotally couple the first section to the second section.
- the machine further includes a vision system.
- the vision system includes a first set of cameras mounted on the first section, a second set of cameras mounted on the second section, and a controller communicably coupled to the first set of cameras and the second set of cameras.
- the controller includes a first image transformation module configured to convert a first image captured by the first set of cameras into a first transformed image based at least on a view type.
- the controller further includes a second image transformation module configured to convert a second image captured by the second set of cameras into a second transformed image based at least on a view type.
- the controller further includes an image combination module configured to receive the first transformed image from the first image transformation module and the second transformed image from the second image transformation module.
- the image combination module is further configured to combine the first transformed image and the second transformed image to obtain a combined image with reference to the articulation joint.
- the combined image is indicative of surroundings of the first section and the second section of the machine.
- the vision system further includes a display device communicably coupled to the controller. The display device is configured to display the combined image.
- a method of monitoring surroundings of a machine includes a first section, a second section, and an articulation joint pivotally connecting the first section to the second section.
- the method includes capturing a first image via a first set of cameras.
- the first set of cameras is mounted on the first section of the machine.
- the method further includes capturing a second image via a second set of cameras.
- the second set of cameras is mounted on the second section of the machine.
- the method further includes converting the first image captured by the first set of cameras into a first transformed image based at least on a view type and converting the second image captured by the second set of cameras into a second transformed image based at least on a view type.
- the method further includes combining the first transformed image and the second transformed image to obtain a combined image with reference to the articulation joint.
- the combined image is indicative of surroundings of the first section and the second section of the machine.
- the method further includes displaying the combined image via a display device.
- FIG. 1 is a side view of a machine 100, according to an embodiment of the present disclosure.
- the machine 100 may be an articulated truck used for various industries including, but not limited to, construction, mining, agriculture and transportation. It may also be contemplated that the machine 100 may be any type of on-road or off-road articulated trucks.
- the machine 100 includes a first section 102 and a second section 104 pivotally coupled to the first section 102.
- the first section 102 may correspond to a front frame 103 of the machine 100.
- the second section 104 may correspond to a rear frame 105 of the machine 100.
- the first section 102 includes a first set of ground engaging members 106.
- the first set of ground engaging members 106 may be a pair of wheels. Each of the pair of wheels may be disposed at each side of the first section 102 of the machine 100.
- the second section 104 includes a second set of ground engaging members 108.
- the second set of ground engaging members 108 may be one or more pair of wheels. Each of the pair of wheels may be disposed at each side of the second section 104 of the machine 100.
- at least one of the first set and the second set of ground engaging members 106, 108 may be a track assembly.
- the machine 100 further includes an articulation joint 110.
- the articulation joint 110 is configured to pivotally couple the first section 102 to the second section 104.
- the first section 102 may be movable to various angular positions relative to the second section 104 about a vertical axis 'A' defined by the articulation joint 110.
- the second section 104 further includes an operator cab 112.
- the operator cab 112 may include input devices, such as control levers, joystick, and the like, for driving the machine 100 to forward and backward directions, and for controlling operation of a load carrying body 114 disposed at the second section 104.
- the input devices may also be used for turning the first section 102 to different angular positions relative to the second section 104 about the vertical axis 'A'.
- the machine 100 further includes a vision system 120 configured to display surroundings of the machine 100 via a display device 122.
- the surroundings of the machine 100 may be a ground surface of a worksite around the machine 100.
- the vision system 120 is further configured to display the surroundings of the machine 100 with respect to the first section 102 and the second section 104 of the machine 100.
- a view of the surroundings of the machine 100 may be defined within a predefined perimeter relative to the machine 100.
- the display device 122 is disposed within the operator cab 112. In other embodiments, the display device 122 may be disposed at a remote location so that the surroundings of the machine 100 in the worksite may be viewed from the remote location.
- FIG. 2 is a schematic top view of the machine 100 having a first set of cameras 124 and a second set of cameras 128 disposed thereon, according to an embodiment of the present disclosure.
- the vision system 120 includes the first set of cameras 124 mounted on the first section 102 of the machine 100.
- the first set of cameras 124 is configured to capture a first image 'FI' corresponding to surroundings of the first section 102.
- the first set of cameras 124 includes a front camera 124a mounted adjacent to a front end 126 of the first section 102 and a first side camera 124b mounted adjacent to each of the sides of the first section 102.
- the vision system 120 further includes the second set of cameras 128 mounted on the second section 104 of the machine 100.
- the second set of cameras 128 is configured to capture a second image 'SI' corresponding to surroundings of the second section 104.
- the second set of cameras 128 includes a rear camera 128a mounted adjacent to a rear end 130 of the second section 104 and a second side camera 128b mounted adjacent to each of the sides of the second section 104.
- images captured by the cameras 128a and 128b may be stitched together to form the second image SI.
- at least one camera may also be disposed on the second section 104 of the machine 100.
- the first set of cameras 124 and the second set of cameras 128 may be disposed adjacent to bottom surfaces of the front frame 103 and the rear frame 105 of the first section 102 and the second section 104, respectively.
- the first set of cameras 124 and the second set of cameras 128 may be disposed at any location of the first section 102 and the second section 104, respectively.
- the number of first and second set of cameras 124, 128 shown in FIG. 2 is exemplary in nature, and in various embodiments, the first section 102 and the second section 104 may include any number of cameras as per requirements.
- the vision system 120 further includes a controller 132 communicably coupled to the first set of cameras 124 and the second set of cameras 128.
- the controller 132 may be disposed within the operator cab 112. However, it may be contemplated that the controller 132 may be disposed at any location of the machine 100. In other embodiments, the controller 132 may be disposed at a remote location.
- the controller 132 is also configured to be in communication with the display device 122.
- the display device 122 may be integrally configured with the controller 132 depends on applications of the vision system 120 for the machine 100.
- the controller 132 may be embodied as a vision control unit associated the machine 100.
- the controller 132 may further include input/output ports configured to be coupled with the first set of cameras 124, the second set of cameras 128 and the display device 122.
- the controller 132 may also configured to be in communication with one or more input devices to receive inputs from an operator.
- FIG. 3 shows a block diagram of the controller 132, according to an embodiment of the present disclosure.
- the controller 132 includes a first image transformation module 134a configured to convert the first image 'FI' into a first transformed image.
- the first image transformation module 134a is configured to receive images from the front camera 124a and each of the first side cameras 124b.
- the received camera images from the first set of cameras 124 may be collectively referred as the first image 'FI'.
- Each of the received camera images may be further transformed to the first transformed image, such as a top surround view.
- the first transformed image may correspond to the surroundings of the first section 102 of the machine 100.
- the controller 132 further includes a second image transformation module 134b configured to convert the second image 'SI' into a second transformed image.
- a second image transformation module 134b configured to convert the second image 'SI' into a second transformed image.
- images received from the rear camera 128a and each of the second side cameras 128b, collectively referred as the second image 'SI' may be transformed to the second transformed image.
- the second transformed image may correspond to the surroundings of the second section 104 of the machine 100.
- the controller 132 further includes a first image configuration module 136a.
- the first image configuration module 136a is configured to be in communication with the first image transformation module 134a and an image combination module 138.
- the first image configuration module 136a may be configured to provide input corresponding to a view type of the image of the surroundings and the machine 100.
- the view type of the image is one of a three dimensional view, a perspective view, a two dimensional view, a top surround view and a side view based on input from the operator.
- the controller 132 further includes a second image configuration module 136b.
- the second image configuration module 136b is configured to be in communication with the second image transformation module 134b and the image combination module 138.
- the image combination module 138 is configured to receive the first transformed image from the first image transformation module 134a and the second transformed image from the second image transformation module 134b.
- the image combination module 138 is further configured to receive inputs corresponding to the view type of the image from the first and the second image configuration module 136a, 136b.
- the image combination module 138 is further configured to combine the first transformed image and the second transformed image to obtain a combined image (shown in FIG. 4 ) with reference to the articulation joint 110 based on the view type.
- the controller 132 further includes a first image selection module 140a and a second image selection module 140b.
- the first image selection module 140a is configured to be in communication with the first image configuration module 136a and the second image selection module 140b is configured to be in communication with the second image configuration module 136b.
- the first and the second image selection modules 140a, 140b are configured to communicate the view type with the first and the second image configuration modules 136a, 136b, respectively, based on input from the operator.
- the first and the second image transformation module 134a, 134b are configured to convert the first image 'FI' and the second image 'SI' into the first transformed image and the second transformed image based at least the view type selected by the operator via the first and the second image selection module 140a, 140b, respectively.
- the controller 132 further includes a first image correction module 142a configured to correct the first image 'FI' captured by the first set of cameras 124 based on a calibration parameter associated with the first set of cameras 124.
- the first image correction module 142a is configured to be in communication with the front camera 124a and each of the first side cameras 124b to receive corresponding individual image signals.
- the first image correction module 142a is further configured to rectify distortion in each of the camera images received from the first set of cameras 124. Distortion in the images may be rectified based on the calibration parameter preset in the front camera 124a and each of the first side cameras 124b.
- the calibration parameter may include, but not limited to, focal length, image sensor format and principal point. Further, the calibration parameter may be set based on pixel quality of the first image 'FI' to be captured.
- the calibration parameter of the front camera 124a and each of the first side cameras 124b may be preset by manufacturer of the cameras. It may also be contemplated that the calibration parameter of each camera may be set by the operator.
- the first image correction module 142a is further configured to be in communication with the first image transformation module 134a. Thus, the first image correction module 142a receives the first image 'FI' from the first set of cameras 124 and communicates a corrected image with the first image transformation module 134a.
- the controller 132 further includes a second image correction module 142b configured to correct the second image 'SI' captured by the second set of cameras 128 based on a calibration parameter associated with the second set of cameras 128.
- the second image correction module 142b is also configured to receive the second image 'SI' from the rear camera 128a and each of the second side cameras 128b, and communicates the corrected image to the second image transformation module 134b.
- FIG. 4 shows the display device 122 displaying the combined image of the machine 100, according to an embodiment of the present disclosure.
- the image combination module 138 may communicate the combined image with the display device 122 to display the image of the machine 100 along with the surroundings of the first section 102 and the second section 104.
- the combined image is indicative of the surroundings of the first section 102 and the second section 104 of the machine 100.
- the display device 122 may be a CRT display, a LED display, a LCD display, or any other display device known in the art.
- the display device 122 may receive the first transformed image corresponding to the surroundings of the first section 102.
- the second transformed image corresponding to the surroundings of the second section 104 may be received by the display device 122.
- the first transformed image and the second transformed image may be further combined with reference to the articulation joint 110 of the machine 100 and displayed as the combined image on the display device 122.
- the present disclosure relates to the vision system 120 and a method 500 of monitoring the surroundings of the machine 100 in the worksite.
- the vision system 120 includes the first set of cameras 124 disposed on the first section 102 and the second set of cameras 128 disposed on the second section 104 to display the surroundings of the first section 102 and the second section 104, respectively.
- the image combination module 138 combines the surroundings of the first section 102 and the second section 104 and display as the combined image along with the image of the machine 100 in the display device 122.
- FIG. 5 is a flowchart of the method 500 of monitoring the surroundings of the machine 100, according to an embodiment of the present disclosure.
- the method 500 may be described in detail with respect to various steps.
- the method 500 includes capturing the first image 'FI' via the first set of cameras 124 mounted on the first section 102 of the machine 100.
- the front camera 124a captures the surroundings of the machine 100 adjacent to the front end 126 thereof.
- Each of the first side cameras 124b captures the surroundings of the machine 100 adjacent to the respective sides of the first section 102.
- the first image 'FI' corresponding to the surroundings of the first section 102 of the machine 100 is captured by the first set of cameras 124.
- the method 500 includes capturing the second image 'SI' via the second set of cameras 128 mounted on the second section 104 of the machine 100.
- the rear camera 128a captures the surroundings of the machine 100 adjacent to the rear end 130 thereof.
- Each of the second side cameras 128b captures the surroundings of the machine 100 adjacent to the respective sides of the second section 104.
- the second image 'SI' corresponding to the surroundings of the second section 104 of the machine 100 is captured by the second set of cameras 128.
- the first and the second set of cameras 124, 128 may capture image of the surroundings of the first section 102 and the second section 104, respectively, based on input from the operator. In another embodiment, the first and the second set of cameras 124, 128 may capture image of the surroundings of the first section 102 and the second section 104, respectively, based on machine operating conditions, such as activation of an engine associated with the machine 100.
- the method 500 further includes correcting the first image 'FI' captured by the first set of cameras 124 based on the calibration parameter associated with the first set of cameras 124.
- the first image 'FI' captured by the first set of cameras 124 is communicated to the first image correction module 142a.
- the first image correction module 142a rectifies distortion in the image based on the calibration parameter.
- the method 500 further includes correcting the second image 'SI' captured by the second set of cameras 128 based on the calibration parameter associated with the second set of cameras 128.
- the second image 'SI' captured by the second set of cameras 128 is communicated to the second image correction module 142b.
- the second image correction module 142b rectifies distortion in the image based on the calibration parameter.
- the method 500 includes converting the first image 'FI' captured by the first set of cameras 124 into the first transformed image based at least on the view type.
- the corrected camera image from the first image correction module 142a is communicated to the first image transformation module 134a.
- the first image transformation module 134a converts the first image 'FI' into the first transformed image. Further, the first image 'FI' is converted into the first transformed image based on the input corresponding to the view type received from the first image configuration module 136a.
- the first image configuration module 136a receives input from the first image selection module 140a based on the input from the operator.
- the method 500 includes converting the second image 'SI' captured by the second set of cameras 128 into the second transformed image based at least on the view type.
- the corrected camera image from the second image correction module 142b is communicated to the second image transformation module 134b.
- the second image transformation module 134b converts the second image 'SI' into the second transformed image.
- the second image 'SI' is converted into the second transformed image based on the signal corresponding to the view type from the second image configuration module 136b.
- the second image configuration module 136b receives input from the second image selection module 140b based on the input from the operator.
- the method 500 includes combining the first transformed image and the second transformed image to obtain the combined image with reference to the articulation joint 110.
- the first image transformation module 134a communicates the first transformed image with the image combination module 138.
- the second image transformation module 134b communicates the second transformed image with the image combination module 138.
- the image combination module 138 combines the first transformed image and the second transformed image to form the combined image with reference to the articulation joint 110.
- the method 500 includes displaying the combined image via the display device 122.
- the image combination module 138 communicates signals corresponding to the combined image to the display device 122.
- the surroundings of the first section 102 and the surroundings of the second section 104 is combined with reference to the articulation joint 110 of the machine 100.
- the vision system 120 and the above method 500 of monitoring the surroundings of the machine 100 may process the images received from the first set of cameras 124 and the second set of cameras 128 separately.
- the images may be transformed into the first transformed image and the second transformed image.
- the first transformed image and the second transformed image may be combined with reference to the articulation joint 110 of the machine 100.
- overlapping of the first set of cameras 124 and the second set of cameras 128 may be avoided.
- processing of the images received from the first and the second set of cameras 124, 128 may be simplified as separate control module for eliminating overlapping of the images is not required.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Closed-Circuit Television Systems (AREA)
- Studio Devices (AREA)
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/601,320 US9667875B2 (en) | 2015-01-21 | 2015-01-21 | Vision system and method of monitoring surroundings of machine |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3048012A1 true EP3048012A1 (de) | 2016-07-27 |
EP3048012B1 EP3048012B1 (de) | 2018-01-17 |
Family
ID=55069649
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15003618.4A Active EP3048012B1 (de) | 2015-01-21 | 2015-12-18 | Sichtsystem und verfahren zur überwachung der umgebung einer maschine |
Country Status (2)
Country | Link |
---|---|
US (1) | US9667875B2 (de) |
EP (1) | EP3048012B1 (de) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3020868B1 (de) * | 2014-11-14 | 2020-11-04 | Caterpillar Inc. | Maschine mit einem Maschinenkörper und einem hinsichtlich des Körpers beweglichen Werkzeug mit einem System zur Unterstützung eines Benutzers der Maschine |
JPWO2016157463A1 (ja) * | 2015-03-31 | 2018-01-18 | 株式会社小松製作所 | 作業機械の周辺監視装置 |
JP6754594B2 (ja) * | 2016-03-23 | 2020-09-16 | 株式会社小松製作所 | モータグレーダ |
WO2017131243A1 (ja) * | 2017-02-09 | 2017-08-03 | 株式会社小松製作所 | 作業車両の周辺監視システム、作業車両、及び作業車両の周辺監視方法 |
US11409299B2 (en) * | 2020-08-19 | 2022-08-09 | Caterpillar Inc. | Determining object detection area based on articulation angle |
CN111942391B (zh) * | 2020-09-15 | 2024-06-11 | 徐工集团工程机械股份有限公司道路机械分公司 | 铰接式工程机械、全景环视系统及其标定方法 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2012105158A (ja) * | 2010-11-11 | 2012-05-31 | Mitsubishi Fuso Truck & Bus Corp | 連結車両用鳥瞰図表示システム |
US20140088824A1 (en) | 2011-05-13 | 2014-03-27 | Hitachi Construction Machinery Co., Ltd. | Device for Monitoring Area Around Working Machine |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6693524B1 (en) * | 1998-06-02 | 2004-02-17 | George R. Payne | Vehicle backup monitoring and alarm system |
US6268803B1 (en) * | 1998-08-06 | 2001-07-31 | Altra Technologies Incorporated | System and method of avoiding collisions |
JP3298851B2 (ja) * | 1999-08-18 | 2002-07-08 | 松下電器産業株式会社 | 多機能車載カメラシステムと多機能車載カメラの画像表示方法 |
ATE431605T1 (de) * | 2002-07-17 | 2009-05-15 | Fico Mirrors Sa | Einrichtung zur aktiven überwachung der äusseren sicherheitsbegrenzung eines kraftfahrzeugs |
DE102004045974A1 (de) * | 2004-09-22 | 2006-03-23 | Mekra Lang Gmbh & Co. Kg | System zur Übertragung von Signalen in einem Kraftfahrzeug |
JP5124351B2 (ja) * | 2008-06-04 | 2013-01-23 | 三洋電機株式会社 | 車両操作システム |
JP5269026B2 (ja) * | 2010-09-29 | 2013-08-21 | 日立建機株式会社 | 作業機械の周囲監視装置 |
JP2012252675A (ja) * | 2011-06-07 | 2012-12-20 | Komatsu Ltd | ダンプトラック |
JP5750344B2 (ja) | 2011-09-16 | 2015-07-22 | 日立建機株式会社 | 作業機の周囲監視装置 |
US9168871B2 (en) * | 2011-10-28 | 2015-10-27 | Ford Global Technologies | Rear-view mirror with multi-mode display screen, system comprising same, and method of using same |
JP6167042B2 (ja) | 2012-01-12 | 2017-07-19 | 日立建機株式会社 | 自走式産業機械の周囲監視装置 |
JP5938292B2 (ja) | 2012-08-03 | 2016-06-22 | 日立建機株式会社 | 運搬車両の監視装置 |
-
2015
- 2015-01-21 US US14/601,320 patent/US9667875B2/en active Active
- 2015-12-18 EP EP15003618.4A patent/EP3048012B1/de active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2012105158A (ja) * | 2010-11-11 | 2012-05-31 | Mitsubishi Fuso Truck & Bus Corp | 連結車両用鳥瞰図表示システム |
US20140088824A1 (en) | 2011-05-13 | 2014-03-27 | Hitachi Construction Machinery Co., Ltd. | Device for Monitoring Area Around Working Machine |
Also Published As
Publication number | Publication date |
---|---|
EP3048012B1 (de) | 2018-01-17 |
US20160212352A1 (en) | 2016-07-21 |
US9667875B2 (en) | 2017-05-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3048012B1 (de) | Sichtsystem und verfahren zur überwachung der umgebung einer maschine | |
JP4221014B2 (ja) | ロボット制御装置 | |
US10007853B2 (en) | Image generation device for monitoring surroundings of vehicle | |
CN104460361B (zh) | 车辆自动运动控制系统 | |
EP3374965B1 (de) | Fahrzeugkamerasystem mit multikameraausrichtung | |
EP2410740B1 (de) | Umgebungsbereichsüberwachungsvorrichtung für fahrzeuge und umgebungsbereichsüberwachungsverfahren für fahrzeuge | |
EP2237224B1 (de) | Kalibriervorrichtung für eine eingebaute Fahrzeugkamera | |
CN102959953B (zh) | 图像校准方法和装置 | |
AU2004314869B2 (en) | Video image positional relationship correction apparatus steering assist apparatus having the video image positional relationship correction apparatus and video image positional relationship correction method | |
US20150009329A1 (en) | Device for monitoring surroundings of machinery | |
WO2015029934A1 (ja) | カメラ校正装置、カメラ校正システム、及びカメラ校正方法 | |
EP2717570A1 (de) | Vorrichtung zur überwachung des bereichs rund um eine arbeitsmaschine | |
US10017112B2 (en) | Surroundings monitoring device of vehicle | |
JP2008143701A (ja) | 産業車両の視界改善システム及び方法 | |
CN101340566A (zh) | 图像处理系统和方法 | |
EP2991343B1 (de) | Vorrichtung zur erzeugung eines zu verarbeitenden bildes, verfahren zur erzeugung eines zu verarbeitenden bildes und betriebshilfssystem | |
JP2010246088A (ja) | 車載カメラ装置の画像調整装置及び方法、並びに車載カメラ装置 | |
JP2008269139A (ja) | 運転支援システム並びに車両 | |
JP5959311B2 (ja) | データ導出装置、及び、データ導出方法 | |
WO2014054753A1 (ja) | 画像処理装置及び車両前方監視装置 | |
CN109070801B (zh) | 使用后置相机的拖车角度检测 | |
WO2016129552A1 (ja) | カメラパラメータ調整装置 | |
WO2014054752A1 (ja) | 画像処理装置及び車両前方監視装置 | |
US10889958B2 (en) | Display system for machine | |
EP2490175A1 (de) | Verfahren zur Kalibrierung und/oder Ausrichtung einer auf einem Automobil montierten Kamera sowie zugehörige Kamera |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
17P | Request for examination filed |
Effective date: 20161214 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20170802 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602015007508 Country of ref document: DE Ref country code: AT Ref legal event code: REF Ref document number: 964177 Country of ref document: AT Kind code of ref document: T Effective date: 20180215 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20180117 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 964177 Country of ref document: AT Kind code of ref document: T Effective date: 20180117 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180417 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180417 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180517 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180418 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602015007508 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
26N | No opposition filed |
Effective date: 20181018 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181218 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20181231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181218 Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181231 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181218 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180117 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180117 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20151218 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230517 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231121 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20231121 Year of fee payment: 9 |