EP3620578B1 - Mobile measuring apparatus and mobile measurement method - Google Patents

Mobile measuring apparatus and mobile measurement method Download PDF

Info

Publication number
EP3620578B1
EP3620578B1 EP19191961.2A EP19191961A EP3620578B1 EP 3620578 B1 EP3620578 B1 EP 3620578B1 EP 19191961 A EP19191961 A EP 19191961A EP 3620578 B1 EP3620578 B1 EP 3620578B1
Authority
EP
European Patent Office
Prior art keywords
plane
mobile object
protector
mobile
measuring device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP19191961.2A
Other languages
German (de)
French (fr)
Other versions
EP3620578A1 (en
Inventor
Shoh Tsuritani
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ricoh Co Ltd
Original Assignee
Ricoh Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ricoh Co Ltd filed Critical Ricoh Co Ltd
Publication of EP3620578A1 publication Critical patent/EP3620578A1/en
Application granted granted Critical
Publication of EP3620578B1 publication Critical patent/EP3620578B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • EFIXED CONSTRUCTIONS
    • E01CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
    • E01CCONSTRUCTION OF, OR SURFACES FOR, ROADS, SPORTS GROUNDS, OR THE LIKE; MACHINES OR AUXILIARY TOOLS FOR CONSTRUCTION OR REPAIR
    • E01C23/00Auxiliary devices or arrangements for constructing, repairing, reconditioning, or taking-up road or like surfaces
    • E01C23/01Devices or auxiliary means for setting-out or checking the configuration of new surfacing, e.g. templates, screed or reference line supports; Applications of apparatus for measuring, indicating, or recording the surface configuration of existing surfacing, e.g. profilographs

Landscapes

  • Engineering & Computer Science (AREA)
  • Architecture (AREA)
  • Civil Engineering (AREA)
  • Structural Engineering (AREA)
  • Measurement Of Optical Distance (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Traffic Control Systems (AREA)

Description

    BACKGROUND Technical Field
  • Embodiments of the present disclosure relate to a mobile measuring apparatus and a mobile measurement method.
  • Background Art
  • A mobile measuring apparatus is known that includes a measuring device, such as a distance-measuring device equipped with a camera and a laser, mounted on a mobile object, such as a vehicle and a drone. Such a mobile measuring apparatus is capable of measuring the conditions of the external environment of the own vehicle, such as vehicles around the driver's vehicle, the road conditions, and the tunnel conditions. As an example of the measuring device that measures the conditions of the external environment, a camera that includes a wide-angle lens and a plurality of cameras (stereo cameras), and has a configuration incorporating a laser device for measurement is known (see JP-5729164-B ( JP-2013-009211-A )).
  • In such a mobile measuring apparatus, the pressure (air pressure) of air flowing along the shape of the mobile object after hitting the front part of the mobile object is applied to the measuring device mounted on the mobile object. As a result, the position of the measuring device might be shifted, which might cause an erroneous measurement of the distance between the measurement target, such as a road surface, and the measuring device.
  • However, the configuration of JP-5729164-B ( JP-2013-009211-A ) fails to handle the flow of air around the mobile object, such as a vehicle, at the time of measurement during the movement of the mobile object. That is, the configuration of JP-5729164-B ( JP-2013-009211-A ) fails to prevent the shift of the position of the measuring device due to the air pressure, thus resulting in the erroneous measurement of the distance between the target such as the road surface and the measuring device. CN 201 358 436 Y and CN 108 330 792 A disclose background art to the invention.
  • SUMMARY
  • In one aspect of the invention, there is provided an improved mobile measuring apparatus as defined by claim 1. In another aspect of the invention, there is provided an improved mobile measurement method as defined by claim 4.
  • The embodiments of the present disclosure substantially prevent an erroneous measurement due to the flow of air around the mobile object at the time of measurement during the movement of the mobile object.
  • BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
  • A more complete appreciation of the disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings, wherein:
    • FIG. 1A is a side view of a mobile measuring apparatus according to an embodiment of the present invention;
    • FIG. 1B is a perspective view of the mobile measuring apparatus in FIG. 1A seen from the back side;
    • FIG. 1C is a rear view of the mobile measuring apparatus in FIG. 1A;
    • FIG. 2 is an illustration of the state at which the mobile measuring apparatus according to an embodiment measures a measurement target;
    • FIG. 3 is an example of the flow of air around a mobile object that is moving;
    • FIG. 4 is a cross-sectional view of a measuring device attached to the mobile measuring apparatus according to an embodiment;
    • FIG. 5 is an enlarged perspective view of a part of an area C indicated in FIG. 4;
    • FIG. 6 is a perspective view of the measuring device attached to the mobile measuring apparatus according to an embodiment, as viewed from below;
    • FIG. 7A is a rear view of a fixed frame attached to the mobile measuring apparatus according to an embodiment;
    • FIG. 7B is a perspective view of the fixed frame as viewed from the back side;
    • FIG. 7C is an illustration of the fixed frame as viewed from the lower side;
    • FIG. 7D is an enlarged view of a part of an area E indicated in FIG. 7B;
    • FIG. 8 is an illustration of an example of the advantageous effects of a fixing method performed by the fixed frame; and
    • FIG. 9 is an illustration of a mobile measuring apparatus according to another example not part of the claimed invention.
  • The accompanying drawings are intended to depict embodiments of the present invention and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted.
  • DETAILED DESCRIPTION
  • The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
  • In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.
  • The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result. Hereinafter, modes for carrying out the invention will be described referring to the drawings. Note that identical components are provided with the same reference numerals and repeated descriptions of the same components are omitted.
  • A mobile measuring apparatus 800 according to an embodiment of the present disclosure is described with reference to FIGs. 1A, 1B, and 1C, and 2. FIG. 1A is a side view of the mobile measuring apparatus 800 according to an embodiment of the present disclosure. FIG. 1B is a perspective view of the mobile measuring apparatus 800 in FIG. 1A seen from the back side. FIG. 1C is a rear view of the mobile measuring apparatus 800 in FIG. 1A.
  • FIG. 2 is an illustration of the state at which the mobile measuring apparatus 800 measures a measurement target. In the following description, the direction of travel (movement) of a vehicle 700 is forward (forward direction), the opposite direction of the direction of travel of the vehicle 700 is backward (backward direction). Further, the direction of gravity is downward, and the opposite direction thereof is upward. The right direction is to the right along the horizontal direction relative to the direction of movement. The opposite direction thereof is the left direction. The width direction (right-to-left direction) intersects with each of the direction of movement and the direction of gravity. These directions are indicated in FIGs. 1A, 1B, and 1C, and 2.
  • The mobile measuring apparatus 800 indicated by a dashed dotted line in FIG. 1A includes a vehicle (an example of a mobile object) 700, a fixed frame 750 attached to the vehicle 700, and a measuring device 100 attached to the vehicle 700 using the fixed frame 750. The vehicle 700 includes a vehicle body 701 and tires 702 attached to the vehicle body 701.
  • In the mobile measuring apparatus 800 in FIGs. 1A, 1B, and 1C, and 2, the fixed frame 750 is attached to the back side of the vehicle 700. As illustrated in FIG. 2, a stereo camera (image-capturing device) 130 is attached to the measuring device 100 with the image-capturing area S facing downward. The measuring device 100 is attached to the vehicle body 701 via the fixed frame 750 so as to be disposed within a space 300 (placement space) indicated by a thick broken line in FIGs. 1A, 1B, and 1C, and 2. Further, the measuring device 100 is attached to the vehicle body 701 via the fixed frame 750 so as to be disposed within a plane (placement plane) in the space 300.
  • The space 300 is behind the vehicle 700, and is defined by an upper plane 301, a lower plane 302, a first side plane 303, and a second side plane 304. The upper plane 301 is a plane that intersects with the direction of gravity (the up-to-down direction), and includes the upper end of the vehicle body 701. The lower plane 302 is a plane that intersects with the direction of gravity, and includes the lower end of the vehicle body 701. The first side plane 303 (see FIG. 1C) is a plane that intersects with the width direction, and includes one end of the width direction of the vehicle body 701. The second side plane 304 (see FIG. 1C) is a plane that intersects with the width direction, and includes the other end of the vehicle body 701 along the width direction.
  • The placement plane is within a space behind the vehicle 700, and is defined by the upper plane 301, the lower plane 302, the first side plane 303, and the second side plane 304 as described above.
  • In FIG. 1C, the length Lx in FIG. 1C is the length (width) of the vehicle body 701 in the width direction, and corresponds to the distance between the first side plane 303 and the second side plane 304. The length Ly is the length (height) of the vehicle body 701 in the direction of gravity, and corresponds to the distance between the upper plane 301 and the lower plane 302.
  • With such a configuration, the mobile measuring apparatus 800 captures an image of a road surface 900 (the measurement target) using the stereo camera 130, while moving forward F (in the direction of movement). Thus, the unevenness of the road surface 900 to be measured and the road conditions (whether the white line has been disappeared) can be measured. More specifically, it is possible to measure the flatness (the unevenness of the road surface in the direction of movement), rutting depth, and crack ratio of the road surface 900. Based on the three types of road surface properties, a maintenance control index (MCI) is obtained. The MCI may be obtained by transmitting measurement data to an external device, such as a personal computers (PC) and a tablet terminal, after the measurement. Alternatively, a PC may be mounted on the vehicle 700 so as to obtain the MCI during the measurement that is performed while the vehicle 700 is moving.
  • The placement position of the measuring device 100 with respect to the vehicle 700 is described in more detail below.
  • First, a description is given of the flow of air generated around the vehicle body 701 due to the movement of the vehicle 700 during the measurement of the mobile measuring apparatus 800 with reference to FIG. 3. FIG. 3 is an example of the flow of air around the vehicle body 701 while the vehicle 700 is running (moving). In this example, the wind is blowing backward at a wind speed of 30 meter (m)/second (s), and the vehicle 700 is running (moving) at the speed of 60 kilometers (km)/hour (h). The flow of air generated around the vehicle body 701 is simulated in FIG. 3.
  • In FIG. 3, the flow of air is indicated by a large number of small arrows around the vehicle body 701. The direction of the arrow indicates the direction of air flow, and the size of the arrow indicates the size of the air flow. The density of arrows indicates the density of the air flow.
  • As illustrated in FIG. 3, air collides with the front part of the vehicle body 701 and then flows backward along the shape of the vehicle body 701, around the moving vehicle body 701. For example, air collides with the front panel of the vehicle body 701 in the space area 31 and turns obliquely upward and downward, and then flows backward along the shape of the vehicle body 701. Further, in the space area 32, air collides with the windshield of the vehicle body 701 and turns obliquely upward, and then flows backward along the shape of the vehicle body 701. As described above, air that has collided with the front part of the vehicle body 701 flows backward. Accordingly, a larger air flow is generated in the space area 33 and the space area 34 in the back of the vehicle body 701, than in the space area 35 apart from the vehicle body 701.
  • For example, when the measuring device 100 is attached to the vehicle 700 so as to be positioned in the space area 33 and the space area 34 where a large air flow is generated, the position and angle of the measuring device 100 is shifted due to the pressure (wind pressure) caused by the air flow, which might cause an erroneous measurement of the distance between the measuring device 100 and the road surface 900. In addition, the shaking of the stereo camera 130 due to the wind pressure causes a blur on an image captured by the stereo camera 130, which might further cause a distance measurement error due to the blur of the captured image or might cause a breakage at the position where the measuring device 100 is attached to the fixed frame 750.
  • In view of such circumstances, in an embodiment of the present disclosure, the measuring device 100 is attached to the vehicle 700 via the fixed frame 750 so as to be disposed within the space 300. In the space 300 as illustrated in FIG. 3, the number of arrows is sparse and small, and the size of the arrows is small. This means that the flow of air in the space 300 is smaller than in the space area 33 and the space area 34. Accordingly, this configuration, in which the measuring device 100 is disposed within the space 300, reduces the influence of the air flow and substantially avoids an erroneous measurement of the distance between the measuring device 100 and the road surface 900 caused by the flow of air around the mobile object at the time of measurement during the driving of the mobile object. Further, the breakage at the position where the measuring device 100 is attached to the fixed frame 750 can be avoided as well.
  • In the example of FIGs. 1A, 1B, 1C, 2, and 3, the space 300 is indicated as a space defined in both the forward and backward directions for convenience, but may not be defined in the forward and backward directions as long as the space 300 is behind the vehicle 700. Accordingly, the measuring device 100 may be disposed at a position apart from the vehicle 700 in the back direction within the space 300. However, in view of the stability of the position and angle of the measuring device 100, it is preferable to dispose the measuring device 100 as close as possible to the vehicle 700 within the space 300.
  • Next, the measuring device 100 incorporated in the mobile measuring apparatus 800 according to an embodiment is described with reference to FIG. 4. FIG. 4 is a cross-sectional view of the measuring device 100 incorporated in the mobile measuring apparatus 800 according to an embodiment.
  • The measuring device 100 includes a base frame 110, a camera mount 120, a stereo camera 130, and a protective cover unit 200. One end of the base frame 110 is attached to the fixed frame 750 (see FIGs. 1A, 1B, 1C). The camera mount 120 is attached to the other end of the base frame 110. The stereo camera 130 is fixed by the camera mount 120. In some embodiments, the camera mount 120 may be provided with an angle adjuster that adjusts an image-capturing angle of the stereo camera 130.
  • The stereo camera 130 has two lenses 131a and 131b and is configured to measure the distance to the measurement target by processing acquired image data. Although the following description is given assuming that the measuring device 100 is provided with the stereo camera 130, this is only one example. For example, in some embodiments, the measuring device 100 may include a distance measuring device incorporating a laser so as to measure the distance between the measuring device and the measurement target. Further, the number of distance measuring devices, such as the stereo camera 130, is not limited to one, and a plurality of distance measuring devices may be incorporated in the measuring device 100. By incorporating a plurality of distance measuring devices in the measuring device 100, a wider range of measurement data can be acquired. Further, by employing wide-angle lenses as the lenses 131a and 131b of the stereo camera 130, a wider range of measurement data can be obtained as well.
  • The protective cover unit 200 includes a protector body 210, movable protectors 220a and 220b, and a protector 230. The protector body 210 has openings 211, 212a, and 212b. The movable protectors 220a and 220b are movable in a direction to the opening 211. The protector 230 is configured to protect the opening 211.
  • The protector body 210 includes, for example, a resin plate and an aluminum frame supporting the resin plate, and forms a box having openings 211 on the lower side (that is, the image-capturing side) of the image-capturing device. The stereo camera 130 is disposed in the internal space of the protector body 210.
  • Further, the protector body 210 has a side opening 212a formed at the lower end of the right side, and a side opening 212b is formed at the lower end of the left side. The opening 211 and the side opening 212a communicate with each other, and the opening 211 and the side opening 212b communicate with each other. The positions of the side opening 212b and the side opening 212a differ along the right-to-left direction of the protector body 210, which is the only difference between the side opening 212b and the side opening 212a.
  • As described above, the openings 211, 212a, and 212b forms a single opening that ranges from the lower end of the right side to the lower end of the left side through the lower side of the protector body 210. Further, the protector body 210 includes second protector holders 215a and 215b configured to hold the protector 230 when the opening 211 is open. The positions of the second protector holders 215a and 215b are different from each other along the right-to-left direction of the protector body 210, which is the only difference between the second protector holders 215a and 215b.
  • The movable protector 220a is formed of, for example, a resin plate and an aluminum plate for supporting the resin plate, and is attached to the protector body 210 via a direct-acting member 221a such as a linear slider. The movable protector 220a is a protector movable in the vertical direction and capable of opening and closing the side opening 212a formed on the right side of the protector body 210.
  • That is, the movable protector 220a is configured to close the side opening 212a formed at the lower end of the right side of the protector body 210 so as to protect the lower end of the right side of the protective cover unit 200. Further, the movable protector 220a includes a first protector holder 225a at the lower side, and the first protector holder 225a is configured to hold the protector 230 when the side opening 212a is closed. The movable protector 220a has a supporting mechanism configured to support the movable protector 220a at a predetermined upper position so as to prevent the movable protector 220a from dropping by its own weight. Further, the movable protector 220a is configured not to fall below a predetermined lower position.
  • The movable protector 220b has the same configuration as that of the movable protector 220a. The movable protector 220b is a protector movable in the vertical direction and capable of opening and closing the side opening 212b formed on the left side of the protector body 210.
  • Further, the movable protector 220b includes a first protector holder 225b at the lower side, and the first protector holder 225b is configured to hold the protector 230 when the side opening 212b is closed. The positions of the movable protectors 220a and 220b differ along the right-to-left direction of the protector body 210, which is the only difference between the movable protectors 220a and 220b.
  • The protector 230 is formed of, for example, a flexible resin plate. The right and left end parts of the protector 230 are held so as to be movable along the front-to-back direction. The moving of the protector 230 along the front-to-back direction opens and closes the opening 211 formed at the lower-side of the protector body 210. Preferably, the protector 230 is white.
  • With such a configuration, the sunlight directed to the stereo camera 130 is reflected by the white protector 230, which can reduce or prevent an increase in the temperature inside the stereo camera 130. Further, the material of the protector 230 is preferably made of polycarbonate, which is excellent in impact resistance and heat resistance, for protecting the stereo camera 130 when the opening 211 is closed.
  • The right part of the protector 230 is held by either one or both of the second protector holder 215a of the protector body 210 and the first protector holder 225a of the movable protector 220a. The left part of the protector 230 is held by either one or both of the second protector holder 215b of the protector body 210 and the first protector holder 225b of the movable protector 220b.
  • In the measuring device 100 illustrated in FIG. 4, the second protector holders 215a and 215b are configured as grooves formed in the frames of the protector body 210 and the openings of the grooves face each other along the right-to-left direction as illustrated in FIG. 5.
  • Further, as illustrated in FIG. 5, the first protector holders 225a and 225b are configured as grooves formed in the frames at the lower sides of the movable protector 220, and the opening of the grooves face each other along the right-to-left direction. The protector 230 is held by the grooves with the right-and-left end parts of the protector 230 inserted into the grooves. Such a method of holding the protector 230 is just one example.
  • In the measuring device 100 according to an embodiment, the movable protector 220 is raised to uncover the side openings 212a and 212b at the time of measurement, so that the measurement range can be increased in the right-to-left direction. When the measurement is not performed, the side openings 212a and 212b are closed so that the optical components, such as the lenses 131 of the stereo camera 130, can be prevented from getting dirty with the external environment.
  • The exterior of the protective cover unit 200 is preferably white. With such a configuration, the sunlight directed to the protective cover unit 200 is reflected by the white exterior of the protective cover unit 200, which can substantially prevent an increased in the temperature of the stereo camera 130 disposed within the protective cover unit 200 due to the sunlight.
  • Further, each of the second protector holders 215a and 215b of the protector body 210 preferably has a round bending part. Each of the second protector holders 215a and 215b are formed to extend forward from the opening 211 to the bending portion and extend upward from the bending portion. With this configuration, the width of the second protector holders 215a and 215b along the front-to-back direction can be shortened, and thus the size of the measuring device 100 can be reduced.
  • Next, FIG. 6 is a perspective view of the measuring device 100 as viewed from below. The measuring device 100 includes three stereo cameras 130a, 130b, and 130c. In the example of FIG. 6, the opening 211 at the lower side of the protective cover unit 200 is sectioned by the frames 216a and 216b into three openings 211a, 211b, and 211c. The three openings 211a, 211b, and 211c are provided with the protectors 230a, 230b, and 230c for protecting the respective openings 211a, 211b, and 211c while enabling the openings 211a, 211b, and 211c to be open and closed.
  • A side opening 212a is provided on the right side of the protector body 210A, and is protected by the movable protector 220a so as to be opened and closed. Further, a side opening 212b is provided on the left side of the protector body 210A, and is protected by the movable protector 220b so as to be opened and closed.
  • The holding structure for holding the right part of the protector 230a is the same as the holding structure for holding the right part of the protector 230. In the holding structure for holding the left part of the protector 230a, the left part of the protector 230a is held by a protector holder 217a that is a groove provided at the right side of the frame 216a.
  • In the holding structure for holding the right part of the protector 230b, the left part of the protector 230b is held by a protector holder 217b that is a groove provided at the right side of the frame 216a. In the holding structure for holding the left part of the protector 230b, the left part of the protector 230b is held by a protector holder 217c that is a groove provided at the right side of the frame 216b.
  • In the holding structure for holding the right part of the protector 230c, the right part of the protector 230c is held by a protector holder 217d that is a groove provided at the left side of the frame 216b. The holding structure for holding the left part of the protector 230c is the same as the holding structure for holding the left part of the protector 230 according to an embodiment.
  • With such a configuration, the protectors are openable and closable according to the use situation of the stereo cameras 130a, 130b, and 130c in the measuring device 100. When only the stereo camera 130b is used for example, only the protector 230b is moved to a position to be open, and the movable protectors 220a, 220b and the protectors 230a, 230c remains at the positions to be closed. Accordingly, the stereo cameras 130a and 130c not used for measurement can be protected.
  • The above-described configuration that includes three stereo cameras 130a, 130b, and 130c is just one example. As described above, the number of distance measuring devices, such as stereo cameras, is not limited to any particular number. In some embodiments, the measuring device may include one stereo camera, for example.
  • Next, the fixed frame 750 incorporated in the mobile measuring apparatus 800 according to an embodiment is described with reference to FIGs. 7A, 7B, 7C, and 7D. FIGs. 7A, 7B, 7C, and 7D are illustrations of one example of the configuration of the fixed frame 750. FIG. 7A is a rear view of the fixed frame 750 attached to the mobile measuring apparatus 800 according to an embodiment. FIG. 7B is a perspective view of the fixed frame 750 as viewed from the back side. FIG. 7C is an illustration of the fixed frame 750 as viewed from the lower side. FIG. 7D is an enlarged view of a part of an area E indicated in FIG. 7B.
  • The fixed frame 750 includes vertical columns 751a, 751b, horizontal columns 752a, 752b, a bottom connecting part 753, a fixing plate 754, and L-shaped brackets 755a, 755b, which are attached to a tow pole fixing part 781 and a spare tire fixing part (spare tire carrier) 782 mounted on the vehicle body 701, for example.
  • The tow pole fixing part 781 is attached to the bottom of the vehicle body 701, and is a part to which a tow pole is attached. In the present embodiment, instead of the tow pole, the bottom connecting part 753 to be described later is attached to the tow pole fixing part 781.
  • The tow pole fixing part 781 is formed of a metal prism whose longitudinal direction is along the width direction, and has a rectangular through-hole 781a at the central part of the tow pole fixing part 781 along the width direction, to which a tow pole is inserted in the direction of movement of the vehicle body 701. Further, the central part of the tow pole fixing part 781 in the width direction is positioned at the center of the width of the vehicle body 701 along the width direction.
  • Further, the spare tire fixing part 782 is disposed at the central portion of the back surface of the vehicle body 701, and serves as a part to which a spare tire is fixed. However, in the present embodiment, the fixing plate 754 to be described later is fixed to the spare tire fixing part 782, instead of a spare tire.
  • The spare tire fixing part 782 is a U-shaped bracket having a U-shaped cross section, and a tap hole for screwing the spare tire is provided on the back surface of the U-shaped bracket. Further, the open side of the U-shaped bracket is fixed to the back of the vehicle body 701.
  • The vertical column 751a is formed by connecting a first aluminum frame to a second aluminum frame shorter than the first aluminum frame so as to form an L-shape. The vertical column 751a is disposed such that the longitudinal direction of the first aluminum frame is along the up-to-down direction, and the longitudinal direction of the second aluminum frame is along the direction of movement of the vehicle body 701. The second aluminum frame is attached to the left end face of the tow pole fixing part 781 with the L-shaped bracket 755a (see FIG. 7D) so that the vertical column 751a is attached to the vehicle body 701. The vertical column 751b has the same configuration as that of the vertical column 751a. That is, the vertical column 751b is attached to the vehicle body 701 by attaching the second aluminum frame to the right end face of the tow pole fixing part 781 with the L-shaped bracket 755b. The measuring device 100 is screwed to the vertical columns 751a and 751b.
  • The horizontal column 752a is formed of an aluminum frame. The longitudinal direction of the horizontal column 752a is along the width direction. The horizontal columns 752a is disposed so as to intersect with each of the vertical columns 751a and 751b, and is coupled to the vertical columns 751a and 751b at the respective points of intersection.
  • The horizontal column 752b is disposed at a lower position than the position of the horizontal column 752a, and the longitudinal direction of the horizontal column 752b is parallel to the horizontal column 752a. The horizontal columns 752b is disposed so as to intersect with each of the vertical columns 751a and 751b, and is coupled to the vertical columns 751a and 751b at the respective points of intersection.
  • The bottom connecting part 753 is made of an aluminum frame, and is attached to the tow pole fixing part 781 through the through-hole 781a. The back end surface of the bottom connecting part 753 is provided with a tapped hole for screwing.
  • The fixing plate 754 is a flat metal plate. The fixing plate 754 is disposed so that the flat portion intersects with the direction of movement of the vehicle body 701. Further, the flat portion on the front side of the fixing plate 754 is in contact with the back surface of the U-shaped spare tire fixing part 782 so as to be screwed to the spare tire fixing part 782. Further, the flat portion on the front side of the fixing plate 754 abuts against the end face of the bottom connecting part 753 so as to be screwed to the bottom connecting part 753. Another flat portion on the back side of the fixing plate 754 is in contact with the horizontal column 752b so as to be screwed to the horizontal column 752b.
  • A spare tire cover 790 in FIGs. 7A, 7B, and 7C serves as a cover to protect a spare tire mounted on the vehicle body 701. However, in the present embodiment, since a spare tire is not mounted on the vehicle body 701, the spare tire cover 790 serves to cover, instead of a spare tire, a part of the fixing plate 754 attached to the spare tire fixing part 782. The spare tire cover 790 can be opened and closed by pivoting with, for example, an axis in the up-to-down direction including the left end portion as a pivot axis.
  • In the above-described embodiment, the fixed frame 750 is fixed to the center portion of the back surface of the vehicle body 701 via the spare tire fixing part 782, and also to the bottom of the vehicle body 701 via the tow pole fixing part 781, so that the fixed frame 750 is attached to the vehicle body 701. More specifically, the fixed frame 750 is fixed at two positions, that is, the right end face and the left end face of the tow pole fixing part 781. In other words, the fixed frame 750 is fixed at two positions symmetrical with respect to the center of the vehicle 700 in the width direction, so as to be attached to the bottom of the vehicle body 701. As described above, the measuring device 100 is attached to the vehicle 700 via the fixed frame 750.
  • In some cases, the measuring device 100 rotates around the pitch axis (axis along the width direction), the yaw axis (axis along the direction of gravity), and/or the roll axis (axis along the direction of movement) with the movement of the vehicle 700.
  • Specifically, when the fixed frame 750 is fixed by one position, i.e., the central portion of the back surface of the vehicle body 701, for example, the fixed frame 750 might rotate around the fixed position as a rotation center due to a force, such as wind pressure, applied to a position at which the measuring device 100 is attached to the fixed frame 750.
  • With the rotation of the fixed frame 750, the position and angle of the measuring device 100 is shifted, which might cause an erroneous measurement of the distance between the measuring device 100 and the road surface 900. Further, the shaking of the stereo camera 130 cause a blur on images captured by the stereo camera 130, which might cause a distance measurement error.
  • Furthermore, when the rotation around the pitch axis (pitching) and around the yaw axis (yawing) occurs, a trapezoidal distortion might be generated in the images acquired by the stereo camera 130. If the rotation angles of pitching and yawing are unknown, such a trapezoidal distortion cannot be corrected by, for example, image processing, and the trapezoidal distortion of the image acquired by the stereo camera 130 causes a distance measurement error.
  • In particular, while the vehicle 700 is moving (traveling) during the measurement, the wind direction and the wind speed constantly change, and the rotation angle is likely to be unknown. Further, when a vehicle travelling from the opposite direction passes by the vehicle 700, or when the vehicle 700 passes through the entrance or exits of the tunnel during the measurement, the wind pressure suddenly changes so that the rotation angle is more likely to be unknown.
  • In the present embodiment, the fixed frame 750 is fixed to the vehicle body 701 at two positions: the central portion of the back surface of the vehicle body 701 and the bottom of the vehicle body 701. With this configuration, the rotation such as pitching, yawing, and/or rotation (rolling) around the roll axis is substantially prevented, and thus an erroneous measurement of the distance between the measuring device 100 and the road surface 900 can also be substantially prevented.
  • Further, in the present embodiment, the fixed frame 750 is fixed at two positions symmetrical with respect to the center of the vehicle body 701 in the width direction, so as to be attached to the bottom of the vehicle body 701. With this configuration, for example, the rolling is substantially prevented so as to be increase the stability of the fixed frame 750 attached to the vehicle body 701. Accordingly, the erroneous measurement of the distance between the measuring device 100 and the road surface 900 is further prevented.
  • FIG. 8 is an illustration of an example of the advantageous effects of a fixing method performed by the fixed frame 750. In the example of FIG. 8 for example, the fixed frame 750 is fixed at one position of the central portion of the back surface of the vehicle 700, so as to be attached to the vehicle body 701. In this case, the fixed frame 750 might rotate around the central portion 81 as the rotation center in a direction indicated by arrow 83 due to the wind pressure that acts on the position 82 at which the measuring device 100 is attached to the fixed frame 750. In the present embodiment, since the fixed frame 750 is fixed to the center portion of the back surface of the vehicle body 701 and to the bottom of the vehicle body 701, a force for reducing the rotation is applied to the fixed position 84 of the bottom of the vehicle body 701, which substantially prevents the rotation such as pitching.
  • Note that the fixed frame 750 in FIG. 8 is preferably fixed to the vehicle body 701 with a force applied to the back side of the fixed frame 750, so as to prevent the fixed frame 750 from falling to the vehicle 700 side in the backward direction.
  • In this example, the fixed frame 750 is fixed to the vehicle body 701 using the tow pole fixing part 781 and the spare tire fixing part 782 of the vehicle body 701. However, this is just one example. The fixed frame 750 may be fixed to another part of the vehicle body 701.
  • In the present embodiment, by using the components and mechanism, such as the tow pole fixing part 781 and the spare tire fixing part 782, disposed on the vehicle 700, the fixed frame 750 is attached (fixed) to the vehicle body 701. With this configuration, the fixed frame 750 and the measuring device 100 are attached to the vehicle 700 without any particular modification made to the vehicle 700. If any particular modification is made to the vehicle 700, the vehicle 700 is treated as a dedicated car, and the vehicle license plate has to be changed to one for the dedicated car. However, there is no need to do such a work in the present embodiment.
  • Next, a mobile measuring apparatus according to another embodiment is described below. Like reference signs are given to elements similar to those of the as described above, and overlapping be omitted.
  • In the mobile measuring device according to the above-described embodiment, it is assumed that the measuring device 100 is mounted on the vehicle 700 that travels (moves) on land as a mobile object. This is just one example. As illustrated in FIG. 9 for example, the mobile measuring apparatus 800b may include a measuring device 100b attached to a flying object (drone) 700b that moves in the air.
  • FIG. 9 is an illustration of measurement state of the mobile measuring apparatus 800b according to another embodiment. The mobile measuring apparatus 800b includes the flying object 700b and the measuring device 100b attached to the flying object 700b.
  • With the configuration provided with the flying body 700b, it is possible to measure (inspect) a target in a place (for example, a bridge) where people can not easily approach. Moreover, a rail car is also applicable as a mobile object. In this case, instead of the condition of the road, the condition of the rail on which the rail car travels, the tunnel wall of the subway, and the station's home can be measured with the mobile measuring apparatus.
  • Although the foregoing description is given assuming that the measuring devices 100, 100b according to the above-described embodiments are provided with the stereo camera 130, this is only one example. For example, in some embodiments, the measuring device 100 may include a distance measuring device incorporating a laser so as to measure the distance between the measuring device and the measurement target. Further, the measuring device is not limited to the above-described device for distance measurement, and may be devices for measuring other information.
  • The embodiments also include a mobile measurement method. For example, the mobile measurement method includes attaching, to a mobile object, a measuring device 100, so as to constitute a mobile measuring apparatus 800; and measuring a distance between the measuring device 100 and a measurement target. The measuring device 100 is disposed within a plane in a space 300 in the back of the mobile object in a direction of movement of the mobile object, and the space 300 is defined by an upper plane 301, a lower plane 302, a first side plane 303, and a second side plane 304. The upper plane 301 is a plane that includes an upper end of the mobile object and intersects with a direction of gravity. The lower plane 302 is a plane that includes a lower end of the mobile object and intersects with the direction of gravity. The first side plane 303 is a plane that intersects with a width direction intersecting with each of the direction of movement and the direction of gravity, and includes one end of the mobile object in the width direction, and the second side plane 304 is a plane that intersects with the width direction, and includes the other end of the mobile object in the width direction. Such a mobile measurement method exhibits the same advantageous effects as those of the above-described mobile measuring apparatus.
  • Numerous additional modifications and variations are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the embodiments may be practiced otherwise than as specifically described herein. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of this disclosure and appended claims.

Claims (4)

  1. A mobile measuring apparatus (800) comprising:
    a mobile object (700);
    a fixed frame (750); and
    a measuring device (100) attached to the mobile object (700) via the fixed frame (750) and configured to measure a measurement target,
    wherein the measuring device (100) is disposed within a plane in a space (300) in the back of the mobile object (700) in a direction of movement of the mobile object (700), and the space (300) is defined by an upper plane (301), a lower plane (302), a first side plane (303), and a second side plane (304),
    the upper plane (301) is a plane that includes an upper end of the mobile object (700) and intersects with a direction of gravity,
    the lower plane (302) is a plane that includes a lower end of the mobile object (700) and intersects with the direction of gravity,
    the first side plane (303) is a plane that intersects with a width direction intersecting with each of the direction of movement and the direction of gravity, and includes one end of the mobile object (700) in the width direction, and
    the second side plane (304) is a plane that intersects with the width direction, and includes the other end of the mobile object (700) in the width direction, and characterized in that
    the fixed frame (750) is fixed to a center portion of a back surface of the mobile object (700) and to at least two positions of the bottom of the mobile object (700), the at least two positions being symmetrical with respect to the center of the mobile object (700) along the width direction.
  2. The mobile measuring apparatus (800) according to claim 1,
    wherein the measuring device (100) includes a stereo camera (130).
  3. The mobile measuring apparatus (800) according to claim 1 or 2,
    wherein the mobile object (700) is a vehicle that includes a vehicle body (701) and tires (702) mounted on the vehicle body (701),
    the upper end of the mobile object (700) is an upper end of the vehicle, the lower end of the mobile object (700) is a lower end of the vehicle, the one end of the mobile object (700) in the width direction is one end of the vehicle in the width direction, and the other end of the mobile object (700) in the width direction is the other end of the vehicle in the width direction.
  4. A mobile measurement method comprising:
    attaching a measuring device (100) to a fixed frame (750) and attaching the fixed frame (750) to a mobile object (700), such that the measuring device (100) is disposed within a plane in a space (300) in the back of the mobile object (700) in a direction of movement of the mobile object (700), so as to constitute a mobile measuring apparatus (800); and
    measuring a distance between the measuring device (100) and a measurement target,
    wherein the space (300) is defined by an upper plane (301), a lower plane (302), a first side plane (303), and a second side plane (304),
    the upper plane (301) is a plane that includes an upper end of the mobile object (700) and intersects with a direction of gravity,
    the lower plane (302) is a plane that includes a lower end of the mobile object (700) and intersects with the direction of gravity,
    the first side plane (303) is a plane that intersects with a width direction intersecting with each of the direction of movement and the direction of gravity, and includes one end of the mobile object (700) in the width direction, and
    the second side plane (304) is a plane that intersects with the width direction, and includes the other end of the mobile object (700) in the width direction, and characterized in that
    attaching the fixed frame (750) to the mobile object (700) includes fixing the fixed frame (750) to a center portion of a back surface of the mobile object (700) and to at least two positions of the bottom of the mobile object (700), the at least two positions being symmetrical with respect to the center of the mobile object (700) along the width direction.
EP19191961.2A 2018-09-06 2019-08-15 Mobile measuring apparatus and mobile measurement method Active EP3620578B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2018167287A JP7255110B2 (en) 2018-09-06 2018-09-06 Movement measurement device and movement measurement method

Publications (2)

Publication Number Publication Date
EP3620578A1 EP3620578A1 (en) 2020-03-11
EP3620578B1 true EP3620578B1 (en) 2021-08-04

Family

ID=67658997

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19191961.2A Active EP3620578B1 (en) 2018-09-06 2019-08-15 Mobile measuring apparatus and mobile measurement method

Country Status (3)

Country Link
EP (1) EP3620578B1 (en)
JP (1) JP7255110B2 (en)
ES (1) ES2887785T3 (en)

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS56112213A (en) 1980-02-13 1981-09-04 Kimura Shindai Kogyo Kk Drive apparatus for elevating floor part
JP3418328B2 (en) * 1998-02-02 2003-06-23 株式会社朋栄 Method and apparatus for measuring road surface unevenness
DE10061954A1 (en) * 2000-12-11 2002-06-27 Oris Fahrzeugteile Riehle H Storage system for transporting articles on back of vehicle comprises cross-bar forming integral part of rear bumper and which has apertures into which pins on hollow shelf fit
AU2003274459A1 (en) 2002-10-31 2004-05-25 Fabio Pedrini Vehicle-mounted equipment carrier
CN201358436Y (en) 2009-02-10 2009-12-09 长安大学 Road surface track laser detection device
JP5729164B2 (en) 2011-06-27 2015-06-03 株式会社富士通ゼネラル Car camera
JP6269409B2 (en) 2014-09-17 2018-01-31 三菱電機株式会社 Gauge measuring device, gauge measuring method
US10190269B2 (en) * 2016-01-15 2019-01-29 Fugro Roadware Inc. High speed stereoscopic pavement surface scanning system and method
CN108330792A (en) 2018-02-07 2018-07-27 北京北铃专用汽车有限公司 Road detection vehicle

Also Published As

Publication number Publication date
EP3620578A1 (en) 2020-03-11
JP2020041824A (en) 2020-03-19
ES2887785T3 (en) 2021-12-27
JP7255110B2 (en) 2023-04-11

Similar Documents

Publication Publication Date Title
US10829058B2 (en) Combinatory sensor apparatus
CN101326424B (en) Stereo camera
JP6807543B2 (en) Imaging device
ES2637173T3 (en) Vision system for vehicles, in particular commercial vehicles
WO2013080706A1 (en) Vehicle-mounted camera attaching device
KR20090092602A (en) System of Traffic Regulation Using Image Fusion
EP2952388B1 (en) A mounting apparatus for mounting an automotive sensor
KR102506787B1 (en) Sensor combination module for a motor vehicle
RU2603170C2 (en) Display system and vehicle equipped with it
CN109421612A (en) The sensor assemblies of pilotless automobile
EP3620578B1 (en) Mobile measuring apparatus and mobile measurement method
JP5851009B2 (en) In-vehicle camera mounting device
US8115161B2 (en) Imaging apparatus for taking images of objects in a plurality of directions and vehicle incorporating the same
CN204340870U (en) The recording device of a kind of single camera panorama
ES2952551T3 (en) Vehicle scanning equipment
CN212781218U (en) Sensor device and autonomous vehicle
US11856304B2 (en) Measurement apparatus, movable apparatus, and vehicle
JP7052421B2 (en) Measuring device and mobile measuring device
KR101439247B1 (en) Lens structure for camera of black box equipped in a vehicle
JP7354706B2 (en) Measuring device unit
US11634087B1 (en) Housing structure for a vehicle mounted optical device
EP3620577A1 (en) Measuring device and mobile measuring apparatus incorporating the measuring device
CN109471315A (en) A kind of vehicle bottom image acquiring device
KR20110058224A (en) Apparatus for recording accident information of a vehicle
ITRM20040113U1 (en) SUPPORT FOR MOTORIZED TILTING OF MICRO CAMERAS.

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190815

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210326

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1417113

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210815

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602019006583

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2887785

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20211227

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1417113

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210804

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211104

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211206

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211104

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211105

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602019006583

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210815

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20220506

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210815

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220831

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220831

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230522

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20190815

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20230821

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230825

Year of fee payment: 5

Ref country code: GB

Payment date: 20230822

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230828

Year of fee payment: 5

Ref country code: DE

Payment date: 20230821

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231027

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210804