WO2019176118A1 - Superimposed display system - Google Patents

Superimposed display system Download PDF

Info

Publication number
WO2019176118A1
WO2019176118A1 PCT/JP2018/010642 JP2018010642W WO2019176118A1 WO 2019176118 A1 WO2019176118 A1 WO 2019176118A1 JP 2018010642 W JP2018010642 W JP 2018010642W WO 2019176118 A1 WO2019176118 A1 WO 2019176118A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
laser scanner
scan line
superimposed
posture
Prior art date
Application number
PCT/JP2018/010642
Other languages
French (fr)
Japanese (ja)
Inventor
吉田　光伸
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to PCT/JP2018/010642 priority Critical patent/WO2019176118A1/en
Priority to JP2020506104A priority patent/JP7003219B2/en
Priority to TW107118478A priority patent/TWI645160B/en
Publication of WO2019176118A1 publication Critical patent/WO2019176118A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/40—Means for monitoring or calibrating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images

Definitions

  • the present invention relates to a superimposed display system for calibration of a camera and a laser scanner used in a mobile mapping system (hereinafter referred to as MMS).
  • MMS mobile mapping system
  • a device for estimating a vehicle position, a camera, and a laser scanner are mounted on a measurement vehicle. It is widely performed that the shape of the road and the surroundings of the road is measured by the traveling of the MMS measuring vehicle.
  • the mounting position or mounting posture of the camera and laser scanner may change due to aging and small collisions.
  • the measurement accuracy deteriorates. Therefore, in order to prevent measurement accuracy deterioration, calibration for ensuring accuracy is performed (for example, Patent Document 1).
  • an object of the present invention is to provide a system for performing calibration without mounting a camera and a laser scanner on a vehicle.
  • the superimposed display system of the present invention includes: A base to which a camera and a laser scanner are fixed; An attitude changing mechanism that fixes the base to which the camera and the laser scanner are fixed, and changes a roll angle, a pitch angle, and a yaw angle of the base; A measurement object having a measurement pattern which is a pattern photographed by the camera and a pattern in which a difference in reflected luminance appears by scanning with the laser scanner; A camera image of the measurement pattern photographed by the camera and a scan line that is a scan result of the measurement pattern by the laser scanner are acquired, and the camera image and the scan line are superimposed and displayed on a display device.
  • a computer A computer.
  • FIG. 3 is a diagram illustrating the hardware configuration of the computer according to the first embodiment.
  • FIG. 3 is a diagram illustrating the rotation of the laser scanner in the first embodiment.
  • FIG. 4 is another diagram showing the rotation of the laser scanner in the first embodiment.
  • FIG. 5 is another view showing the rotation of the laser scanner in the first embodiment.
  • FIG. 3 is a diagram of Embodiment 1 and shows a yaw angle difference between a camera and a laser scanner.
  • FIG. 3 is a diagram of the first embodiment and shows a pitch angle difference between the camera and the laser scanner.
  • FIG. 4 is a diagram of Embodiment 1, showing a roll angle difference between a camera and a laser scanner.
  • FIG. 3 is another diagram showing the roll angle difference between the camera and the laser scanner in the first embodiment.
  • FIG. 1 shows an overview of the superimposed display system 100.
  • FIG. 2 is a perspective view of the test apparatus 30 and the measurement object 40.
  • the measurement object 40 has a measurement pattern 41.
  • a roll shaft 21, a pitch shaft 22, and a yaw shaft 23, which will be described later, are not visible.
  • the test apparatus 30 is disposed on the floor 82.
  • FIG. 3 shows a hardware configuration of the computer 50.
  • the display control unit 51 a of the computer 50 includes the camera image 11 a of the measurement pattern 41 captured by the camera 11 and the measurement pattern 41 by the laser scanner 12 via the input / output interface device 54. And the scan line 12a which is the scan result of. Then, the display control unit 51a of the computer 50 superimposes the camera image 11a of the measurement pattern 41 and the scan line 12a and displays them on the display device 60.
  • the measurement object 40 has a caster 43.
  • the caster 43 is a moving mechanism.
  • the measurement object 40 can move on the floor 82 by a caster 43. The operator can move the measurement object 40 to bring the measurement pattern 41 closer to the test apparatus 30 or move the measurement pattern 41 away from the test apparatus 30.
  • the operator changes the roll angle ⁇ , pitch angle ⁇ , and yaw angle ⁇ of the base 10 to which the camera 11 and the laser scanner 12 are fixed, and the scan line 12a of the laser scanner 12, the camera image 11a, and the like.
  • the calibration parameters can be determined by looking at the screen of the display device 60 so that they are correctly superimposed.
  • the calibration parameter is information indicating a relative attitude difference between the camera 11 and the laser scanner 12 that is fixed to the base 10. Specifically, there are a roll angle difference ⁇ , a pitch angle difference ⁇ , and a yaw angle difference ⁇ between the attachment posture of the camera 11 and the attachment posture of the laser scanner 12.
  • the roll angle difference ⁇ , the pitch angle difference ⁇ , and the yaw angle difference ⁇ may be simply referred to as ⁇ , ⁇ , and ⁇ . Details of ⁇ , ⁇ , and ⁇ will be described later.
  • ⁇ , ⁇ , and ⁇ are all posture difference information described later.
  • the posture difference information is information representing a relative posture difference between the camera 11 and the laser scanner 12.
  • ⁇ , ⁇ , and ⁇ which are calibration parameters, are obtained using the measurement object 40 and the test apparatus 30.
  • the test apparatus 30 on which the camera 11 and the laser scanner 12 are mounted does not move. Therefore, unlike Patent Document 1, it is not necessary to run a vehicle on which the camera 11 and the laser scanner 12 are mounted, so that the calibration parameter can be easily determined.
  • the superimposed display system 100 includes a test apparatus 30, a computer 50, a display apparatus 60, and a measurement object 40.
  • the test apparatus 30 includes a base 10 and a posture change mechanism 20.
  • a camera 11 and a laser scanner 12 are fixed to the base 10.
  • the base 10, the camera 11, and the laser scanner 12 can be regarded as one rigid body as a whole.
  • the base 10 to which the base 10 and the camera 11 are fixed does not move with respect to the floor 82.
  • the posture changing mechanism 20 has a base 10 to which the camera 11 and the laser scanner 12 are fixed fixed.
  • the posture change mechanism 20 can change the roll angle ⁇ , pitch angle ⁇ , and yaw angle ⁇ of the base 10.
  • the camera 11 is connected to the computer 50.
  • the camera 11 outputs the camera image 11a of the measurement pattern 41 to the computer 50.
  • the laser scanner 12 is connected to the computer 50.
  • the laser scanner 12 outputs a scan line 12 a that is a scan result of the measurement pattern 41 to the computer 50.
  • the base 10, the camera 11, and the laser scanner 12, which can be regarded as one rigid body as a whole, are arranged in the test apparatus 30 in a state where they are actually mounted on the MMS measurement vehicle. That is, the base 10, the camera 11, and the laser scanner 12 surrounded by the alternate long and short dash line 81 are actually mounted on the MMS measurement vehicle.
  • the base 10 is supported by the posture change mechanism 20.
  • the posture change mechanism 20 includes a roll shaft 21, a pitch shaft 22, a yaw shaft 23, and a support member 24.
  • a roll axis 21, a pitch axis 22, and a yaw axis 23 are set by the posture change mechanism 20.
  • the roll axis 21, the pitch axis 22, and the yaw axis 23 may be referred to as an X axis, a Y axis, and a Z axis, respectively.
  • the direction of the laser beam emitted from the laser scanner 12 is the normal direction of the arrangement surface 42 on which the measurement pattern 41 of the measurement object 40 is arranged.
  • 4 to 6 are views showing that the laser scanner 12 rotates.
  • FIG. 4 shows a state in which the laser scanner 12 is zero degrees in the XY plane.
  • FIG. 5 shows a state in which the laser scanner 12 is rotated clockwise from the zero degree state in the XY plane.
  • FIG. 6 shows a state in which the laser scanner 12 is rotated counterclockwise from the zero degree state in the XY plane.
  • the laser scanner 12 rotates in the range of zero degrees to +90 degrees and ⁇ 90 degrees in the XY plane, and emits laser light. That is, the laser scanner 12 can rotate 90 degrees to the right and 90 degrees to the left around the Z-axis, assuming that the state of FIG. 4 is zero degrees.
  • the normal direction of the arrangement surface 42 is a zero-degree emission direction of the laser light, and the zero-degree emission direction of the laser light is set as the X-axis direction.
  • the normal direction of the plane including the laser beams of +90 degrees and ⁇ 90 degrees is the Z-axis direction.
  • the normal direction of the XZ plane is the Y-axis direction.
  • the roll axis 21, the pitch axis 22, and the yaw axis 23, which are the X axis, the Y axis, and the X axis, are set.
  • the base 10 to which the camera 11 and the laser scanner 12 are fixed can rotate around a roll axis 21 (X axis), a pitch axis 22 (Y axis), and a yaw axis 23 (Z axis).
  • the angle around the roll axis 21 is the roll angle ⁇
  • the angle around the pitch axis 22 is the pitch angle ⁇
  • the angle around the yaw axis 23 is the yaw angle ⁇ .
  • the roll shaft 21, the pitch shaft 22, and the yaw shaft 23 are supported by a support member 24. That is, the base 10 to which the camera 11 and the laser scanner 12 are fixed is supported by the support member 24 via the roll shaft 21, the pitch shaft 22, and the yaw shaft 23.
  • the measurement object 40 has a measurement pattern 41.
  • the measurement pattern 41 is a pattern photographed by the camera 11 and is a pattern in which a difference in reflection luminance appears by scanning with the laser scanner 12.
  • an example of the measurement pattern 41 is a check pattern.
  • black and white have the same shape, but black and white may not have the same shape.
  • the measurement pattern 41 is not limited to the check pattern as long as the difference in reflection luminance appears by scanning with the laser scanner 12.
  • the measurement object 40 has a caster 43. The measurement object 40 can move on the floor 82 by the caster 43. That is, the position of the measurement object 40 can be freely changed with respect to the test apparatus 30.
  • the computer 50 includes a processor 51, a main storage device 52, an auxiliary storage device 53, and an input / output interface device 54 as hardware.
  • the processor 51 is connected to other hardware via the signal line 55, and controls these other hardware.
  • the processor 51 is an IC (Integrated Circuit) that performs arithmetic processing.
  • Specific examples of the processor 51 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).
  • the main storage device 52 is a volatile storage device that can be read and written. Specific examples of the main storage device 52 are SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory).
  • the auxiliary storage device 53 is a non-volatile storage device that can be read and written.
  • the auxiliary storage device 53 stores a program for realizing the function of the computer 50 and other data.
  • the auxiliary storage device 53 is, as a specific example, a magnetic disk device (Hard Disk Drive).
  • the auxiliary storage device 53 may be a storage device using a portable storage medium such as an optical disc, a compact disc, a Blu-ray (registered trademark) disc, or a DVD (Digital Versatile Disk).
  • the input / output interface device 54 is an interface device for the processor 5 to communicate with the camera 11, the laser scanner 12, the display device 60, and the input device 70.
  • the computer 50 includes a display control unit 51a and a correction unit 51b as functional elements.
  • the functions of the display control unit 51a and the correction unit 51b are realized by a superimposed display program.
  • the auxiliary storage device 53 stores a superimposed display program that realizes the functions of the display control unit 51a and the correction unit 51b.
  • the superimposed display program is read and executed by the processor 51. Thereby, the function of the display control part 51a and the correction
  • the superposition display program causes the computer 50 to execute each process, each procedure, or each process in which “part” of each part of the display control unit 51a and the correction unit 51b is read as “process”, “procedure”, or “process”.
  • the superimposed display method is a method performed by the computer 50 executing a superimposed display program.
  • the superimposed display program may be provided by being stored in a computer-readable recording medium or may be provided as a program product.
  • the computer 50 may include a plurality of processors that replace the processor 51.
  • the plurality of processors share the execution of the functions of the display control unit 51a and the correction unit 51b.
  • Each processor is an IC that performs arithmetic processing in the same manner as the processor 51.
  • the processor 51 and the plurality of processors are collectively referred to as a processing circuit.
  • FIG. 7 shows an example in which there is a yaw angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12.
  • FIG. 7 shows a case where the camera image 11a and the scan line 12a are displayed on the display device 60 in a superimposed manner by the display control unit 51a. An operation in which the camera video 11a and the scan line 12a are displayed on the display device 60 in a superimposed manner will be described.
  • the operator photographs the measurement pattern 41 with the camera 11 of the test apparatus 30 and scans the measurement pattern 41 with the laser scanner 12.
  • the camera image 11a and the scan line 12a are transmitted to the display control unit 51a via the input / output interface device 54.
  • the display control unit 51a superimposes and displays the camera video 11a and the scan line 12a on the display device 60 via the input / output interface device 54.
  • the scan line 12a is displayed as a line, but is actually a point group that is a set of points. This point group is called a laser point group.
  • the laser point group also displays the reflected luminance.
  • (6) When the postures in the yaw direction of the camera 11 and the laser scanner 12 match, the scan line 12a is correctly superimposed on the camera image 11a. That is, “correctly superimposed” means that the reflected luminances of white and black overlap with the camera image 11a.
  • a scan line 12a-1 indicates a scan line correctly superimposed on the camera image 11a.
  • the black and white pattern of the scan line 12a-2 shows a state shifted to the right with respect to the black and white pattern of the scan line 12a-1.
  • the monochrome pattern of the scan line 12a-3 is shifted to the left with respect to the monochrome pattern of the scan line 12a-1.
  • the scan line 12a-2 and the scan line 12a-3 are not correctly superimposed on the camera image 11a.
  • the yaw angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12 can be corrected by correcting the shift amount in the left direction or the right direction. In FIG. 7, a rightward shift and a leftward shift are described together, but in actuality, either a rightward shift or a leftward shift appears.
  • a leftward shift will be described as an example.
  • the black and white pattern of the scan line 12a-3 is shifted by a length L1 in the left direction from the normal black and white pattern of the scan line 12a-1.
  • the distance from the reference point of the test apparatus 30 to the measurement pattern 41 is a distance L0. Therefore, the yaw angle difference ⁇ between the camera 11 and the laser scanner 12 can be calculated by the following equation 1.
  • ⁇ tan ⁇ 1 (L1 / L0) (Formula 1)
  • the operator calculates the yaw angle difference ⁇ by reading the length L1 from the camera image 11a and the scan line 12a-3 displayed on the display device 60.
  • the yaw angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12 is obtained.
  • the operator inputs the yaw angle difference ⁇ , which is posture difference information, to the computer 50 via the input device 70 such as the keyboard 71 or the mouse 72. That is, the yaw angle difference ⁇ , which is attitude difference information, is input to the correction unit 51 b of the computer 50 via the input device 70.
  • the correction unit 51b of the computer 50 corrects the position of the scan line 12a displayed on the display device 60 using the yaw angle difference ⁇ that is the posture difference information.
  • posture information of the camera 11 and the laser scanner 12 with respect to the base 10 is set in the superimposed display program that realizes the correction unit 51b.
  • the correction unit 51b corrects the posture information when the yaw angle difference ⁇ is acquired. Correcting the posture information has an effect of correcting the yaw angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12 by actually using a tool.
  • the display control unit 51a superimposes and displays the scan line 12a whose position is corrected by the correction unit 51b on the camera image 11a of the measurement pattern 41. By this superimposed display, the operator can check the correction state of the yaw angle difference ⁇ . (11) The operator used the value obtained by Equation 1 as the yaw angle difference ⁇ .
  • the operator may input an equivalent value corresponding to the yaw angle difference ⁇ into the calculator 50. Specifically, the operator may input a corresponding value of a different value to the computer 50 until a normal scan line is obtained, and obtain a scan line in which the yaw angle ⁇ is calibrated.
  • the equivalent value is posture difference information.
  • the operator rotates the base 10, but when using an equivalent value, the operator does not necessarily need to rotate the base 10.
  • the posture changing mechanism 20 may be omitted. Therefore, in FIG. 1, the base 10 to which the camera 11 and the laser scanner 12 are fixed may be disposed on the floor 32 or may be disposed on the support member 24.
  • FIG. 8 shows an example in which there is a pitch angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12.
  • FIG. 8 shows a case where the measurement pattern 41 and the scan line 12a-4 or the scan line 12a-5 are superimposed and displayed on the display device 60 by the display control unit 51a.
  • the scan line is expressed as a scan line 12a.
  • An operation in which the measurement pattern 41 and the scan line 12a are superimposed on each other and displayed on the display device 60 will be described.
  • the operator photographs the measurement pattern 41 with the camera 11 of the test apparatus 30 and scans the measurement pattern 41 with the laser scanner 12.
  • the camera image 11a and the scan line 12a are transmitted to the display control unit 51a via the input / output interface device 54.
  • the display control unit 51a superimposes and displays the camera video 11a and the scan line 12a on the display device 60 via the input / output interface device 54.
  • the scan line 12a is correctly superimposed on the measurement pattern 41. That is, “correctly superimposed” is as follows. As described above, FIG. 8 shows a case where the posture in the pitch direction between the camera 11 and the laser scanner 12 is shifted.
  • the scan line 12a moves on the screen of the display device 60 as the base 10 rotates.
  • FIG. 1 it is assumed that the operator has rotated the base 10 around the left of the pitch axis 22 (Y axis). In that case, in FIG. 8, the scan line 12a moves from the scan line 12a-4 to the scan line 12a-5.
  • the posture of the camera 11 and the laser scanner 12 is shifted in the pitch direction, when moving from the scan line 12a-4 to the scan line 12a-5, the black and white of the scan line is different from the black and white of the camera image 11a. The reverse timing is shifted.
  • the pitch angle difference ⁇ between the attitude of the camera 11 and the attitude of the laser scanner 12 can be corrected by correcting the downward shift amount.
  • the downward shift amount is L3-L2. Therefore, as in the case of FIG.
  • the pitch angle difference ⁇ between the camera 11 and the laser scanner 12 can be calculated by the following equation 2.
  • ⁇ tan ⁇ 1 ((L3 ⁇ L2) / L0) (Formula 2)
  • the operator reads the length “L3-L2” from the camera image 11a displayed on the display device 60, the scan line 12a-4, and the scan line 12a-5, thereby calculating the pitch angle difference ⁇ .
  • the pitch angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12 is obtained.
  • the operator inputs the pitch angle difference ⁇ , which is posture difference information, to the computer 50 via the input device 70 such as the keyboard 71 or the mouse 72.
  • the correction unit 51 b of the computer 50 receives the pitch angle difference ⁇ that is posture difference information via the input device 70.
  • the correction unit 51b of the computer 50 corrects the position of the scan line 12a displayed on the display device 60 as in the case of the yaw angle difference ⁇ , using the pitch angle difference ⁇ that is the posture difference information.
  • the correction unit 51b acquires the pitch angle difference ⁇
  • the correction unit 51b corrects the posture information described in the description of FIG. Correcting the posture information has an effect of correcting the pitch angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12 by actually using a tool.
  • the display control unit 51a superimposes and displays the scan line 12a whose position is corrected by the correction unit 51b on the camera image 11a of the measurement pattern 41. By this superimposed display, the operator can check the correction state of the pitch angle difference ⁇ .
  • the operator used the value obtained by Equation 2 as the pitch angle difference ⁇ . However, instead of using the value obtained by Equation 2 as the pitch angle difference ⁇ , the operator may input an equivalent value corresponding to the pitch angle difference ⁇ into the calculator 50.
  • the equivalent value is posture difference information. Specifically, the operator may input equivalent values of different values to the computer 50 until a normal scan line is obtained, and obtain a scan line with the calibrated pitch angle ⁇ .
  • the equivalent value is posture difference information.
  • the operator rotates the base 10, but when using an equivalent value, the operator does not necessarily have to rotate the base 10.
  • the posture changing mechanism 20 may be omitted. Therefore, in FIG. 1, the base 10 to which the camera 11 and the laser scanner 12 are fixed may be disposed on the floor 32 or may be disposed on the support member 24.
  • FIG. 9 shows a case where there is no roll angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12.
  • FIG. 9 shows a state where the roll angle ⁇ is correctly calibrated.
  • the scan line 12a-6 is a scan line in a state where the roll angle ⁇ is correctly calibrated. When the roll angle ⁇ is correctly calibrated, the black and white of the scan line 12 a-6 matches the black and white of the measurement pattern 41.
  • FIG. 10 shows a case where there is a roll angle difference ⁇ between the posture of the camera 11 and the posture of the laser scanner 12.
  • FIG. 10 shows a state where the roll angle ⁇ is not correctly calibrated.
  • the scan line 12a-7 is a scan line in a state where the roll angle ⁇ is not correctly calibrated. When the roll angle ⁇ is not correctly calibrated, the black and white of the scan line 12a-7 does not match the black and white of the measurement pattern 41.
  • the calibration of the roll angle ⁇ will be described assuming that the scan line 12a-7 is corrected to the scan line 12a-6.
  • the operator inputs an equivalent value corresponding to the roll angle difference ⁇ to the computer 50 via the input device 70.
  • An equivalent value of the roll angle difference ⁇ which is posture difference information, is input to the correction unit 51 b of the computer 50 via the input device 70.
  • the correction unit 51b corrects the position of the scan line 12a-7 displayed on the display device 60 using the input equivalent value as posture difference information.
  • the correction unit 51b corrects the roll angle ⁇ , which is posture information, in the same manner as when the yaw angle difference ⁇ is acquired.
  • the display control unit 51a superimposes and displays the scan line whose roll angle ⁇ is corrected by the correction unit 51b on the camera image 11a of the measurement pattern 41. By this superimposed display, the operator can check the correction state of the roll angle ⁇ .
  • the operator inputs an equivalent value obtained by changing the value into the computer 50 until the scan line 12a-6 is obtained.
  • the posture changing mechanism 20 may be omitted. Therefore, in FIG. 1, the base 10 to which the camera 11 and the laser scanner 12 are fixed may be disposed on the floor 32 or may be disposed on the support member 24.
  • a roll angle difference ⁇ , a pitch angle difference ⁇ , and a yaw angle difference ⁇ appear together.
  • the operator rotates the base 10 around the roll axis 21 (X axis), the pitch axis 22 (Y axis), or the yaw axis 23 (Z axis). Then, the operator obtains the state in which the yaw angle difference ⁇ appears as shown in FIG. 7 and the state in which the pitch angle difference ⁇ appears as shown in FIG. 8, and sets the yaw angle difference ⁇ , the pitch angle difference ⁇ , etc. Check individually.
  • the method for obtaining ⁇ , ⁇ , and ⁇ has been described.
  • the posture and position of the camera 11 and the laser scanner 12 are similar to the general calibration parameter estimation method. Can be used to estimate calibration parameters including
  • the calibration parameter estimation principle including the posture and position of the camera 11 and the laser scanner 12 is the same as the calibration for running the MMS measurement vehicle.
  • the positions of the camera 11 and the laser scanner 12 are estimated near the measurement pattern 41, and the posture is estimated far from the measurement pattern 41.
  • the test apparatus 30 captures the measurement pattern 41 from the front in a stationary state.
  • it is effective to determine the posture of the camera 11 and the laser scanner 12 or the position of the camera 11 and the laser scanner 12 while the camera 11 and the laser scanner 12 are moved. That is, it is preferable that the operator moves the measurement object 40 or that the camera 11 and the laser scanner 12 rotate around the roll axis 21, the pitch axis 22, or the yaw axis 23. Even if the camera image 11a is finally taken from any angle and from any position, the calibration is completed when the scan line 12a is correctly superimposed on the camera image 11a.
  • the distance between the camera 11 and the laser scanner 12 and the measurement pattern 41 may be set longer than usual.
  • the reason for this is that the influence of the posture appears larger as the distance increases. For example, for an error of 0.1 deg, it is 1.7 mm for 1 m but 17.4 cm for 100 m.
  • the test apparatus 30 performs the estimation at a distance close to the measurement pattern 41. This is because the influence of the position does not depend on the distance between the camera 11 and the laser scanner 12 and the measurement pattern 41, so that a position error appears more in the vicinity where the influence of the posture is small. These are the same as the running calibration method.

Abstract

This superimposed display system (100) comprises a base (10), an orientation changing mechanism (20), an object (40) of measurement, and a computer (50). The base (10) has a camera (11) and a laser scanner (12) fixed thereto. The base (10), which has the camera (11) and laser scanner (12) fixed thereto, is fixed to the orientation changing mechanism (20). The orientation changing mechanism (20) changes the roll, pitch, and yaw of the base (10). The object (40) of measurement has a measurement pattern (41). The computer (50) acquires a camera image (11a) of the measurement pattern (41) photographed by the camera (11) and a scan line (12a) on the measurement pattern (41) produced by the laser scanner (12) and displays the camera image (11a) and scan line (12a) on the display device (60) so as to be superimposed.

Description

重畳表示システムSuperimposition display system
　この発明は、モービルマッピングシステム（以下、ＭＭＳと記す）に使用される、カメラ及びレーザスキャナのキャリブレーションのための、重畳表示システムに関する。 The present invention relates to a superimposed display system for calibration of a camera and a laser scanner used in a mobile mapping system (hereinafter referred to as MMS).
　ＭＭＳでは、車両位置を推定する装置、カメラ及びレーザスキャナが計測車両に搭載される。ＭＭＳ計測車両の走行によって、道路及び道路周辺の形状が計測されることは広く行われている。ＭＭＳ計測車両では、経年変化及び小さな衝突などにより、カメラ、レーザスキャナの取り付け位置または取り付け姿勢が変化し得る。カメラ、レーザスキャナの取り付け位置または取り付け姿勢が変化すると、計測の精度が劣下する。
　そこで、計測の精度劣化の防止のために、精度の確保のためのキャリブレーションが実施される（例えば特許文献１）。 In MMS, a device for estimating a vehicle position, a camera, and a laser scanner are mounted on a measurement vehicle. It is widely performed that the shape of the road and the surroundings of the road is measured by the traveling of the MMS measuring vehicle. In the MMS measurement vehicle, the mounting position or mounting posture of the camera and laser scanner may change due to aging and small collisions. When the mounting position or mounting posture of the camera or laser scanner changes, the measurement accuracy deteriorates.
Therefore, in order to prevent measurement accuracy deterioration, calibration for ensuring accuracy is performed (for example, Patent Document 1).
特開２０１０－１７５４２３号公報JP 2010-175423 A
　しかし、従来では、キャリブレーションの際には、カメラ及びレーザスキャナが搭載された車両を走行させていた（例えば、特許文献１）。そのため、キャリブレーションの負担が大きかった。 However, conventionally, at the time of calibration, a vehicle equipped with a camera and a laser scanner is run (for example, Patent Document 1). Therefore, the burden of calibration was great.
　そこで、本発明は、カメラ及びレーザスキャナを車両に搭載せずにキャリブレーションを行うシステムの提供を目的とする。 Therefore, an object of the present invention is to provide a system for performing calibration without mounting a camera and a laser scanner on a vehicle.
　この発明の重畳表示システムは、
　カメラとレーザスキャナとが固定されたベースと、
　前記カメラと前記レーザスキャナとが固定された前記ベースが固定され、前記ベースのロール角度、ピッチ角度及びヨー角度を変化させる姿勢変化機構と、
　前記カメラに撮影される模様であり、前記レーザスキャナによるスキャンによって反射輝度の相違が現れる模様である計測模様を有する計測対象物と、
　前記カメラによって撮影された前記計測模様のカメラ映像と、前記レーザスキャナによる前記計測模様のスキャン結果であるスキャンラインとを取得し、前記カメラ映像と前記スキャンラインとを重畳して表示装置に表示する計算機と
を備える。 The superimposed display system of the present invention includes:
A base to which a camera and a laser scanner are fixed;
An attitude changing mechanism that fixes the base to which the camera and the laser scanner are fixed, and changes a roll angle, a pitch angle, and a yaw angle of the base;
A measurement object having a measurement pattern which is a pattern photographed by the camera and a pattern in which a difference in reflected luminance appears by scanning with the laser scanner;
A camera image of the measurement pattern photographed by the camera and a scan line that is a scan result of the measurement pattern by the laser scanner are acquired, and the camera image and the scan line are superimposed and displayed on a display device. A computer.
　本発明によれば、カメラ及びレーザスキャナを車両に搭載せずにキャリブレーションを行うシステムを提供できる。 According to the present invention, it is possible to provide a system for performing calibration without mounting a camera and a laser scanner on a vehicle.
実施の形態１の図で、重畳表示システムの概要を示す図。The figure of Embodiment 1, The figure which shows the outline | summary of a superimposition display system. 実施の形態１の図で、試験装置及び計測対象部物の斜視図。In the figure of Embodiment 1, it is a perspective view of a test device and a measurement object part. 実施の形態１の図で、計算機のハードウェア構成図。FIG. 3 is a diagram illustrating the hardware configuration of the computer according to the first embodiment. 実施の形態１の図で、レーザスキャナの回転を示す図。FIG. 3 is a diagram illustrating the rotation of the laser scanner in the first embodiment. 実施の形態１の図で、レーザスキャナの回転を示す別の図。FIG. 4 is another diagram showing the rotation of the laser scanner in the first embodiment. 実施の形態１の図で、レーザスキャナの回転を示すさらに別の図。FIG. 5 is another view showing the rotation of the laser scanner in the first embodiment. 実施の形態１の図で、カメラとレーザスキャナとのヨー角度差を示す図。FIG. 3 is a diagram of Embodiment 1 and shows a yaw angle difference between a camera and a laser scanner. 実施の形態１の図で、カメラとレーザスキャナとのピッチ角度差を示す図。FIG. 3 is a diagram of the first embodiment and shows a pitch angle difference between the camera and the laser scanner. 実施の形態１の図で、カメラとレーザスキャナとのロール角度差を示す図。FIG. 4 is a diagram of Embodiment 1, showing a roll angle difference between a camera and a laser scanner. 実施の形態１の図で、カメラとレーザスキャナとのロール角度差を示す別の図。FIG. 3 is another diagram showing the roll angle difference between the camera and the laser scanner in the first embodiment.
　以下、本発明の実施の形態について、図を用いて説明する。なお、各図中、同一または相当する部分には、同一符号を付している。実施の形態の説明において、同一または相当する部分については、説明を適宜省略または簡略化する。 Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals. In the description of the embodiments, the description of the same or corresponding parts will be omitted or simplified as appropriate.
実施の形態１．
　図１、図２、図３を参照して実施の形態１の重畳表示システム１００を説明する。
図１は、重畳表示システム１００の概要を示している。
図２は、試験装置３０及び計測対象物４０の斜視図である。計測対象物４０は計測模様４１を有する。図２では、後述のロール軸２１、ピッチ軸２２及びヨー軸２３は見えない状態である。図１及び図２では、試験装置３０は床８２に配置されている。
図３は、計算機５０のハードウェア構成を示す。 Embodiment 1 FIG.
A superimposed display system 100 according to the first embodiment will be described with reference to FIGS. 1, 2, and 3.
FIG. 1 shows an overview of the superimposed display system 100.
FIG. 2 is a perspective view of the test apparatus 30 and the measurement object 40. The measurement object 40 has a measurement pattern 41. In FIG. 2, a roll shaft 21, a pitch shaft 22, and a yaw shaft 23, which will be described later, are not visible. 1 and 2, the test apparatus 30 is disposed on the floor 82.
FIG. 3 shows a hardware configuration of the computer 50.
　図１に示す重畳表示システム１００では、計算機５０の表示制御部５１ａは、入出力インタフェース装置５４を介して、カメラ１１によって撮影された計測模様４１のカメラ映像１１ａと、レーザスキャナ１２による計測模様４１のスキャン結果であるスキャンライン１２ａとを取得する。そして、計算機５０の表示制御部５１ａは、計測模様４１のカメラ映像１１ａとスキャンライン１２ａとを重畳して、表示装置６０に表示する。
　計測対象物４０はキャスタ４３を有する。キャスタ４３は移動機構である。計測対象物４０はキャスタ４３によって床８２の上を移動できる。作業者は計測対象物４０を動かすことで、計測模様４１を試験装置３０に近づけ、または計測模様４１を試験装置３０から遠ざけることができる。また、作業者は、カメラ１１及びレーザスキャナ１２が固定されているベース１０の、ロール角度φ、ピッチ角度θ及びヨー角度ψを変化させて、レーザスキャナ１２のスキャンライン１２ａと、カメラ映像１１ａとが、正しく重畳するように、表示装置６０の画面をみて、キャリブレーションパラメータを決めることができる。
　キャリブレーションパラメータとは、ベース１０に固定されている、カメラ１１とレーザスキャナ１２との相対的な姿勢の差を示す情報である。
　具体的には、カメラ１１の取り付け姿勢とレーザスキャナ１２の取り付け姿勢との間の、ロール角度差△φ、ピッチ角度差△θ及びヨー角度差△ψである。以下では、ロール角度差△φ、ピッチ角度差△θ及びヨー角度差△ψを、単に、△φ、△θ、△ψと記載する場合がある。△φ、△θ、△ψの詳細は後述する。△φ、△θ、△ψは、いずれも後述する姿勢相違情報である。姿勢相違情報とはカメラ１１とレーザスキャナ１２との相対的な姿勢の相違を表す情報である。 In the superimposed display system 100 illustrated in FIG. 1, the display control unit 51 a of the computer 50 includes the camera image 11 a of the measurement pattern 41 captured by the camera 11 and the measurement pattern 41 by the laser scanner 12 via the input / output interface device 54. And the scan line 12a which is the scan result of. Then, the display control unit 51a of the computer 50 superimposes the camera image 11a of the measurement pattern 41 and the scan line 12a and displays them on the display device 60.
The measurement object 40 has a caster 43. The caster 43 is a moving mechanism. The measurement object 40 can move on the floor 82 by a caster 43. The operator can move the measurement object 40 to bring the measurement pattern 41 closer to the test apparatus 30 or move the measurement pattern 41 away from the test apparatus 30. Further, the operator changes the roll angle φ, pitch angle θ, and yaw angle ψ of the base 10 to which the camera 11 and the laser scanner 12 are fixed, and the scan line 12a of the laser scanner 12, the camera image 11a, and the like. However, the calibration parameters can be determined by looking at the screen of the display device 60 so that they are correctly superimposed.
The calibration parameter is information indicating a relative attitude difference between the camera 11 and the laser scanner 12 that is fixed to the base 10.
Specifically, there are a roll angle difference Δφ, a pitch angle difference Δθ, and a yaw angle difference Δψ between the attachment posture of the camera 11 and the attachment posture of the laser scanner 12. Hereinafter, the roll angle difference Δφ, the pitch angle difference Δθ, and the yaw angle difference Δψ may be simply referred to as Δφ, Δθ, and Δψ. Details of Δφ, Δθ, and Δψ will be described later. Δφ, Δθ, and Δψ are all posture difference information described later. The posture difference information is information representing a relative posture difference between the camera 11 and the laser scanner 12.
　重畳表示システム１００によれば、計測対象物４０と試験装置３０とを用いて、キャリブレーションパラメータである、△φ、△θ及び△ψが得られる。重畳表示システム１００では、カメラ１１及びレーザスキャナ１２が搭載された試験装置３０は、移動しない。
　よって、特許文献１のように、カメラ１１及びレーザスキャナ１２が搭載された車両を走行させる必要がないので、簡易にキャリブレーションパラメータが決定できる。 According to the superimposed display system 100, Δφ, Δθ, and Δψ, which are calibration parameters, are obtained using the measurement object 40 and the test apparatus 30. In the superimposed display system 100, the test apparatus 30 on which the camera 11 and the laser scanner 12 are mounted does not move.
Therefore, unlike Patent Document 1, it is not necessary to run a vehicle on which the camera 11 and the laser scanner 12 are mounted, so that the calibration parameter can be easily determined.
＊＊＊構成の説明＊＊＊
　以下、重畳表示システム１００を詳しく説明する。図１に示すように、重畳表示システム１００は、試験装置３０、計算機５０、表示装置６０及び計測対象物４０を備えている。 *** Explanation of configuration ***
Hereinafter, the superimposed display system 100 will be described in detail. As shown in FIG. 1, the superimposed display system 100 includes a test apparatus 30, a computer 50, a display apparatus 60, and a measurement object 40.
　試験装置３０は、ベース１０と姿勢変化機構２０とを備えている。ベース１０には、カメラ１１とレーザスキャナ１２とが固定されている。ベース１０と、カメラ１１及びレーザスキャナ１２は、全体でひとつの剛体とみなせる。ベース１０とカメラ１１とが固定されたベース１０は床８２に対して移動しない。
　姿勢変化機構２０は、カメラ１１とレーザスキャナ１２とが固定されたベース１０が固定されている。姿勢変化機構２０は、ベース１０のロール角度φ、ピッチ角度θ及びヨー角度ψを変化させることができる。 The test apparatus 30 includes a base 10 and a posture change mechanism 20. A camera 11 and a laser scanner 12 are fixed to the base 10. The base 10, the camera 11, and the laser scanner 12 can be regarded as one rigid body as a whole. The base 10 to which the base 10 and the camera 11 are fixed does not move with respect to the floor 82.
The posture changing mechanism 20 has a base 10 to which the camera 11 and the laser scanner 12 are fixed fixed. The posture change mechanism 20 can change the roll angle φ, pitch angle θ, and yaw angle ψ of the base 10.
　カメラ１１は、計算機５０と接続されている。カメラ１１は、計測模様４１のカメラ映像１１ａを計算機５０へ出力する。レーザスキャナ１２は、計算機５０と接続されている。レーザスキャナ１２は、計測模様４１のスキャン結果であるスキャンライン１２ａを、計算機５０へ出力する。 The camera 11 is connected to the computer 50. The camera 11 outputs the camera image 11a of the measurement pattern 41 to the computer 50. The laser scanner 12 is connected to the computer 50. The laser scanner 12 outputs a scan line 12 a that is a scan result of the measurement pattern 41 to the computer 50.
　全体でひとつの剛体とみなせる、ベース１０、カメラ１１及びレーザスキャナ１２は、ＭＭＳ計測車両に実際に搭載される状態で、試験装置３０に配置される。つまり、一点鎖線８１で囲まれたベース１０、カメラ１１及びレーザスキャナ１２は、ＭＭＳ計測車両に実際に搭載される状態である。 The base 10, the camera 11, and the laser scanner 12, which can be regarded as one rigid body as a whole, are arranged in the test apparatus 30 in a state where they are actually mounted on the MMS measurement vehicle. That is, the base 10, the camera 11, and the laser scanner 12 surrounded by the alternate long and short dash line 81 are actually mounted on the MMS measurement vehicle.
　ベース１０は、姿勢変化機構２０に支持されている。姿勢変化機構２０は、ロール軸２１、ピッチ軸２２、ヨー軸２３及び支持部材２４を有する。 The base 10 is supported by the posture change mechanism 20. The posture change mechanism 20 includes a roll shaft 21, a pitch shaft 22, a yaw shaft 23, and a support member 24.
　試験装置３０には、姿勢変化機構２０によって、ロール軸２１、ピッチ軸２２及びヨー軸２３が設定されている。ロール軸２１、ピッチ軸２２及びヨー軸２３は、それぞれ、Ｘ軸、Ｙ軸、Ｚ軸と表記する場合がある。ロール軸２１は、レーザスキャナ１２の出射するレーザ光の方向が、計測対象物４０の計測模様４１が配置されている配置面４２の法線方向である。図４から図６はレーザスキャナ１２が回転することを示す図である。
図４は、レーザスキャナ１２が、ＸＹ平面内でゼロ度の状態を示す。
図５は、レーザスキャナ１２が、ＸＹ平面内でゼロ度の状態から右回りに回転した状態を示す。
図６は、レーザスキャナ１２が、ＸＹ平面内でゼロ度の状態から左回りに回転した状態を示す。図４に示すように、レーザスキャナ１２は、ＸＹ平面内でゼロ度から＋９０度、－９０度の範囲で回転し、レーザ光を出射する。つまり、レーザスキャナ１２は、図４の状態をゼロ度とすると、Ｚ軸まわりに右に９０度、左に９０度回転可能である。図４では、配置面４２の法線方向がレーザ光のゼロ度の出射方向であり、このレーザ光のゼロ度の出射方向がＸ軸方向として設定されている。＋９０度及び－９０度のレーザ光を含む平面の法線方向がＺ軸方向である。ＸＺ平面の法線方向がＹ軸方向である。以上のように、Ｘ軸、Ｙ軸、Ｘ軸である、ロール軸２１、ピッチ軸２２、ヨー軸２３が設定されている。 In the test apparatus 30, a roll axis 21, a pitch axis 22, and a yaw axis 23 are set by the posture change mechanism 20. The roll axis 21, the pitch axis 22, and the yaw axis 23 may be referred to as an X axis, a Y axis, and a Z axis, respectively. In the roll axis 21, the direction of the laser beam emitted from the laser scanner 12 is the normal direction of the arrangement surface 42 on which the measurement pattern 41 of the measurement object 40 is arranged. 4 to 6 are views showing that the laser scanner 12 rotates.
FIG. 4 shows a state in which the laser scanner 12 is zero degrees in the XY plane.
FIG. 5 shows a state in which the laser scanner 12 is rotated clockwise from the zero degree state in the XY plane.
FIG. 6 shows a state in which the laser scanner 12 is rotated counterclockwise from the zero degree state in the XY plane. As shown in FIG. 4, the laser scanner 12 rotates in the range of zero degrees to +90 degrees and −90 degrees in the XY plane, and emits laser light. That is, the laser scanner 12 can rotate 90 degrees to the right and 90 degrees to the left around the Z-axis, assuming that the state of FIG. 4 is zero degrees. In FIG. 4, the normal direction of the arrangement surface 42 is a zero-degree emission direction of the laser light, and the zero-degree emission direction of the laser light is set as the X-axis direction. The normal direction of the plane including the laser beams of +90 degrees and −90 degrees is the Z-axis direction. The normal direction of the XZ plane is the Y-axis direction. As described above, the roll axis 21, the pitch axis 22, and the yaw axis 23, which are the X axis, the Y axis, and the X axis, are set.
　カメラ１１及びレーザスキャナ１２が固定されているベース１０は、ロール軸２１（Ｘ軸）、ピッチ軸２２（Ｙ軸）及びヨー軸２３（Ｚ軸）の周りに回転できる。ロール軸２１回りの角度がロール角度φ、ピッチ軸２２回りの角度がピッチ角度θ、ヨー軸２３回りの角度がヨー角度ψである。 The base 10 to which the camera 11 and the laser scanner 12 are fixed can rotate around a roll axis 21 (X axis), a pitch axis 22 (Y axis), and a yaw axis 23 (Z axis). The angle around the roll axis 21 is the roll angle φ, the angle around the pitch axis 22 is the pitch angle θ, and the angle around the yaw axis 23 is the yaw angle ψ.
　ロール軸２１、ピッチ軸２２及びヨー軸２３は、支持部材２４によって支持される。つまり、カメラ１１及びレーザスキャナ１２が固定されているベース１０は、ロール軸２１、ピッチ軸２２及びヨー軸２３を介して、支持部材２４に支持される。 The roll shaft 21, the pitch shaft 22, and the yaw shaft 23 are supported by a support member 24. That is, the base 10 to which the camera 11 and the laser scanner 12 are fixed is supported by the support member 24 via the roll shaft 21, the pitch shaft 22, and the yaw shaft 23.
　計測対象物４０は、計測模様４１を有する。計測模様４１は、カメラ１１に撮影される模様であり、かつ、レーザスキャナ１２によるスキャンによって反射輝度の相違が現れる模様である。実施の形態１では、計測模様４１の例はチェック模様である。実施の形態１のチェック模様では、黒と白は同じ形状であるが、黒と白とは同じ形状でなくても構わない。また、レーザスキャナ１２によるスキャンによって反射輝度の相違が現れる模様であれば、計測模様４１はチェック模様には限られない。
　計測対象物４０はキャスタ４３を有する。キャスタ４３によって、計測対象物４０は床８２の上を移動できる。つまり、試験装置３０に対して、計測対象物４０の位置は自由に変えることができる。 The measurement object 40 has a measurement pattern 41. The measurement pattern 41 is a pattern photographed by the camera 11 and is a pattern in which a difference in reflection luminance appears by scanning with the laser scanner 12. In the first embodiment, an example of the measurement pattern 41 is a check pattern. In the check pattern of the first embodiment, black and white have the same shape, but black and white may not have the same shape. In addition, the measurement pattern 41 is not limited to the check pattern as long as the difference in reflection luminance appears by scanning with the laser scanner 12.
The measurement object 40 has a caster 43. The measurement object 40 can move on the floor 82 by the caster 43. That is, the position of the measurement object 40 can be freely changed with respect to the test apparatus 30.
　計算機５０は、ハードウェアとして、プロセッサ５１、主記憶装置５２、補助記憶装置５３、入出力インタフェース装置５４を備える。プロセッサ５１は、信号線５５を介して他のハードウェアと接続され、これら他のハードウェアを制御する。 The computer 50 includes a processor 51, a main storage device 52, an auxiliary storage device 53, and an input / output interface device 54 as hardware. The processor 51 is connected to other hardware via the signal line 55, and controls these other hardware.
　プロセッサ５１は、演算処理を行うＩＣ（Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ）である。プロセッサ５１は、具体例としては、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、ＤＳＰ（Ｄｉｇｉｔａｌ　Ｓｉｇｎａｌ　Ｐｒｏｃｅｓｓｏｒ）、ＧＰＵ（Ｇｒａｐｈｉｃｓ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）である。 The processor 51 is an IC (Integrated Circuit) that performs arithmetic processing. Specific examples of the processor 51 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).
　主記憶装置５２は、読み書きが可能な揮発性の記憶装置である。主記憶装置５２の具体例としては、ＳＲＡＭ（Ｓｔａｔｉｃ　Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＤＲＡＭ（Ｄｙｎａｍｉｃ　Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）である。 The main storage device 52 is a volatile storage device that can be read and written. Specific examples of the main storage device 52 are SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory).
　補助記憶装置５３は、読み書きが可能な不揮発性の記憶装置である。補助記憶装置５３には、計算機５０の機能を実現するためのプログラム及び他のデータが記憶される。補助記憶装置５３は、具体例としては、磁気ディスク装置（Ｈａｒｄ　Ｄｉｓｋ　Ｄｒｉｖｅ）である。また、補助記憶装置５３は、光ディスク、コンパクトディスク、ブルーレイ（登録商標）ディスク、ＤＶＤ（Ｄｉｇｉｔａｌ　Ｖｅｒｓａｔｉｌｅ　Ｄｉｓｋ）といった可搬記憶媒体を使用する記憶装置であってもよい。 The auxiliary storage device 53 is a non-volatile storage device that can be read and written. The auxiliary storage device 53 stores a program for realizing the function of the computer 50 and other data. The auxiliary storage device 53 is, as a specific example, a magnetic disk device (Hard Disk Drive). The auxiliary storage device 53 may be a storage device using a portable storage medium such as an optical disc, a compact disc, a Blu-ray (registered trademark) disc, or a DVD (Digital Versatile Disk).
　入出力インタフェース装置５４は、プロセッサ５が、カメラ１１、レーザスキャナ１２、表示装置６０及び入力装置７０と通信するためのインタフェース装置である。 The input / output interface device 54 is an interface device for the processor 5 to communicate with the camera 11, the laser scanner 12, the display device 60, and the input device 70.
　計算機５０は、機能要素として、表示制御部５１ａ及び補正部５１ｂを備える。表示制御部５１ａ及び補正部５１ｂの機能は、重畳表示プログラムにより実現される。補助記憶装置５３には、表示制御部５１ａ及び補正部５１ｂの機能を実現する重畳表示プログラムが記憶されている。重畳表示プログラムは、プロセッサ５１により読み込まれ実行される。これにより、表示制御部５１ａ及び補正部５１ｂの機能が実現される。 The computer 50 includes a display control unit 51a and a correction unit 51b as functional elements. The functions of the display control unit 51a and the correction unit 51b are realized by a superimposed display program. The auxiliary storage device 53 stores a superimposed display program that realizes the functions of the display control unit 51a and the correction unit 51b. The superimposed display program is read and executed by the processor 51. Thereby, the function of the display control part 51a and the correction | amendment part 51b is implement | achieved.
　重畳表示プログラムは、表示制御部５１ａ及び補正部５１ｂの各部の「部」を「処理」、「手順」あるいは「工程」に読み替えた各処理、各手順あるいは各工程を計算機５０に実行させる。また、重畳表示方法は、計算機５０が重畳表示プログラムを実行することにより行われる方法である。重畳表示プログラムは、コンピュータ読取可能な記録媒体に格納されて提供されてもよいし、プログラムプロダクトとして提供されてもよい。 The superposition display program causes the computer 50 to execute each process, each procedure, or each process in which “part” of each part of the display control unit 51a and the correction unit 51b is read as “process”, “procedure”, or “process”. The superimposed display method is a method performed by the computer 50 executing a superimposed display program. The superimposed display program may be provided by being stored in a computer-readable recording medium or may be provided as a program product.
　図３では、プロセッサ５１は、１つだけ示されている。しかし、計算機５０は、プロセッサ５１を代替する複数のプロセッサを備えていてもよい。これら複数のプロセッサは、表示制御部５１ａ及び補正部５１ｂの機能の実行を分担する。それぞれのプロセッサは、プロセッサ５１と同じように、演算処理を行うＩＣである。プロセッサ５１及び複数のプロセッサは、総称としてプロセッシングサーキットリと呼ばれる。 In FIG. 3, only one processor 51 is shown. However, the computer 50 may include a plurality of processors that replace the processor 51. The plurality of processors share the execution of the functions of the display control unit 51a and the correction unit 51b. Each processor is an IC that performs arithmetic processing in the same manner as the processor 51. The processor 51 and the plurality of processors are collectively referred to as a processing circuit.
＊＊＊動作の説明＊＊＊
　図７から図１０を参照して、重畳表示システム１００の動作を説明する。 *** Explanation of operation ***
The operation of the superimposed display system 100 will be described with reference to FIGS.
（１）図７は、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間に、ヨー角度差△ψがある例を示す。
（２）図７は、表示制御部５１ａによって、カメラ映像１１ａとスキャンライン１２ａとが、重畳して表示装置６０に表示されている場合を示している。カメラ映像１１ａとスキャンライン１２ａとが、重畳して表示装置６０に表示される動作を説明する。
（３）作業者は、試験装置３０のカメラ１１によって計測模様４１を撮影し、また、レーザスキャナ１２によって計測模様４１をスキャンする。
（４）カメラ映像１１ａ及びスキャンライン１２ａは、入出力インタフェース装置５４を介して、表示制御部５１ａに送信される。
（５）表示制御部５１ａは、入出力インタフェース装置５４を介して、表示装置６０にカメラ映像１１ａ及びスキャンライン１２ａを重畳して表示する。スキャンライン１２ａは線で表示しているが、実際には点の集合である点群である。この点群は、レーザ点群と呼ばれる。レーザ点群は反射輝度も表示している。
（６）カメラ１１とレーザスキャナ１２とのヨー方向の姿勢が一致している場合は、カメラ映像１１ａに対して、スキャンライン１２ａが正しく重畳される。つまり、「正しく重畳される」とは、白と黒の反射輝度がカメラ映像１１ａと重なる。図７において、スキャンライン１２ａ－１は、カメラ映像１１ａに正しく重畳されているスキャンラインを示す。
（７）スキャンライン１２ａ－２の白黒パターンは、スキャンライン１２ａ－１の白黒パターンに対して、右方向にずれている状態を示す。スキャンライン１２ａ－３の白黒パターンは、スキャンライン１２ａ－１の白黒パターンに対して、左方向にずれている。スキャンライン１２ａ－２及びスキャンライン１２ａ－３は、カメラ映像１１ａに正しく重畳されていない。
（８）左方向または右方向のずれ量を修正することにより、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間の、ヨー角度差△ψを修正できる。なお、図７には、右方向のずれと左方向のずれとが一緒に記載されているが、実際には、右方向のずれと左方向のずれとのどちらかが現れる。
（９）左方向のずれを例に説明する。スキャンライン１２ａ－３の白黒パターンは、正常なスキャンライン１２ａ－１の白黒パターンに対して、左方向に長さＬ１ずれている。
一方、図１に示すように、試験装置３０の基準点から計測模様４１までの距離は、距離Ｌ０である。よって、カメラ１１とレーザスキャナ１２とのヨー角度差△ψは、以下の式１で計算できる。
△ψ＝ｔａｎ－１（Ｌ１／Ｌ０）　　　（式１）
作業者は、表示装置６０に表示されているカメラ映像１１ａとスキャンライン１２ａ－３とから長さＬ１を読み取ることで、ヨー角度差△ψを算出する。
以上により、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間の、ヨー角度差△ψが得られる。
（１０）ヨー角度差△ψが算出された場合、作業者は、キーボード７１またはマウス７２のような入力装置７０を介して、姿勢相違情報であるヨー角度差△ψを計算機５０に入力する。
つまり、計算機５０の補正部５１ｂには、入力装置７０を介して、姿勢相違情報であるヨー角度差△ψが入力される。計算機５０の補正部５１ｂは、姿勢相違情報であるヨー角度差△ψを用いて、表示装置６０に表示されるスキャンライン１２ａの位置を補正する。具体的には、補正部５１ｂを実現する重畳表示プログラムには、ベース１０に対する、カメラ１１及びレーザスキャナ１２の姿勢情報が設定されている。補正部５１ｂはヨー角度差△ψを取得した場合、この姿勢情報を補正する。姿勢情報を補正することは、実際に工具を用いて、カメラ１１の姿勢とレーザスキャナ１２の姿勢との間の、ヨー角度差△ψを修正する効果を有する。表示制御部５１ａは、補正部５１ｂによって位置が補正されたスキャンライン１２ａを、計測模様４１のカメラ映像１１ａに重畳して表示する。この重畳表示によって、作業者は、ヨー角度差△ψの修正状態を確認できる。
（１１）作業者は式１で得られた値をヨー角度差△ψとして用いた。しかし、式１で得られた値をヨー角度差△ψとして使用する代わりに、作業者は、ヨー角度差△ψに相当する相当値を計算機５０に入力してもよい。具体的には、作業者は、正常なスキャンラインが得られるまで、異なる値の相当値を計算機５０に入力し、ヨー角度ψがキャリブレーションされたスキャンラインを得てもよい。相当値は姿勢相違情報である。
　また、式１の結果を用いる場合には作業者はベース１０を回転させるが、相当値を使用する場合、作業者は、必ずしも、ベース１０を回転させなくてもよい。ベース１０を回転しない場合、姿勢変化機構２０は無くてもよい。よって図１において、カメラ１１とレーザスキャナ１２とが固定されたベース１０は、床３２に配置されても良いし、支持部材２４に配置されてもよい。 (1) FIG. 7 shows an example in which there is a yaw angle difference Δψ between the posture of the camera 11 and the posture of the laser scanner 12.
(2) FIG. 7 shows a case where the camera image 11a and the scan line 12a are displayed on the display device 60 in a superimposed manner by the display control unit 51a. An operation in which the camera video 11a and the scan line 12a are displayed on the display device 60 in a superimposed manner will be described.
(3) The operator photographs the measurement pattern 41 with the camera 11 of the test apparatus 30 and scans the measurement pattern 41 with the laser scanner 12.
(4) The camera image 11a and the scan line 12a are transmitted to the display control unit 51a via the input / output interface device 54.
(5) The display control unit 51a superimposes and displays the camera video 11a and the scan line 12a on the display device 60 via the input / output interface device 54. The scan line 12a is displayed as a line, but is actually a point group that is a set of points. This point group is called a laser point group. The laser point group also displays the reflected luminance.
(6) When the postures in the yaw direction of the camera 11 and the laser scanner 12 match, the scan line 12a is correctly superimposed on the camera image 11a. That is, “correctly superimposed” means that the reflected luminances of white and black overlap with the camera image 11a. In FIG. 7, a scan line 12a-1 indicates a scan line correctly superimposed on the camera image 11a.
(7) The black and white pattern of the scan line 12a-2 shows a state shifted to the right with respect to the black and white pattern of the scan line 12a-1. The monochrome pattern of the scan line 12a-3 is shifted to the left with respect to the monochrome pattern of the scan line 12a-1. The scan line 12a-2 and the scan line 12a-3 are not correctly superimposed on the camera image 11a.
(8) The yaw angle difference Δψ between the posture of the camera 11 and the posture of the laser scanner 12 can be corrected by correcting the shift amount in the left direction or the right direction. In FIG. 7, a rightward shift and a leftward shift are described together, but in actuality, either a rightward shift or a leftward shift appears.
(9) A leftward shift will be described as an example. The black and white pattern of the scan line 12a-3 is shifted by a length L1 in the left direction from the normal black and white pattern of the scan line 12a-1.
On the other hand, as shown in FIG. 1, the distance from the reference point of the test apparatus 30 to the measurement pattern 41 is a distance L0. Therefore, the yaw angle difference Δψ between the camera 11 and the laser scanner 12 can be calculated by the following equation 1.
Δψ = tan −1 (L1 / L0) (Formula 1)
The operator calculates the yaw angle difference Δψ by reading the length L1 from the camera image 11a and the scan line 12a-3 displayed on the display device 60.
As described above, the yaw angle difference Δψ between the posture of the camera 11 and the posture of the laser scanner 12 is obtained.
(10) When the yaw angle difference Δψ is calculated, the operator inputs the yaw angle difference Δψ, which is posture difference information, to the computer 50 via the input device 70 such as the keyboard 71 or the mouse 72.
That is, the yaw angle difference Δψ, which is attitude difference information, is input to the correction unit 51 b of the computer 50 via the input device 70. The correction unit 51b of the computer 50 corrects the position of the scan line 12a displayed on the display device 60 using the yaw angle difference Δψ that is the posture difference information. Specifically, posture information of the camera 11 and the laser scanner 12 with respect to the base 10 is set in the superimposed display program that realizes the correction unit 51b. The correction unit 51b corrects the posture information when the yaw angle difference Δψ is acquired. Correcting the posture information has an effect of correcting the yaw angle difference Δψ between the posture of the camera 11 and the posture of the laser scanner 12 by actually using a tool. The display control unit 51a superimposes and displays the scan line 12a whose position is corrected by the correction unit 51b on the camera image 11a of the measurement pattern 41. By this superimposed display, the operator can check the correction state of the yaw angle difference Δψ.
(11) The operator used the value obtained by Equation 1 as the yaw angle difference Δψ. However, instead of using the value obtained by Equation 1 as the yaw angle difference Δψ, the operator may input an equivalent value corresponding to the yaw angle difference Δψ into the calculator 50. Specifically, the operator may input a corresponding value of a different value to the computer 50 until a normal scan line is obtained, and obtain a scan line in which the yaw angle ψ is calibrated. The equivalent value is posture difference information.
Moreover, when using the result of Formula 1, the operator rotates the base 10, but when using an equivalent value, the operator does not necessarily need to rotate the base 10. When the base 10 is not rotated, the posture changing mechanism 20 may be omitted. Therefore, in FIG. 1, the base 10 to which the camera 11 and the laser scanner 12 are fixed may be disposed on the floor 32 or may be disposed on the support member 24.
（１）図８は、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間に、ピッチ角度差△θがある例を示す。
（２）図８は、表示制御部５１ａによって、計測模様４１と、スキャンライン１２ａ－４またはスキャンライン１２ａ－５とが、重畳して表示装置６０に表示されている場合を示している。スキャンライン１２ａ－４とスキャンライン１２ａ－５との区別が不要の場合、スキャンラインはスキャンライン１２ａと表記する。計測模様４１とスキャンライン１２ａとが、重畳して表示装置６０に表示される動作を説明する。
（３）作業者は、試験装置３０のカメラ１１によって計測模様４１を撮影し、レーザスキャナ１２によって計測模様４１をスキャンする。
（４）カメラ映像１１ａ及びスキャンライン１２ａは、入出力インタフェース装置５４を介して、表示制御部５１ａに送信される。
（５）表示制御部５１ａは、入出力インタフェース装置５４を介して、表示装置６０にカメラ映像１１ａ及びスキャンライン１２ａを重畳して表示する。これらは図７と同じである。
（６）カメラ１１とレーザスキャナ１２とのピッチ方向の姿勢が一致している場合は、計測模様４１に対して、スキャンライン１２ａが正しく重畳される。つまり、「正しく重畳される」とは、以下のようである。上記のように、図８は、カメラ１１とレーザスキャナ１２とのピッチ方向の姿勢がずれている場合を示している。作業者がピッチ方向、つまりピッチ軸２２（Ｙ軸）周りにベース１０を回転させた場合、表示装置６０の画面では、ベース１０の回転に伴ってスキャンライン１２ａが移動する。図１において、作業者がピッチ軸２２（Ｙ軸）の左周りにベース１０を回転させたとする。その場合、図８において、スキャンライン１２ａは、スキャンライン１２ａ－４からスキャンライン１２ａ－５へと移動する。カメラ１１とレーザスキャナ１２とのピッチ方向の姿勢がずれている場合は、スキャンライン１２ａ－４からスキャンライン１２ａ－５へ移動するときに、カメラ映像１１ａの白黒に対して、スキャンラインの白黒が反転するタイミングがずれる。つまり、カメラ１１とレーザスキャナ１２とのピッチ方向の姿勢がずれていない場合は、スキャンライン１２ａ－４から長さＬ２の位置でスキャンラインの白黒が反転する。一方、カメラ１１とレーザスキャナ１２とのピッチ方向の姿勢がずれている場合は、キャンライン１２ａ－４から長さＬ３の位置のキャンライン１２ａ－５で、スキャンラインの白黒が反転する。
（７）この例では、下方向のずれ量を修正することにより、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間の、ピッチ角度差△θを修正できる。ここで下方向のずれ量は、Ｌ３－Ｌ２である。
よって、図７の場合と同様に、カメラ１１とレーザスキャナ１２とのピッチ角度差△θは、以下の式２で計算できる。
△θ＝ｔａｎ－１（（Ｌ３－Ｌ２）／Ｌ０）　　　（式２）
作業者は、表示装置６０に表示されているカメラ映像１１ａとスキャンライン１２ａ－４及びスキャンライン１２ａ－５とから、長さ「Ｌ３－Ｌ２」を読み取ることで、ピッチ角度差△θを算出する。
　以上により、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間の、ピッチ角度差△θが得られる。
（８）ピッチ角度差△θが算出された場合、作業者は、キーボード７１またはマウス７２のような入力装置７０を介して、姿勢相違情報であるピッチ角度差△θを計算機５０に入力する。つまり、計算機５０の補正部５１ｂは、入力装置７０を介して、姿勢相違情報であるピッチ角度差△θが入力される。
　計算機５０の補正部５１ｂは、姿勢相違情報であるピッチ角度差△θを用いて、ヨー角度差△ψの場合と同様に表示装置６０に表示されるスキャンライン１２ａの位置を補正する。補正部５１ｂはピッチ角度差△θを取得した場合、図７の説明で述べた姿勢情報を補正する。姿勢情報を補正することは、実際に工具を用いて、カメラ１１の姿勢とレーザスキャナ１２の姿勢との間の、ピッチ角度差△θを修正する効果を有する。表示制御部５１ａは、補正部５１ｂによって位置が補正されたスキャンライン１２ａを、計測模様４１のカメラ映像１１ａに重畳して表示する。この重畳表示によって、作業者は、ピッチ角度差△θの修正状態を確認できる。
（９）作業者は式２で得られた値をピッチ角度差△θとして用いた。しかし、式２で得られた値をピッチ角度差△θとして使用する代わりに、作業者は、ピッチ角度差△θに相当する相当値を計算機５０に入力してもよい。相当値は姿勢相違情報である。
　具体的には、作業者は、正常なスキャンラインが得られるまで、異なる値の相当値を計算機５０に入力し、ピッチ角度θがキャリブレーションされたスキャンラインを得てもよい。相当値は姿勢相違情報である。
　また、式２の結果を用いる場合には作業者はベース１０を回転させるが、相当値を使用する場合、作業者は、必ずしも、ベース１０を回転させなくてもよい。ベース１０を回転しない場合、姿勢変化機構２０は無くてもよい。よって図１において、カメラ１１とレーザスキャナ１２とが固定されたベース１０は、床３２に配置されても良いし、支持部材２４に配置されてもよい。 (1) FIG. 8 shows an example in which there is a pitch angle difference Δθ between the posture of the camera 11 and the posture of the laser scanner 12.
(2) FIG. 8 shows a case where the measurement pattern 41 and the scan line 12a-4 or the scan line 12a-5 are superimposed and displayed on the display device 60 by the display control unit 51a. When it is not necessary to distinguish between the scan line 12a-4 and the scan line 12a-5, the scan line is expressed as a scan line 12a. An operation in which the measurement pattern 41 and the scan line 12a are superimposed on each other and displayed on the display device 60 will be described.
(3) The operator photographs the measurement pattern 41 with the camera 11 of the test apparatus 30 and scans the measurement pattern 41 with the laser scanner 12.
(4) The camera image 11a and the scan line 12a are transmitted to the display control unit 51a via the input / output interface device 54.
(5) The display control unit 51a superimposes and displays the camera video 11a and the scan line 12a on the display device 60 via the input / output interface device 54. These are the same as in FIG.
(6) When the postures in the pitch direction of the camera 11 and the laser scanner 12 match, the scan line 12a is correctly superimposed on the measurement pattern 41. That is, “correctly superimposed” is as follows. As described above, FIG. 8 shows a case where the posture in the pitch direction between the camera 11 and the laser scanner 12 is shifted. When the operator rotates the base 10 in the pitch direction, that is, around the pitch axis 22 (Y axis), the scan line 12a moves on the screen of the display device 60 as the base 10 rotates. In FIG. 1, it is assumed that the operator has rotated the base 10 around the left of the pitch axis 22 (Y axis). In that case, in FIG. 8, the scan line 12a moves from the scan line 12a-4 to the scan line 12a-5. When the posture of the camera 11 and the laser scanner 12 is shifted in the pitch direction, when moving from the scan line 12a-4 to the scan line 12a-5, the black and white of the scan line is different from the black and white of the camera image 11a. The reverse timing is shifted. That is, when the posture in the pitch direction between the camera 11 and the laser scanner 12 is not shifted, the black and white of the scan line is reversed at the position of the length L2 from the scan line 12a-4. On the other hand, when the posture in the pitch direction between the camera 11 and the laser scanner 12 is deviated, the black and white of the scan line is reversed on the can line 12a-5 at the position of the length L3 from the can line 12a-4.
(7) In this example, the pitch angle difference Δθ between the attitude of the camera 11 and the attitude of the laser scanner 12 can be corrected by correcting the downward shift amount. Here, the downward shift amount is L3-L2.
Therefore, as in the case of FIG. 7, the pitch angle difference Δθ between the camera 11 and the laser scanner 12 can be calculated by the following equation 2.
Δθ = tan −1 ((L3−L2) / L0) (Formula 2)
The operator reads the length “L3-L2” from the camera image 11a displayed on the display device 60, the scan line 12a-4, and the scan line 12a-5, thereby calculating the pitch angle difference Δθ. .
As described above, the pitch angle difference Δθ between the posture of the camera 11 and the posture of the laser scanner 12 is obtained.
(8) When the pitch angle difference Δθ is calculated, the operator inputs the pitch angle difference Δθ, which is posture difference information, to the computer 50 via the input device 70 such as the keyboard 71 or the mouse 72. That is, the correction unit 51 b of the computer 50 receives the pitch angle difference Δθ that is posture difference information via the input device 70.
The correction unit 51b of the computer 50 corrects the position of the scan line 12a displayed on the display device 60 as in the case of the yaw angle difference Δψ, using the pitch angle difference Δθ that is the posture difference information. When the correction unit 51b acquires the pitch angle difference Δθ, the correction unit 51b corrects the posture information described in the description of FIG. Correcting the posture information has an effect of correcting the pitch angle difference Δθ between the posture of the camera 11 and the posture of the laser scanner 12 by actually using a tool. The display control unit 51a superimposes and displays the scan line 12a whose position is corrected by the correction unit 51b on the camera image 11a of the measurement pattern 41. By this superimposed display, the operator can check the correction state of the pitch angle difference Δθ.
(9) The operator used the value obtained by Equation 2 as the pitch angle difference Δθ. However, instead of using the value obtained by Equation 2 as the pitch angle difference Δθ, the operator may input an equivalent value corresponding to the pitch angle difference Δθ into the calculator 50. The equivalent value is posture difference information.
Specifically, the operator may input equivalent values of different values to the computer 50 until a normal scan line is obtained, and obtain a scan line with the calibrated pitch angle θ. The equivalent value is posture difference information.
Moreover, when using the result of Formula 2, the operator rotates the base 10, but when using an equivalent value, the operator does not necessarily have to rotate the base 10. When the base 10 is not rotated, the posture changing mechanism 20 may be omitted. Therefore, in FIG. 1, the base 10 to which the camera 11 and the laser scanner 12 are fixed may be disposed on the floor 32 or may be disposed on the support member 24.
　図９は、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間のロール角度差△φがない場合を示している。図９は、ロール角度φが正しくキャリブレーションされている状態を示す。スキャンライン１２ａ－６は、ロール角度φが正しくキャリブレーションされている状態におけるスキャンラインである。ロール角度φが正しくキャリブレーションされている場合では、スキャンライン１２ａ－６の白黒は、計測模様４１の白黒と一致する。
　図１０は、カメラ１１の姿勢と、レーザスキャナ１２の姿勢との間に、ロール角度差△φのある場合を示している。図１０は、ロール角度φが正しくキャリブレーションされていない状態を示す。スキャンライン１２ａ－７は、ロール角度φが正しくキャリブレーションされていない状態におけるスキャンラインである。ロール角度φが正しくキャリブレーションされていない場合では、スキャンライン１２ａ－７の白黒は、計測模様４１の白黒と一致しない。 FIG. 9 shows a case where there is no roll angle difference Δφ between the posture of the camera 11 and the posture of the laser scanner 12. FIG. 9 shows a state where the roll angle φ is correctly calibrated. The scan line 12a-6 is a scan line in a state where the roll angle φ is correctly calibrated. When the roll angle φ is correctly calibrated, the black and white of the scan line 12 a-6 matches the black and white of the measurement pattern 41.
FIG. 10 shows a case where there is a roll angle difference Δφ between the posture of the camera 11 and the posture of the laser scanner 12. FIG. 10 shows a state where the roll angle φ is not correctly calibrated. The scan line 12a-7 is a scan line in a state where the roll angle φ is not correctly calibrated. When the roll angle φ is not correctly calibrated, the black and white of the scan line 12a-7 does not match the black and white of the measurement pattern 41.
　スキャンライン１２ａ－７がスキャンライン１２ａ－６に修正された場合を想定して、ロール角度φのキャリブレーションを説明する。
（１）図１０の状態の場合、作業者は、入力装置７０を介して、ロール角度差△φに相当する相当値を計算機５０に入力する。
（２）計算機５０の補正部５１ｂには、入力装置７０を介して、姿勢相違情報であるロール角度差△φの相当値が入力される。
（３）補正部５１ｂは、入力された相当値を姿勢相違情報として用いて、表示装置６０に表示されるスキャンライン１２ａ－７の位置を補正する。補正部５１ｂは、ヨー角度差△ψを取得した場合と同じように、姿勢情報であるロール角度φを補正する。
（４）表示制御部５１ａは、補正部５１ｂによってロール角度φが補正されたスキャンラインを、計測模様４１のカメラ映像１１ａに重畳して表示する。この重畳表示によって、作業者は、ロール角度φの修正状態を確認できる。
（５）作業者は、スキャンライン１２ａ－６が得られるまで、値を変えた相当値を計算機５０に入力する。
（６）なおロール角度差△φに相当する相当値を使用する場合、作業者は、必ずしも、ベース１０を回転させなくてもよい。ベース１０を回転しない場合、姿勢変化機構２０は無くてもよい。よって図１において、カメラ１１とレーザスキャナ１２とが固定されたベース１０は、床３２に配置されても良いし、支持部材２４に配置されてもよい。 The calibration of the roll angle φ will be described assuming that the scan line 12a-7 is corrected to the scan line 12a-6.
(1) In the state of FIG. 10, the operator inputs an equivalent value corresponding to the roll angle difference Δφ to the computer 50 via the input device 70.
(2) An equivalent value of the roll angle difference Δφ, which is posture difference information, is input to the correction unit 51 b of the computer 50 via the input device 70.
(3) The correction unit 51b corrects the position of the scan line 12a-7 displayed on the display device 60 using the input equivalent value as posture difference information. The correction unit 51b corrects the roll angle φ, which is posture information, in the same manner as when the yaw angle difference Δψ is acquired.
(4) The display control unit 51a superimposes and displays the scan line whose roll angle φ is corrected by the correction unit 51b on the camera image 11a of the measurement pattern 41. By this superimposed display, the operator can check the correction state of the roll angle φ.
(5) The operator inputs an equivalent value obtained by changing the value into the computer 50 until the scan line 12a-6 is obtained.
(6) When using an equivalent value corresponding to the roll angle difference Δφ, the operator does not necessarily have to rotate the base 10. When the base 10 is not rotated, the posture changing mechanism 20 may be omitted. Therefore, in FIG. 1, the base 10 to which the camera 11 and the laser scanner 12 are fixed may be disposed on the floor 32 or may be disposed on the support member 24.
　実際の重畳映像では、ロール角度差△φ、ピッチ角度差△θ及びヨー角度差△ψが混在して現れる。その場合、作業者は、ベース１０を、ロール軸２１（Ｘ軸）、ピッチ軸２２（Ｙ軸）またはヨー軸２３（Ｚ軸）の回りに回転させる。そして、作業者は、図７のようにヨー角度差△ψが現れる状態、図８のようにピッチ角度差△θが現れる状態を得て、ヨー角度差△ψ、ピッチ角度差△θ等を個々に確認する。 In the actual superimposed image, a roll angle difference Δφ, a pitch angle difference Δθ, and a yaw angle difference Δψ appear together. In that case, the operator rotates the base 10 around the roll axis 21 (X axis), the pitch axis 22 (Y axis), or the yaw axis 23 (Z axis). Then, the operator obtains the state in which the yaw angle difference Δψ appears as shown in FIG. 7 and the state in which the pitch angle difference Δθ appears as shown in FIG. 8, and sets the yaw angle difference Δψ, the pitch angle difference Δθ, etc. Check individually.
＊＊＊実施の形態１の効果の説明＊＊＊
　以上に説明した実施の形態１の重畳表示システム１００によれば、カメラ１１及びレーザスキャナ１２が搭載された計測車両が、キャリブレーションのための走行をする必要がない。よって、キャリブレーションの負担が軽減できる。
　また、重畳表示システム１００によれば、カメラ１１及びレーザスキャナ１２が搭載されたベースの製造精度が簡易な方法で確認できる。
　つまりベース１０に固定されたカメラ１１及びレーザスキャナ１２の取り付け姿勢の精度が、簡易な方法で確認できる。 *** Explanation of effects of embodiment 1 ***
According to the superimposed display system 100 of the first embodiment described above, the measurement vehicle on which the camera 11 and the laser scanner 12 are mounted does not need to travel for calibration. Therefore, the burden of calibration can be reduced.
Further, according to the superimposed display system 100, the manufacturing accuracy of the base on which the camera 11 and the laser scanner 12 are mounted can be confirmed by a simple method.
That is, the accuracy of the mounting posture of the camera 11 and the laser scanner 12 fixed to the base 10 can be confirmed by a simple method.
　以上に説明した重畳表示システム１００では、△φ、△θ、△ψを求める方法を説明したが、一般的なキャリブレーションパラメータの推定方法と同様に、カメラ１１とレーザスキャナ１２との姿勢及び位置を含めたキャリブレーションパラメータの推定に使用できる。その場合、カメラ１１とレーザスキャナ１２との姿勢及び位置を含めたキャリブレーションパラメータの推定原理は、ＭＭＳ計測車両を走行させるキャリブレーションと同じである。
　重畳表示システム１００におけるキャリブレーションパラメータの推定の場合も、計測模様４１の近くでカメラ１１及びレーザスキャナ１２の位置が推定され、計測模様４１の遠くで姿勢が推定される。 In the superimposed display system 100 described above, the method for obtaining Δφ, Δθ, and Δψ has been described. However, the posture and position of the camera 11 and the laser scanner 12 are similar to the general calibration parameter estimation method. Can be used to estimate calibration parameters including In this case, the calibration parameter estimation principle including the posture and position of the camera 11 and the laser scanner 12 is the same as the calibration for running the MMS measurement vehicle.
Also in the case of estimation of calibration parameters in the superimposed display system 100, the positions of the camera 11 and the laser scanner 12 are estimated near the measurement pattern 41, and the posture is estimated far from the measurement pattern 41.
　以上の説明では、試験装置３０は静止状態で計測模様４１を正面から写す場合が想定されていた。しかし、カメラ１１及びレーザスキャナ１２が動かされながら、カメラ１１とレーザスキャナ１２との姿勢、またはカメラ１１とレーザスキャナ１２との位置が決められることは有効である。つまり、作業者が計測対象物４０を動かすこと、あるいは、カメラ１１、レーザスキャナ１２が、ロール軸２１、ピッチ軸２２またはヨー軸２３の周りに回転することは好ましい。最終的にどのような角度、どのような位置から撮影されたカメラ映像１１ａであっても、そのカメラ映像１１ａにスキャンライン１２ａが正しく重畳された場合、キャリブレーションは終了である。 In the above description, it is assumed that the test apparatus 30 captures the measurement pattern 41 from the front in a stationary state. However, it is effective to determine the posture of the camera 11 and the laser scanner 12 or the position of the camera 11 and the laser scanner 12 while the camera 11 and the laser scanner 12 are moved. That is, it is preferable that the operator moves the measurement object 40 or that the camera 11 and the laser scanner 12 rotate around the roll axis 21, the pitch axis 22, or the yaw axis 23. Even if the camera image 11a is finally taken from any angle and from any position, the calibration is completed when the scan line 12a is correctly superimposed on the camera image 11a.
　姿勢をより正確に確定したい場合は、カメラ１１及びレーザスキャナ１２と、計測模様４１との距離を、通常よりも長くとればよい。この理由は、姿勢の影響は、遠方ほど大きく現れるからである。例えば、０．１ｄｅｇの誤差に対して１ｍであれば１．７ｍｍであるが１００ｍであれば１７．４ｃｍとなるからである。
　また、位置（Ｘ，Ｙ，Ｚ）を推定する場合は、試験装置３０が計測模様４１に近い距離で推定を行うことが有効である。位置の影響は、カメラ１１及びレーザスキャナ１２と計測模様４１との距離に依存しないので、姿勢の影響の少ない近傍のほうが位置の誤差が大きく現れるからである。
　これらのことは走行キャリブレーション手法と同様である。 When it is desired to determine the posture more accurately, the distance between the camera 11 and the laser scanner 12 and the measurement pattern 41 may be set longer than usual. The reason for this is that the influence of the posture appears larger as the distance increases. For example, for an error of 0.1 deg, it is 1.7 mm for 1 m but 17.4 cm for 100 m.
When estimating the position (X, Y, Z), it is effective that the test apparatus 30 performs the estimation at a distance close to the measurement pattern 41. This is because the influence of the position does not depend on the distance between the camera 11 and the laser scanner 12 and the measurement pattern 41, so that a position error appears more in the vicinity where the influence of the posture is small.
These are the same as the running calibration method.
　以上、実施の形態１について説明したが、実施の形態１に示した構成を部分的に実施しても構わない。なお、本発明は、実施の形態１に限定されるものではなく、必要に応じて種々の変更が可能である。 Although the first embodiment has been described above, the configuration shown in the first embodiment may be partially implemented. In addition, this invention is not limited to Embodiment 1, A various change is possible as needed.
　φ　ロール角度、θ　ピッチ角度、ψ　ヨー角度、△φ　ロール角度差、△θ　ピッチ角度差、△ψ　ヨー角度差、１０　ベース、１１　カメラ、１１ａ　カメラ映像、１２　レーザスキャナ、１２ａ　スキャンライン、２０　姿勢変化機構、２１　ロール軸、２２　ピッチ軸、２３　ヨー軸、２４　支持部材、３０　試験装置、４０　計測対象物、４１　計測模様、４２　配置面、４３　キャスタ、５０　計算機、５１　プロセッサ、５１ａ　表示制御部、５１ｂ　補正部、５２　主記憶装置、５３　補助記憶装置、５４　入出力インタフェース装置、５５　信号線、６０　表示装置、７０　入力装置、７１　キーボード、７２　マウス、８０　姿勢相違情報、８１　一点鎖線、８２　床、１００　重畳表示システム。 φ roll angle, θ pitch angle, ψ yaw angle, Δφ roll angle difference, Δθ pitch angle difference, Δψ yaw angle difference, 10 base, 11 camera, 11a camera image, 12 laser scanner, 12a scan line, 20 posture Change mechanism, 21 roll axis, 22 pitch axis, 23 yaw axis, 24 support member, 30 test device, 40 measurement object, 41 measurement pattern, 42 arrangement surface, 43 caster, 50 computer, 51 processor, 51a display control unit, 51b correction unit, 52 main storage device, 53 auxiliary storage device, 54 input / output interface device, 55 signal line, 60 display device, 70 input device, 71 keyboard, 72 mouse, 80 posture difference information, 81 dash-dot line, 82 floor, 100 Superimposed display system.

Claims (6)

  1. 　カメラとレーザスキャナとが固定されたベースと、
    　前記カメラと前記レーザスキャナとが固定された前記ベースが固定され、前記ベースのロール角度、ピッチ角度及びヨー角度を変化させる姿勢変化機構と、
    　前記カメラに撮影される模様であり、前記レーザスキャナによるスキャンによって反射輝度の相違が現れる模様である計測模様を有する計測対象物と、
    　前記カメラによって撮影された前記計測模様のカメラ映像と、前記レーザスキャナによる前記計測模様のスキャン結果であるスキャンラインとを取得し、前記カメラ映像と前記スキャンラインとを重畳して表示装置に表示する計算機と
    を備える重畳表示システム。     A base to which a camera and a laser scanner are fixed;
    An attitude changing mechanism that fixes the base to which the camera and the laser scanner are fixed, and changes a roll angle, a pitch angle, and a yaw angle of the base;
    A measurement object having a measurement pattern which is a pattern photographed by the camera and a pattern in which a difference in reflected luminance appears by scanning with the laser scanner;
    A camera image of the measurement pattern photographed by the camera and a scan line that is a scan result of the measurement pattern by the laser scanner are acquired, and the camera image and the scan line are superimposed and displayed on a display device. A superimposed display system comprising a computer.
  2. 　前記計測対象物は、
    　前記ベースとの距離を変えることのできる移動機構を有する請求項１に記載の重畳表示システム。     The measurement object is
    The superimposed display system according to claim 1, further comprising a moving mechanism capable of changing a distance from the base.
  3. 　前記計算機は、
    　前記カメラと前記レーザスキャナとの姿勢の相違を示す姿勢相違情報を用いて、前記表示装置に表示される前記スキャンラインの位置を補正し、位置の補正された前記スキャンラインを前記計測模様のカメラ映像と重畳して表示する請求項１または請求項２に記載の重畳表示システム。     The calculator is
    Using the position difference information indicating the position difference between the camera and the laser scanner, the position of the scan line displayed on the display device is corrected, and the position-corrected scan line is used as the measurement pattern camera. The superimposed display system according to claim 1, wherein the superimposed display system displays the image superimposed on the video.
  4. 　前記計算機は、
    　前記姿勢相違情報が入力される請求項３に記載の重畳表示システム。     The calculator is
    The superimposed display system according to claim 3, wherein the posture difference information is input.
  5. 　前記計測模様は、
    　チェック模様である請求項１から請求項４のいずれか一項に記載の重畳表示システム。     The measurement pattern is
    The superimposed display system according to claim 1, wherein the superimposed display system is a check pattern.
  6. 　カメラとレーザスキャナとが固定されたベースと、
    　前記カメラに撮影される模様であり、前記レーザスキャナによるスキャンによって反射輝度の相違が現れる模様である計測模様を有する計測対象物と、
    　前記カメラによって撮影された前記計測模様のカメラ映像と、前記レーザスキャナによる前記計測模様のスキャン結果であるスキャンラインとを取得し、前記カメラ映像と前記スキャンラインとを重畳して表示装置に表示するとともに、前記カメラと前記レーザスキャナとの姿勢の相違を示す姿勢相違情報を用いて、前記表示装置に表示される前記スキャンラインの位置を補正し、位置の補正された前記スキャンラインを前記計測模様のカメラ映像と重畳して表示する計算機と
    を備える重畳表示システム。     A base to which a camera and a laser scanner are fixed;
    A measurement object having a measurement pattern which is a pattern photographed by the camera and a pattern in which a difference in reflected luminance appears by scanning with the laser scanner;
    A camera image of the measurement pattern photographed by the camera and a scan line that is a scan result of the measurement pattern by the laser scanner are acquired, and the camera image and the scan line are superimposed and displayed on a display device. The position difference of the scan line displayed on the display device is corrected using the position difference information indicating the position difference between the camera and the laser scanner, and the position of the scan line is corrected to the measurement pattern. A superimposition display system comprising a computer that superimposes and displays the camera image.
PCT/JP2018/010642 2018-03-16 2018-03-16 Superimposed display system WO2019176118A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2018/010642 WO2019176118A1 (en) 2018-03-16 2018-03-16 Superimposed display system
JP2020506104A JP7003219B2 (en) 2018-03-16 2018-03-16 Superimposed display system
TW107118478A TWI645160B (en) 2018-03-16 2018-05-30 Overlay display system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2018/010642 WO2019176118A1 (en) 2018-03-16 2018-03-16 Superimposed display system

Publications (1)

Publication Number Publication Date
WO2019176118A1 true WO2019176118A1 (en) 2019-09-19

Family

ID=65432167

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/010642 WO2019176118A1 (en) 2018-03-16 2018-03-16 Superimposed display system

Country Status (3)

Country Link
JP (1) JP7003219B2 (en)
TW (1) TWI645160B (en)
WO (1) WO2019176118A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021085679A (en) * 2019-11-25 2021-06-03 トヨタ自動車株式会社 Target device for sensor axis adjustment
CN114964183A (en) * 2022-07-28 2022-08-30 金田产业发展(山东)集团有限公司 Portable positioning navigation engineering surveying device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003505682A (en) * 1999-07-13 2003-02-12 メトロノール・エイエスエイ A system for scanning the geometry of large objects
JP2011087308A (en) * 2010-11-25 2011-04-28 Aisin Seiki Co Ltd Device, method, and program for calibration of in-vehicle camera
JP2016017913A (en) * 2014-07-10 2016-02-01 国立大学法人鳥取大学 Posture information preparation system, posture information preparation method, and posture information preparation program
JP2016045150A (en) * 2014-08-26 2016-04-04 株式会社トプコン Point group position data processing device, point group position data processing system, point group position data processing method, and program
JP2017062848A (en) * 2016-12-13 2017-03-30 三菱電機株式会社 Calibration method, program, and computer

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07229961A (en) * 1994-02-22 1995-08-29 Hitachi Ltd Apparatus and method of detecting object about vehicle
JPH11326495A (en) * 1998-05-13 1999-11-26 Hitachi Ltd Antenna shaft adjusting device for on-vehicle radar
JP3708510B2 (en) * 2002-08-26 2005-10-19 本田技研工業株式会社 In-vehicle radar and in-vehicle camera aiming and inspection system
JP4356573B2 (en) * 2004-09-24 2009-11-04 株式会社日立製作所 How to display the radar installation information confirmation screen and adjustment screen
JP4533824B2 (en) * 2005-08-30 2010-09-01 株式会社日立製作所 Image input device and calibration method
JP2010511212A (en) * 2006-11-03 2010-04-08 テレ アトラス ベスローテン フエンノートシャップ Method and apparatus for identifying and locating planar objects in an image
EP2120009B1 (en) * 2007-02-16 2016-09-07 Mitsubishi Electric Corporation Measuring device and measuring method
US20100157280A1 (en) * 2008-12-19 2010-06-24 Ambercore Software Inc. Method and system for aligning a line scan camera with a lidar scanner for real time data fusion in three dimensions
US9470548B2 (en) * 2011-01-31 2016-10-18 Agency For Defense Development Device, system and method for calibration of camera and laser sensor
DE102011120535A1 (en) * 2011-12-08 2013-06-13 GM Global Technology Operations LLC (n. d. Gesetzen des Staates Delaware) Method for adjusting sensor during manufacture of motor car, involves comparing object positions determined relative to vehicle reference axis, to produce comparative data and adjusting preset sensor as a function of comparative data
US9134402B2 (en) * 2012-08-13 2015-09-15 Digital Signal Corporation System and method for calibrating video and lidar subsystems
US9875557B2 (en) * 2012-11-05 2018-01-23 The Chancellor Masters And Scholars Of The University Of Oxford Extrinsic calibration of imaging sensing devices and 2D LIDARs mounted on transportable apparatus
JP2016057108A (en) * 2014-09-08 2016-04-21 株式会社トプコン Arithmetic device, arithmetic system, arithmetic method and program
KR101672732B1 (en) * 2014-12-24 2016-11-07 한국항공우주연구원 Apparatus and method for tracking object
DE102015118874A1 (en) * 2015-11-04 2017-05-04 Valeo Schalter Und Sensoren Gmbh Method for operating a sensor system of a motor vehicle, driver assistance system and system for calibrating a sensor system of a motor vehicle
JP6854828B2 (en) * 2015-12-18 2021-04-07 ジェラルド ディルク スミッツ Real-time position detection of an object
DE112017001322T5 (en) * 2016-03-16 2018-12-27 Sony Corporation Signal processing apparatus and signal processing method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003505682A (en) * 1999-07-13 2003-02-12 メトロノール・エイエスエイ A system for scanning the geometry of large objects
JP2011087308A (en) * 2010-11-25 2011-04-28 Aisin Seiki Co Ltd Device, method, and program for calibration of in-vehicle camera
JP2016017913A (en) * 2014-07-10 2016-02-01 国立大学法人鳥取大学 Posture information preparation system, posture information preparation method, and posture information preparation program
JP2016045150A (en) * 2014-08-26 2016-04-04 株式会社トプコン Point group position data processing device, point group position data processing system, point group position data processing method, and program
JP2017062848A (en) * 2016-12-13 2017-03-30 三菱電機株式会社 Calibration method, program, and computer

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021085679A (en) * 2019-11-25 2021-06-03 トヨタ自動車株式会社 Target device for sensor axis adjustment
CN114964183A (en) * 2022-07-28 2022-08-30 金田产业发展(山东)集团有限公司 Portable positioning navigation engineering surveying device
CN114964183B (en) * 2022-07-28 2022-10-25 金田产业发展(山东)集团有限公司 Portable positioning navigation engineering surveying device

Also Published As

Publication number Publication date
TW201938988A (en) 2019-10-01
JPWO2019176118A1 (en) 2020-12-03
JP7003219B2 (en) 2022-01-20
TWI645160B (en) 2018-12-21

Similar Documents

Publication Publication Date Title
JP4669047B2 (en) Method and device for position detection in an imaging system
US6798527B2 (en) Three-dimensional shape-measuring system
US20110270562A1 (en) Profile measuring apparatus
US7487491B2 (en) Pattern inspection system using image correction scheme with object-sensitive automatic mode switchability
US20100074511A1 (en) Mask inspection apparatus, and exposure method and mask inspection method using the same
WO2019176118A1 (en) Superimposed display system
JP2009222568A (en) Method, device, and computer program of generating three-dimensional shape data
JP4445472B2 (en) Calibration software for surface reconstruction of small objects
JP2022086966A5 (en)
JP2021076606A (en) Method and device for creating calibration value for calibrating inertial measurement unit for vehicle
JP3999063B2 (en) CMM, CMM calibration method, and computer-readable storage medium storing program for executing the method
JP2014044060A (en) Shape measurement device and shape measurement method
JP5955001B2 (en) Aspherical shape measurement method, shape measurement program, and shape measurement device
US20210256676A1 (en) Inspection apparatus, inspection method, and storage medium
JP2002310641A (en) Coordinate system calibrating method for three- dimensional shape measuring instrument
JP2009288227A (en) Three-dimensional measuring machine
US20230054389A1 (en) Three-dimensional shape registration method and three-dimensional shape data processing device
JP4196842B2 (en) Attitude angle detection method and attitude angle detection device
WO2013084565A1 (en) Data correction apparatus and data correction program
JP2012013593A (en) Calibration method for three-dimensional shape measuring machine, and three-dimensional shape measuring machine
JP2019158852A (en) Speed creation device, speed creation program, and speed creation method
JP6899236B2 (en) Relationship identification method, relationship identification device, relationship identification program, correction method, correction device, and correction program
JP2016008838A (en) Shape measuring device, structure manufacturing system, shape measuring method, and structure manufacturing method
JP4802134B2 (en) Posture change measuring method and apparatus
JP2010151591A (en) Method and program for evaluating wave front

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18909494

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020506104

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18909494

Country of ref document: EP

Kind code of ref document: A1