US20210349195A1 - Arithmetic processing apparatus, distance measuring apparatus, and arithmetic processing method - Google Patents

Arithmetic processing apparatus, distance measuring apparatus, and arithmetic processing method Download PDF

Info

Publication number
US20210349195A1
US20210349195A1 US17/286,685 US201917286685A US2021349195A1 US 20210349195 A1 US20210349195 A1 US 20210349195A1 US 201917286685 A US201917286685 A US 201917286685A US 2021349195 A1 US2021349195 A1 US 2021349195A1
Authority
US
United States
Prior art keywords
distance
light
frequency
error
corrected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US17/286,685
Other languages
English (en)
Inventor
Mitsuharu Ohki
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Semiconductor Solutions Corp
Original Assignee
Sony Semiconductor Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Semiconductor Solutions Corp filed Critical Sony Semiconductor Solutions Corp
Assigned to SONY SEMICONDUCTOR SOLUTIONS CORPORATION reassignment SONY SEMICONDUCTOR SOLUTIONS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: OHKI, MITSUHARU
Publication of US20210349195A1 publication Critical patent/US20210349195A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/497—Means for monitoring or calibrating
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
    • G01S17/36—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G01C3/08—Use of electric radiation detectors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • G01S17/26—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves wherein the transmitted pulses use a frequency-modulated or phase-modulated carrier wave, e.g. for pulse compression of received signals
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar

Definitions

  • the present technology relates to a technical field of an arithmetic processing apparatus that measures a distance to a measurement object by emitting light of irradiation light and receiving reflected light that is the irradiation light reflected by the measurement object, a distance measuring apparatus, and an arithmetic processing method.
  • ToF time of flight
  • a distance to a measurement object is measured by irradiating the measurement object with irradiation light whose intensity is periodically changed so that the intensity change forms a sine wave, and receiving reflected light.
  • the distance to the measurement object is measured by measuring a phase difference between the irradiation light and the reflected light received while synchronizing with the irradiation light.
  • a light receiving sensor that receives the reflected light includes pixels arranged in a two-dimensional array. Each pixel has a light receiving element and can take in light. Furthermore, each pixel can acquire phase and amplitude of the received sine wave by receiving light while synchronizing with the light emission of the irradiation light. Note that the phase reference is a sine wave of the irradiation light.
  • the phase acquired by each pixel corresponds to the time until the irradiation light emitted from the light emitting part is reflected by the measurement object and input to the light receiving sensor. Therefore, by dividing the phase by 2 ⁇ f ( ⁇ : pi, f: the frequency of the sine wave used for intensity modulation), multiplying the division result by the light speed (c: about 300,000 km/s), and dividing the multiplication result by 2 (in order to convert reciprocating distance to one-way distance), the distance to the object imaged in the pixel is calculated.
  • Patent Document 1 discloses a technology using averaging processing and median filter processing as an example of noise reduction.
  • Patent Document 1 WO2017/022152
  • An arithmetic processing apparatus includes an operation processing part that performs processing of: calculating a first distance to a measurement object by emitting and receiving first irradiation light of which intensity is modulated by a first modulation signal of a first frequency; calculating a second distance to the measurement object by emitting and receiving second irradiation light of which intensity is modulated by a second modulation signal of a second frequency different from the first frequency; calculating a corrected first distance using first correction data that has been acquired; and determining the corrected first distance within an error range of the second distance as a third distance using the second distance.
  • the first correction data may be data for correcting an error of a measurement distance caused by an error in the intensity change of the first irradiation light with respect to the sine wave of the first frequency.
  • the first distance is considered to include an error due to the fact that the intensity change of the first irradiation light does not form an accurate sine wave.
  • the first frequency may be a higher frequency than the second frequency.
  • the distance measurement result using the first irradiation light in which the error for an accurate sine wave can be corrected is more suitable for measuring an accurate distance. Furthermore, since the second irradiation light has a lower frequency than the first irradiation light, the distance measuring range is excellent.
  • the third distance may be considered to include an error due to noise
  • the arithmetic processing apparatus may further include processing of: acquiring second correction data for correcting an error of the second distance caused by an error in an intensity change of the second irradiation light with respect to a sine wave of the second frequency; defining a corrected second distance obtained by correcting the second distance using the second correction data; defining a difference between the corrected first distance and the third distance as a first difference, defining a difference between the corrected second distance and the third distance as a second difference; and calculating an error due to the noise so that the first difference and the second difference become smaller.
  • the third distance includes not only an error that can be corrected by the first correction data, that is, an error due to the fact that the intensity change of the first irradiation light does not form an accurate sine wave, but also an error due to the noise component such as natural light entering the light receiving part during distance measurement (error due to noise).
  • the second correction data for correcting the error of the second distance due to the fact that the intensity change of the second irradiation light includes an error with respect to the sine wave is calculated, and moreover, an error due to noise is appropriately calculated from the difference of the third distance with each of the corrected first distance and the corrected second distance.
  • the second correction data may be defined by a correction function approximated by a second-order Taylor series expansion.
  • the error of the third distance may be calculated so that the second difference becomes smaller for a neighboring pixel in spatiotemporal space with respect to a pixel of interest.
  • each coefficient of the approximate expansion is calculated so that not only the first difference and the second difference of the pixel of interest, but also a first difference and a second difference of neighboring pixel in the spatiotemporal space of the pixel of interest become smaller.
  • the neighboring pixel may be a pixel that has been extracted under a condition that the difference between the second distance before correction for the pixel of interest and the second distance before correction for the neighboring pixel is equal to or less than a predetermined threshold.
  • a distance measuring apparatus includes: a light emitting part capable of light emission of first irradiation light of which intensity is modulated by a first modulation signal of a first frequency, and light emission of second irradiation light of which intensity is modulated by a second modulation signal of a second frequency different from the first frequency; a light receiving part that receives reflected light that is light emitted from the light emitting part and reflected by a measurement object; and an operation processing part that performs processing of calculating a first distance to a measurement object by emitting and receiving first irradiation light, calculating a second distance to the measurement object by emitting and receiving second irradiation light, calculating a corrected first distance using first correction data that has been acquired, and determining the corrected first distance within an error range of the second distance as a third distance using the second distance.
  • Pieces of irradiation light whose intensity is modulated at different frequencies can be emitted and each reflected light reflected by the measurement object can be received, and therefore, two pieces of distance measurement data with different distance resolution and distance measurement range can be acquired. It is possible to calculate the third distance as the measurement distance from these two distance measurement data.
  • the distance measuring apparatus may include a storage part in which the first correction data is stored.
  • An arithmetic processing method includes: calculating a first distance between a distance measuring apparatus and a measurement object by emitting and receiving light of which intensity is modulated by a first modulation signal of a first frequency; calculating a second distance between the distance measuring apparatus and the measurement object by emitting and receiving light of which intensity is modulated by a second modulation signal of a second frequency different from the first frequency; calculating a corrected first distance using first correction data that has been acquired; and determining the corrected first distance within an error range of the second distance as a third distance using the second distance.
  • FIG. 1 is a diagram showing a system configuration of a distance measuring apparatus according to an embodiment of the present technology.
  • FIG. 2 is a diagram for explaining a phase difference between irradiation light and reflected light of the embodiment.
  • FIG. 3 is a diagram for explaining indefiniteness of distance of the embodiment.
  • FIG. 4 is a diagram for explaining a relationship between distance resolution and distance measurement range.
  • FIG. 5 is a flowchart for performing distance measurement processing according to the embodiment.
  • a configuration of a distance measuring apparatus 1 will be described. Note that the distance measuring apparatus 1 described here is just an example of an embodiment in which the present technology can be implemented.
  • the distance measuring apparatus 1 includes a control part 1 a, a light emitting part 1 b, a light receiving part 1 c, an amplitude detection part 1 d, a phase difference detection part 1 e, an operation processing part 1 f, a storage part 1 g, and an output part 1 h.
  • the control part 1 a can control each part of the light emitting part 1 b, the light receiving part 1 c, the amplitude detection part 1 d, the phase difference detection part 1 e, the operation processing part 1 f, the storage part 1 g, and the output part 1 h by generating a control signal.
  • the light emitting part 1 b emits irradiation light whose intensity is modulated so that the intensity change forms a sine wave on the basis of a drive signal of a predetermined frequency supplied by the control part 1 a. That is, if the change with time of the light intensity of the irradiation light is represented by a graph, it becomes a sine wave.
  • the light modulated so that the intensity change forms a sine wave of 10 MHz is described as “light of 10 MHz” or the like.
  • the light emitting part 1 b is capable of emitting irradiation light having two types of frequencies.
  • the frequencies are light of 10 MHz and light of 40 MHz.
  • Two types of irradiation light having different frequencies are emitted, for example, in a time-division manner.
  • the light receiving part 1 c receives reflected light that is the irradiation light emitted from the light emitting part 1 b and is reflected by the measurement object 100 returning to the light receiving part 1 c.
  • the light receiving part 1 c has, for example, a light receiving sensor (image sensor) in which light receiving elements are arranged in a two-dimensional array.
  • the reception of the reflected light by the light receiving part 1 c is performed in synchronization with the light emission cycle of the irradiation light by the light emitting part 1 b.
  • the light receiving part 1 c accumulates the reflected light over tens of thousands of times (tens of thousands of cycles) of the intensity change of the irradiation light, and outputs data proportional to the amount of received light accumulated. This is because the amount of reflected light received for one cycle of irradiation light is very small, so that significant data may not be obtained. That is, by accumulating the reflected light for tens of thousands of cycles, a sufficient amount of received light can be obtained, and significant information can be acquired.
  • the light receiving part 1 c may be provided with a condenser lens in order to efficiently receive the reflected light.
  • the amplitude detection part 1 d calculates the amplitude of the reflected light for each pixel from the amount of light received by the light receiving part 1 c.
  • the calculated amplitude information is used in the processing of the operation processing part 1 f, which will be described later.
  • the phase difference detection part 1 e calculates for each pixel a phase difference indicating how much the reflected light received in synchronization with the emission of the irradiation light is deviated from the irradiation light. Since the phase difference is proportional to the distance to the measurement object 100 (hereinafter, also simply referred to as “distance”), the distance to the measurement object 100 can be calculated using the phase difference. This will be described later specifically.
  • the operation processing part 1 f uses the amplitude information for each pixel detected by the amplitude detection part 1 d, the phase difference information for each pixel detected by the phase difference detection part 1 e, the correction data, and the like, to calculate the distance to the measurement object 100 for each pixel. Details of the operation for calculating the distance will be described later.
  • the storage part 1 g stores correction data, correction formulas, and the like used when the operation processing part 1 f calculates the distance.
  • the output part 1 h performs processing of outputting the information of distance to the measurement object 100 calculated for each pixel by the operation processing part 1 f as a distance image.
  • FIG. 2A is a diagram showing the time change of the intensity of the irradiation light of 10 MHz emitted from the light emitting part 1 b.
  • FIG. 2B is a diagram showing the time change of the light intensity when the reflected light of 10 MHz is received by the light receiving part 1 c.
  • phase difference ⁇ L corresponds to the time in which the light reciprocates between the distance measuring apparatus 1 and the measurement object 100 . Note that, in a case where the amplitude of the reflected light shown in FIG. 2B is extremely small, it is difficult to detect the phase difference ⁇ L of the reflected light, and a large number of errors are likely to be included.
  • FIG. 2C is a diagram showing the time change of the intensity of the irradiation light of 40 MHz emitted from the light emitting part 1 b.
  • FIG. 2D is a diagram showing the time change of the light intensity when the reflected light of 40 MHz is received by the light receiving part 1 c.
  • phase difference ⁇ H corresponds to the time in which the light reciprocates between the distance measuring apparatus 1 and the measurement object 100 . Note that, in a case where the amplitude of the reflected light shown in FIG. 2D is extremely small, it is difficult to detect the phase difference ⁇ H of the reflected light, and a large number of errors are likely to be included.
  • the resolution of the phase difference that can be detected by the phase difference detection part 1 e is finite.
  • the distance to the measurement object 100 can be calculated by dividing the phase difference ⁇ L and the phase difference ⁇ H by 2 ⁇ f (f: frequency of the irradiation light), multiplying the division result by the light speed c, and further dividing the multiplication result by 2. Accordingly, the resolution of the distance becomes higher as the value of f increases. That is, the distance resolution is higher when irradiation light of 40 MHz is used than when irradiation light of 10 MHz is used. Specifically, if the frequency f is quadrupled, the distance resolution is also proportionally quadrupled.
  • FIG. 3A is a diagram showing the time change of the intensity of the irradiation light of a predetermined frequency emitted from the light emitting part 1 b.
  • FIG. 3B is a diagram showing the time change of the light intensity when the reflected light that is the irradiation light reflected by the measurement object is received by the light receiving part 1 c.
  • the phase difference ⁇ between FIGS. 3A and 3B is proportional to the distance between the distance measuring apparatus 1 and the measurement object 100 . As the distance between the distance measuring apparatus 1 and the measurement object 100 increases, the phase difference eventually reaches ( ⁇ +2 ⁇ ) ( FIG. 3C ). As similar to this, there is a distance between the distance measuring apparatus 1 and the measurement object 100 whose phase difference is ( ⁇ +4 ⁇ ) ( FIG. 3D ).
  • the distance measuring range in the distance measuring apparatus 1 is a distance at which the phase difference is less than 2 ⁇ .
  • the distance measurement range of 10 MHz is four times longer than that of the case of 40 MHz.
  • FIG. 4 summarizes the distance resolution and distance measurement range.
  • the lower the frequency f the longer the distance measurement range but the lower the distance resolution. Furthermore, the higher the frequency f, the higher the distance resolution but the shorter the distance measurement range.
  • the time change of the light intensity of the irradiation light forms an ideal sine wave by performing the intensity modulation at a predetermined frequency f.
  • the time change of the light intensity of the irradiation light does not form an ideal sine wave, and includes not a little error. Therefore, in the present embodiment, the error of the distance measurement result caused by the intensity change not being an accurate sine wave is corrected.
  • the distance measurement result before the error correction is described as “distance before correction”
  • the distance measurement result calculated as a result of error correction is described as “distance after correction”.
  • the actual distance between the distance measuring apparatus 1 and the measurement object 100 is described as “true distance”.
  • the correction amount for converting the distance before correction into the distance after correction is measured, for example, in the manufacturing process before shipment of the distance measuring apparatus 1 . Specifically, the correspondence between the distance before correction and the true distance is calculated while gradually changing the distance between the distance measuring apparatus 1 and the measurement object 100 .
  • correction amount calculated from the measurement result is stored in the storage part 1g of the distance measuring apparatus 1 .
  • correction amount information may be configured so that it can be acquired from another information processing apparatus via a communication network.
  • the distance after correction is calculated using the distance before correction and the correction value acquired from the storage part 1g.
  • the correction value stored in the storage part 1 g may be a function showing the relationship between the distance before correction and the correction amount, or may be a correspondence table showing the relationship between the distance before correction and the correction amount.
  • the correction amount for the distance before correction that is not stored may be configured to be obtained by linear interpolation or the like.
  • the correction amount is calculated only for one of the two frequencies used (for example, 10 MHz and 40 MHz).
  • the correction amount is unknown, so that the distance before correction including an error cannot be corrected.
  • Equation 1 i, j ⁇ ⁇ ( t, u, v ) t denotes time, ( u, v ) denotes pixel position of the camera screen ⁇ Equation 1
  • each of the subscripts i and j indicates their positions in spatiotemporal space. Furthermore, the subscript i is used to indicate the pixel of interest. In the description below, the subscripts will be described as pixel position i and pixel position j.
  • a correction amount for each distance is required as described above.
  • the correction amount for the frequency f L may be calculated in advance.
  • the correction amount is known for the data measured using the frequency f L , the data can be corrected, and correct distance measurement can be performed if the random noise component as described later is not taken into consideration.
  • Distance measurement data using frequency f L may include an error of the maximum error value e L in terms of distance.
  • the method as described later can be used.
  • the distance before correction D L(j) can be expressed using Equation 3.
  • D (j) is the true distance to the measurement object 100 measured at the pixel position j.
  • the distance before correction D′ H(j,n) can be expressed using Equation 5.
  • n represents an integer of 0 or more, which is an unknown value at this time.
  • D′ H(j,n) calculated using Equation 5 includes an error due to the fact that the time change of the light intensity of the irradiation light is not an ideal sine wave, so that the correction using the correction value is necessary.
  • the correction value E H (d) is known and is stored in, for example, the storage part 1g.
  • E H (D′ H(j,n) ) is a correction value when the distance measurement result before correction is D′ H(j,n) as described above.
  • Equation 7 holds between the distance after correction D H(j,n) calculated by Equation 6 and the true distance D (j) in a case where the random noise component as described later is not taken into consideration.
  • the unknown number n can be calculated by using Equation 3, Equation 4, Equation 5, Equation 6, and Equation 7. Assuming that the calculated unknown number n is n 0 , the distance after correction D H(j,n) is the corrected distance D H(j,n0) . D H(j) is defined by Equation 8 using this distance after correction.
  • Equation 9 holds.
  • Equations 3 and 4 are equations for frequency f L , the unknown number n can be determined by using distance measurement data using not only frequency f H but also frequency f L . That is, the indefiniteness of the unknown number n can be eliminated.
  • the correction amount in a case where distance measurement is performed using the frequency f L is E L (d).
  • d indicates the distance before correction.
  • the correction amount at the pixel position j is E L (D L(j) ).
  • E L (d) is an unknown function because the correction value is not measured in advance.
  • the true distance D (j) can be expressed using Equation 10.
  • D H(j) represents the distance after correction.
  • Equation 12 is obtained by expanding the unknown function E L (d) around D L(i) by Taylor series and approximating it with terms up to the second order.
  • E L ( d ) E 0 E 1 ⁇ ( d ⁇ D L(i) )+ E 2 ⁇ ( d ⁇ D L(i) ) 2 Equation 12
  • Equation 12 E 0 , E 1 , and E 2 in Equation 12 are unknown.
  • Equation 13 is obtained from Equation 10 and Equation 12.
  • D (j) D L(j) ⁇ ( E 0 +E 1 ⁇ ( D L(j) ⁇ D L(i) )+ E 2 ⁇ ( D L(j) ⁇ D L(i) ) 2 )+ R L(j) Equation 13
  • Equation 14 holds.
  • N represents a Gaussian symbol.
  • Equation 15 holds.
  • Equation 16 By finding the unknown numbers E 0 , E 1 , E 2 and D (j) such that the squared value of the error is the minimum value, the maximum likelihood value D (i) of the distance to the measurement object 100 measured by a pixel position of interest i can be calculated.
  • the equation to be solved is shown in Equation 16.
  • the pixel position j for calculating the distance measured at the pixel position of interest i is the pixel position of interest i, and is a pixel position different from the pixel position of interest i in spatiotemporal space.
  • the function E L (d) indicating the correction amount performs a quadratic approximation by Taylor series expansion as shown in Equation 12. Accordingly, it is preferable to use only the pixel positions within the range in which this quadratic approximation holds. That is, the pixel position j may not be any pixel position. That is, the pixel positions j to be added in Equation 16 are limited to those satisfy Equation 17.
  • the threshold Th is a small value.
  • the amount of calculation becomes enormous, and the convenience of the distance measuring apparatus 1 may be impaired. Therefore, it is desirable to add only the pixel positions in the vicinity of the pixel position of interest i as the pixel position j. That is, it is desirable that the pixel positions j to be added in
  • Equation 16 are “the pixel positions of interest i and the pixel positions in the vicinity of the pixel position of interest i” and satisfy Equation 17.
  • the operation processing part if of the distance measuring apparatus 1 selects a first frequency in step S 101 .
  • the first frequency and a second frequency are used, the first frequency is, for example, 40 MHz, which is higher than the second frequency.
  • step S 102 the operation processing part 1 f gives an instruction to emit the irradiation light of which intensity has been modulated using the selected frequency. According to this instruction, the light emitting part 1 b of the distance measuring apparatus 1 emits light.
  • step S 103 the operation processing part if gives an instruction to receive the reflected light in synchronization with the intensity modulation of the irradiation light. Therefore, the light receiving operation of the light receiving part 1 c of the distance measuring apparatus 1 is performed.
  • step S 104 the operation processing part if performs processing of calculating the phase difference for each pixel position.
  • the phase shift between the intensity change of the received reflected light and the intensity change of the light emission is calculated.
  • step S 105 the operation processing part if acquires the amplitude of the reflected light for each pixel position.
  • step S 101 By performing each processing from step S 101 to step S 105 ,
  • step S 106 the operation processing part 1 f performs processing of determining whether or not the second frequency different from the first frequency has been selected. In a case where only the first frequency has been selected and the second frequency has not been selected, the operation processing part 1 f performs processing of selecting the second frequency in step S 107 .
  • the second frequency is, for example, 10 MHz, which is lower than the first frequency.
  • the operation processing part 1 f After selecting the second frequency, the operation processing part 1 f performs various types of processing and instructions by performing processing from step S 102 to step S 105 .
  • step S 106 After performing processing from step S 102 to step S 106 using the first frequency and the second frequency, the operation processing part 1 f checks that the second frequency has been selected in step S 106 , and performs the processing of step S 108 .
  • step S 108 the unselected pixel position of each pixel of the light receiving sensor arranged in the two-dimensional array is selected as the pixel position of interest i. Subsequent each processing from step S 109 to step S 115 is processing to be performed for the pixel position of interest i selected here.
  • the operation processing part 1 f selects an unselected pixel position j in step S 109 .
  • step S 110 the operation processing part if calculates an unknown number n that satisfies Equations 3, 4, 5, 6, and 7 for the pixel position of interest i and pixel position j, and sets the calculated n as no.
  • the operation processing part 1 f calculates DH(j) using Equations 5, 6 and 8 in step S 111 , and calculates D L(j) using Equation 3 in step S 112 .
  • the operation processing part if determines in step S 113 whether or not all the pixel positions j have been selected. In a case where all the pixel positions j have not been selected for one pixel position of interest i, the operation processing part if selects the pixel position that has not been selected as the pixel position j in step S 109 as the pixel position J.
  • the operation processing part 1 f performs each processing from step S 110 to step S 112 for the newly selected pixel position j. That is, each processing from step S 110 to step S 112 is performed for all the pixel positions j.
  • step S 114 the operation processing part 1 f calculates E 0 , E 1 , E 2 and D (j) using Equation 16, and in step S 115 , performs processing of outputting D (i) from the output part 1h as the distance corresponding to the pixel position of interest i.
  • step S 116 the operation processing part 1 f determines whether or not there is a pixel position that has not been selected as the pixel position of interest i, that is, whether or not all the pixel positions have been selected as the pixel position of interest i.
  • step S 108 the operation processing part 1 f performs step S 108 again, selects a new pixel position as the pixel position of interest i, and performs each processing of step S 109 to step S 115 for the new pixel position of interest i.
  • the operation processing part 1 f can calculate and output the distance D (i) for all the pixel positions by perform each processing from step S 108 to step S 116 .
  • an arithmetic processing apparatus includes an operation processing part 1 f that performs processing of: calculating a first distance (D′ H(j,n) ) to a measurement object 100 by emitting and receiving first irradiation light (light of 40 MHz) of which intensity is modulated by a first modulation signal of a first frequency (for example, 40 MHz); calculating a second distance (D L(j) ) to the measurement object 100 by emitting and receiving second irradiation light (light of 10 MHz) of which intensity is modulated by a second modulation signal of a second frequency (for example, 10 MHz) different from the first frequency; calculating a corrected first distance (DH (j,n) ) using first correction data (correction value E H (D′ H(j,n) )that has been acquired; and determining the corrected first distance within an error range of the second distance as a third distance using the second distance (that is, specifying
  • correction data for the distance measurement result acquired by the measurement using one of the plurality of beams of light (that is, the first distance)
  • first correction data for the distance measurement result acquired by the measurement using one of the plurality of beams of light
  • the correction data is prepared in advance before the measurement, only the data for correcting the distance measurement result using one of the plurality of beams of light needs to be prepared, so that the inspection time for acquiring the correction data and the shipment inspection items can be reduced, which can contribute to cost reduction.
  • the measurement result using the light with the higher distance resolution (light of which intensity is modulated by the modulated signal of the higher frequency)
  • the distance measurement result narrowed down to one is regarded as the third distance. That is, it is possible to correctly derive the distance to the measurement object.
  • the first correction data may be data for correcting an error of a measurement distance caused by an error in the intensity change of the first irradiation light with respect to the sine wave of the first frequency (for example, 40 MHz).
  • the first distance is considered to include an error due to the fact that the intensity change of the first irradiation light does not form an accurate sine wave.
  • the first correction data is unique to each individual, according to the error between the first irradiation light and the accurate sine wave differs depending on the individual difference of the distance measuring apparatus. Furthermore, the first correction data is calculated in advance by the inspection performed at the time of manufacturing the distance measuring apparatus, and is referred to every time the distance measuring apparatus performs the distance measuring operation, so that it is not necessary to calculate the first correction data every time the distance measurement is performed, it is possible to quickly derive the corrected distance measurement result.
  • the first frequency (for example, 40 MHz) may be higher than the second frequency (for example, 10 MHz).
  • the distance measurement result using the first irradiation light in which the error for an accurate sine wave can be corrected is more suitable for measuring an accurate distance. Furthermore, since the second irradiation light has a lower frequency than the first irradiation light, the distance measuring range is excellent.
  • the third distance may be considered to include an error due to noise
  • the arithmetic processing apparatus may further include processing of: acquiring second correction data (E L (d)) for correcting an error of the second distance caused by an error in an intensity change of the second irradiation light with respect to a sine wave of the second frequency; defining a corrected second distance (D L(j) -E L (D L(j) ) obtained by correcting the second distance (D L(j) ) using the second correction data; defining a difference between the corrected first distance (D H(j) ) and the third distance (D H(j) +R H(j) ) as a first difference, defining a difference between the corrected second distance and the third distance (D L(j) ⁇ E L (D L(j) )+R L(j) ) as a second difference, and calculating an error due to the noise so that the first difference and the second difference become smaller (that is, the second correction data E L (d) for correcting an error due
  • the third distance includes not only an error that can be corrected by the first correction data, that is, an error due to the fact that the intensity change of the first irradiation light does not form an accurate sine wave, but also an error due to the noise component such as natural light entering the light receiving part during distance measurement (error due to noise).
  • the second correction data for correcting the error of the second distance due to the fact that the intensity change of the second irradiation light includes an error with respect to the sine wave is calculated, and moreover, an error due to noise is appropriately calculated from the difference of the third distance with each of the corrected first distance and the corrected second distance.
  • the second correction data (E L (d)) may be defined by a correction function (Equation 12) approximated by a quadratic Taylor series expansion.
  • the correction data is calculated in advance for only one of the first irradiation light and the second irradiation light, it is possible to shorten the manufacturing process and the inspection process.
  • the error of the third distance may be calculated so that the second difference becomes smaller for a neighboring pixel in spatiotemporal space with respect to a pixel of interest (pixel position of interest i).
  • each coefficient of the approximate expansion is calculated so that not only the first difference and the second difference of the pixel of interest, but also a first difference and a second difference of neighboring pixel in the spatiotemporal space of the pixel of interest become smaller.
  • the accuracy of the coefficient of the approximate expression can be improved, and the third distance as a distance measurement result in the pixel of interest can be corrected more accurately.
  • the neighboring pixel may be a pixel that has been extracted under a condition (Equation 17) that the difference between the second distance before correction for the pixel of interest and the second distance before correction for the neighboring pixel is equal to or less than a predetermined threshold.
  • the error of the third distance is calculated with higher accuracy, and the third distance can be corrected with higher accuracy.
  • the distance measuring apparatus 1 includes: a light emitting part 1 b capable of light emission of first irradiation light (light of 40 MHz) of which intensity is modulated by a first modulation signal of a first frequency (for example, 40 MHz), and light emission of second irradiation light (light of 10 MHz) of which intensity is modulated by a second modulation signal of a second frequency (for example, 10 MHz) different from the first frequency; a light receiving part 1 c that receives reflected light that is light emitted from the light emitting part 1 b and reflected by a measurement object 100 ; and an operation processing part 1 f that performs processing of calculating a first distance (D′ H(j,n) ) to a measurement object 100 by emitting and receiving first irradiation light, calculating a second distance (D L(j) ) to the measurement object 100 by emitting and receiving second irradiation light, calculating a corrected first distance (D H(j,n) )using first correction data
  • Pieces of irradiation light whose intensity is modulated at different frequencies can be emitted and each reflected light reflected by the measurement object can be received, and therefore, two pieces of distance measurement data with different distance resolution and distance measurement range can be acquired. It is possible to calculate the third distance as the measurement distance from these two distance measurement data.
  • the distance measuring apparatus 1 may include a storage part 1 g in which the first correction data is stored.
  • the corrected first distance can be calculated quickly, and the third distance can also be calculated quickly.
  • An arithmetic processing apparatus including an operation processing part that performs processing of:
  • the first correction data is data for correcting an error of a measurement distance caused by an error in an intensity change of the first irradiation light with respect to a sine wave of the first frequency.
  • the first frequency is set to be higher than the second frequency
  • the arithmetic processing apparatus further includes processing of:
  • the second correction data is defined by a correction function approximated by a second-order Taylor series expansion.
  • the error of the third distance is calculated so that the second difference becomes smaller for a neighboring pixel in spatiotemporal space with respect to a pixel of interest.
  • the neighboring pixel is a pixel that has been extracted under a condition that the difference between the second distance before correction for the pixel of interest and the second distance before correction for the neighboring pixel is equal to or less than a predetermined threshold.
  • a distance measuring apparatus including:
  • a light emitting part capable of light emission of first irradiation light of which intensity is modulated by a first modulation signal of a first frequency, and light emission of second irradiation light of which intensity is modulated by a second modulation signal of a second frequency different from the first frequency;
  • a light receiving part that receives reflected light that is light emitted from the light emitting part and reflected by a measurement object
  • an operation processing part that performs processing of calculating a first distance to a measurement object by emitting and receiving first irradiation light, calculating a second distance to the measurement object by emitting and receiving the second irradiation light, calculating a corrected first distance using first correction data that has been acquired, and determining the corrected first distance within an error range of the second distance as a third distance using the second distance.
  • the distance measuring apparatus according to (8), further including
  • An arithmetic processing method performed by an arithmetic processing apparatus including:

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Measurement Of Optical Distance (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
US17/286,685 2018-10-25 2019-07-23 Arithmetic processing apparatus, distance measuring apparatus, and arithmetic processing method Pending US20210349195A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2018200617A JP2020067385A (ja) 2018-10-25 2018-10-25 演算処理装置、測距装置及び演算処理方法
JP2018-200617 2018-10-25
PCT/JP2019/028872 WO2020084851A1 (fr) 2018-10-25 2019-07-23 Dispositif de traitement informatique, dispositif de télémétrie et procédé de traitement informatique

Publications (1)

Publication Number Publication Date
US20210349195A1 true US20210349195A1 (en) 2021-11-11

Family

ID=70331504

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/286,685 Pending US20210349195A1 (en) 2018-10-25 2019-07-23 Arithmetic processing apparatus, distance measuring apparatus, and arithmetic processing method

Country Status (4)

Country Link
US (1) US20210349195A1 (fr)
JP (1) JP2020067385A (fr)
CN (1) CN112771409B (fr)
WO (1) WO2020084851A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023003094A (ja) * 2021-06-23 2023-01-11 ソニーセミコンダクタソリューションズ株式会社 測距装置および方法、並びにプログラム
CN114859365A (zh) * 2022-07-05 2022-08-05 武汉市聚芯微电子有限责任公司 一种深度测量方法及装置、存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030223053A1 (en) * 2000-12-11 2003-12-04 Xingiao Liu Methods and devices for charge management for three-dimensional and color sensing
US20120013887A1 (en) * 2010-07-16 2012-01-19 Microsoft Corporation Method and system for multi-phase dynamic calibration of three-dimensional (3d) sensors in a time-of-flight system

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6052190A (en) * 1997-09-09 2000-04-18 Utoptics, Inc. Highly accurate three-dimensional surface digitizing system and methods
JP2006201037A (ja) * 2005-01-20 2006-08-03 Ricoh Co Ltd 形状測定装置
CN101806897B (zh) * 2009-02-17 2012-05-30 南京德朔实业有限公司 光电测距方法及其装置
WO2013129387A1 (fr) * 2012-03-01 2013-09-06 日産自動車株式会社 Dispositif pour trouver une plage et procédé pour trouver une plage
WO2014177750A1 (fr) * 2013-04-29 2014-11-06 Nokia Corporation Procédé et appareil pour fusionner des données de distance provenant d'une caméra de détection de distance avec une image
US10598783B2 (en) * 2016-07-07 2020-03-24 Microsoft Technology Licensing, Llc Multi-frequency unwrapping
US10425628B2 (en) * 2017-02-01 2019-09-24 Microsoft Technology Licensing, Llc Alternating frequency captures for time of flight depth sensing
CN106707290A (zh) * 2017-03-08 2017-05-24 深圳市芯盛传感科技有限公司 一种光学测距模组

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030223053A1 (en) * 2000-12-11 2003-12-04 Xingiao Liu Methods and devices for charge management for three-dimensional and color sensing
US20120013887A1 (en) * 2010-07-16 2012-01-19 Microsoft Corporation Method and system for multi-phase dynamic calibration of three-dimensional (3d) sensors in a time-of-flight system
EP2593810B1 (fr) * 2010-07-16 2016-03-23 Microsoft Technology Licensing, LLC Procédé et système d'étalonnage dynamique multiphase de capteurs tridimensionnels (3d) dans un système de temps de vol

Also Published As

Publication number Publication date
WO2020084851A1 (fr) 2020-04-30
JP2020067385A (ja) 2020-04-30
CN112771409B (zh) 2024-09-20
CN112771409A (zh) 2021-05-07

Similar Documents

Publication Publication Date Title
US9557167B2 (en) Three-dimensional-shape measurement apparatus, three-dimensional-shape measurement method, and non-transitory computer-readable storage medium
US10901090B2 (en) TOF camera system and a method for measuring a distance with the system
US9194953B2 (en) 3D time-of-light camera and method
JP5019117B2 (ja) 距離画像生成装置
US11536814B2 (en) Distance measuring apparatus having distance correction function
US20210349195A1 (en) Arithmetic processing apparatus, distance measuring apparatus, and arithmetic processing method
US20130321584A1 (en) Depth image generating method and apparatus and depth image processing method and apparatus
US9900581B2 (en) Parametric online calibration and compensation in ToF imaging
CN103998949A (zh) 对飞行时间信号处理的改进或与之相关的改进
EP2477043A1 (fr) Caméra temps de vol 3D et procédé
EP3129803B1 (fr) Procédé et appareil d'annulation d'erreurs d'harmoniques de signaux
US11047962B2 (en) Method for binning time-of-flight data
US20210263137A1 (en) Phase noise and methods of correction in multi-frequency mode lidar
EP3842824A3 (fr) Procédé et dispositif pour traiter des signaux radar
US8665269B2 (en) Method of determining features of events or processes having a dynamic evolution in space and/or time
CN112945221A (zh) 相位角校正值计算装置和计算相位角校正值的方法
CN114450711A (zh) 工件的表面缺陷检测装置及检测方法、工件的表面检查系统以及程序
US20200033119A1 (en) Inter-reflection detection apparatus and inter-reflection detection method
US11353565B2 (en) Time-of-flight imaging apparatus and method
CN105005043A (zh) 用于在目标空间中检测对象的运动的装置和方法
JP2020193957A (ja) 測距異常を補正する距離画像生成装置
KR101192994B1 (ko) 거리 측정을 위한 레이저 스캐너 제어 장치 및 그 제어 방법
JPWO2020174567A1 (ja) 測距装置及び測距方法
US20220035039A1 (en) Tof camera
US11630190B2 (en) Electronic device, method and computer program

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY SEMICONDUCTOR SOLUTIONS CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OHKI, MITSUHARU;REEL/FRAME:056387/0660

Effective date: 20210525

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

ZAAB Notice of allowance mailed

Free format text: ORIGINAL CODE: MN/=.