JP2024079416A - Information processing device, information processing method, and program - Google Patents

Abstract

[Problem] To provide an information processing device capable of achieving both high distance measurement speed and high distance measurement accuracy when measuring a distance to a subject using pattern light. [Solution] A luminance evaluation value is calculated based on first image information obtained by irradiating a subject with pattern light having predetermined characteristics and acquiring reflected light, and a distance measurement means is controlled to calculate the distance to the subject in a predetermined distance measurement mode based on the characteristic information of the pattern light, the predetermined distance measurement mode including a first distance measurement mode that calculates the distance based on the first image information and a second distance measurement mode that calculates the distance based on a difference image generated based on the first image information and second image information acquired by the imaging means in a state where the pattern light is not irradiated, and the distance measurement means is controlled as to whether to calculate the distance in the first distance measurement mode or the second distance measurement mode based on the luminance evaluation value. [Selected Figure] Figure 1

Description

The present invention relates to a distance measuring system that uses pattern light to measure the subject distance.

In recent years, ranging technology that measures the distance to a subject has become widely used in fields such as in-vehicle sensing, autonomous mobile robots, unmanned guided vehicles, and unmanned aerial vehicles. A representative example of ranging technology is stereo ranging, in which the same subject is captured by multiple cameras and distance information to the subject is calculated from the parallax of the captured images. This method has the advantage of not requiring a special light source, but has the problem that accurate distance information cannot be obtained for subjects without texture.

To address the above issues, a distance measurement method called the pattern light illumination method has been proposed. This method calculates the distance to a subject from the characteristics of the pattern light when a specific pattern of light is illuminated on the subject. Using this method, it is possible to perform highly accurate distance measurements even for subjects without texture.

However, with the pattern light illumination method, if strong disturbance light other than the illuminated pattern light is illuminated on the subject, the pattern light becomes buried in the disturbance light, making it difficult to detect the characteristics of the pattern light. Therefore, Patent Document 1 proposes a technology that uses a difference image between when the pattern light is illuminated and when it is not illuminated to extract the components of only the pattern light, thereby improving the accuracy of measuring the subject distance.

JP 2022-53871 A

However, as in the technology described in Patent Document 1, if a difference image between when pattern light is irradiated and when it is not irradiated is generated every time distance measurement information is acquired, the processing load of distance measurement calculations increases. In addition, distance measurement information can only be acquired once every two frames, which reduces the time resolution (response speed) in distance measurement.

The object of the present invention is to provide an information processing device that can achieve both high distance measurement speed and high distance measurement accuracy when measuring the subject distance using pattern light.

In order to achieve the above object, the present invention relates to a method for manufacturing a semiconductor device comprising the steps of:
a calculation means for calculating a luminance evaluation value indicating an intensity of disturbance light based on first image information obtained by an imaging means of reflected light when a pattern light having a predetermined characteristic is irradiated onto a subject by a pattern light irradiating means;
a control means for controlling the distance measuring means so as to calculate the distance to the subject in a predetermined distance measuring mode based on the characteristic information of the pattern light,
The predetermined distance measurement mode is
a first distance measurement mode in which the distance is calculated based on the first image information;
a second distance measurement mode in which the distance is calculated based on a difference image generated based on the first image information and second image information acquired by the imaging means in a state in which the pattern light is not irradiated,
The control means controls the distance measurement means as to whether to calculate the distance in a first distance measurement mode or in a second distance measurement mode based on the brightness evaluation value calculated by the calculation means.

According to the present invention, it is possible to achieve both high distance measurement speed and high distance measurement accuracy when measuring the subject distance using pattern light.

1 is a configuration diagram of a distance measuring system according to a first embodiment. 5 is an explanatory diagram of a control method for a high-speed distance measurement mode according to the first embodiment. FIG. 5 is an explanatory diagram of a control method for a high-precision distance measurement mode in the first embodiment. FIG. 4 is a flowchart of a distance measurement mode switching method according to the first embodiment. 5 is an explanatory diagram of a process of a luminance evaluation value calculation unit according to the first embodiment. FIG. FIG. 11 is an explanatory diagram of a driving control method according to a second embodiment.

First Embodiment
Hereinafter, a first embodiment of the present invention will be described.

Figure 1 is a configuration diagram of a distance measurement system in embodiment 1. The distance measurement system shown in Figure 1 is composed of a pattern light irradiation unit 101, an image capture unit 102, a brightness evaluation value calculation unit 103, a distance measurement unit 104, a memory unit 105, a driving control unit 106, and a system control unit 107.

The pattern light irradiation unit 101 is a means for irradiating the subject with pattern light, and is composed of a light source unit 108 that emits light, and a pattern generation unit 109 that generates a specific pattern shape from the light emitted from the light source. The light source unit 108 emits light with a wavelength in the infrared range. For example, an infrared LED or the like is used. This embodiment will be described on the premise of infrared light, but it is not necessarily infrared light, and light in other wavelength ranges may be used. The pattern generation unit 109 is for generating pattern light by changing the transmittance of the infrared light emitted by the light source unit 108 for each region. For example, there is a method of spatially branching light using a diffractive optical element, a method of physically inserting an ND filter with a different transmittance for each region, and a method of electrically changing the transmittance for each region using a liquid crystal projector. The above-mentioned method is not necessarily required, and other methods may be used as long as the same effect can be obtained. In this embodiment, a means capable of generating at least pattern light is used. The pattern light irradiation unit 101 controls switching between irradiation and non-irradiation of pattern light according to a light emission control signal transmitted from the system control unit 107. The light emission control method in this embodiment will be described in detail later.

The imaging unit 102 is composed of a lens 110, a photoelectric conversion unit 111, and an A/D conversion unit 112.

The lens 110 is for forming an image of the light reflected by the subject from the light irradiated by the pattern light irradiation unit 101. The optical image formed by the lens 110 is converted into an analog electric signal by the photoelectric conversion unit 111 (photodiode). The photoelectric conversion unit 111 in this embodiment has sensitivity to at least infrared light. It may be configured to generate both color images and infrared images using four types of filters, R (red), G (green), B (blue), and IR (infrared), or may be configured to generate only infrared images using only an IR filter. Alternatively, it may be configured to generate only monochrome images without using any filters. The analog electric signal converted by the photoelectric conversion unit 111 is converted into a digital image signal by the A/D conversion unit 112. The digital image signal generated by the A/D conversion unit 112 is output to the brightness evaluation value calculation unit 103. The imaging unit 102 also controls the exposure time according to an exposure control signal transmitted from the system control unit 107. Details of the exposure control method in this embodiment will be described later.

The brightness evaluation value calculation unit 103 receives the digital image signal output by the imaging unit 102, and calculates a brightness accumulation value in a predetermined area of the image signal. The brightness accumulation value is used to determine the intensity of ambient light, and the system control unit 107 switches the distance measurement mode to the optimal mode based on the calculation result of the brightness accumulation value. The distance measurement mode switching method will be described in detail later. In addition, although the brightness accumulation value is used in this embodiment, other indices such as the average brightness value or a histogram may be used as the brightness evaluation value. After the brightness evaluation value calculation unit 103 calculates the brightness accumulation value, the image signal is output to the distance measurement unit 104.

The distance measuring unit 104 measures the subject distance based on the luminance information of the image signal output by the luminance evaluation value calculation unit 103. Specifically, the distance to the subject can be measured by comparing the characteristics of the pattern light in the image signal output from the imaging unit 102 with the characteristics of the pattern light irradiated by the pattern light irradiation unit 101 and performing distance measurement calculation processing. This subject distance measurement method can be realized by using the existing pattern light irradiation method.

As an example of an existing pattern light irradiation method, a case where a coded pattern is used will be described. The pattern light irradiation unit 101 is provided with a two-dimensional pattern in which the horizontal and vertical coordinates of the irradiated pattern light are coded as unique markers, and the two-dimensional pattern is used to irradiate the subject with light. The distance measurement unit 104 performs feature point detection processing, viewpoint conversion processing, pattern matching processing, etc. for each area in the image acquired by the imaging unit 102 to detect markers present in each area. Then, by associating the positions of the markers of the irradiated pattern light with the positions of the markers present in the acquired image, it is possible to calculate the distance to the subject based on the principle of triangulation. Then, the distance calculation result can be subjected to smoothing processing, outlier removal processing, etc., to interpolate missing distance information and improve accuracy. In this embodiment, a general distance measurement calculation method for the pattern light irradiation method, including the method using the above-mentioned coded pattern, is used.

The storage unit 105 is a means for storing the image signal output from the imaging unit 102, the luminance integration result calculated by the luminance evaluation value calculation unit 103, the distance measurement calculation result calculated by the distance measurement unit 104, etc. It can also store a threshold value for determining whether to switch distance measurement modes in the luminance evaluation value calculation unit 103, and various programs for controlling this distance measurement system.

The driving control unit 106 is a means for controlling the driving of a mobile object equipped with this distance measurement system. Specifically, it is possible to control the start and stop of driving, changes in driving speed, turning, etc. The driving control method in this embodiment will be described in detail later.

The system control unit 107 controls the entire distance measurement system. Specifically, it controls the light emission of the pattern light irradiation unit 101, the exposure of the image capture unit 102, and performs brightness integration processing using the brightness evaluation value calculation unit 103. It also performs distance measurement calculation processing using the distance measurement unit 104, writing and reading image signals and various information to and from the memory unit 105, and driving control of the driving control unit 106. Details will be described later, but this distance measurement system has two distance measurement modes, and performs various processes required for system control, such as a determination process for switching the distance measurement mode and switching the light emission control method and driving control method associated with switching the distance measurement mode.

The above is the configuration of the distance measurement system in embodiment 1.

The details of the distance measurement modes in this embodiment are explained below. This distance measurement system has two distance measurement modes: a high-speed distance measurement mode and a high-precision distance measurement mode.

<High-speed distance measurement mode>
The high-speed distance measurement mode is a distance measurement mode in which distance measurement speed takes precedence over distance measurement accuracy, and is used when the influence of ambient light is small and distance measurement accuracy is sufficient, etc. A specific control method of the high-speed distance measurement mode will be described with reference to FIG.

Figure 2(a) is a graph showing the light irradiation state of the pattern light irradiation unit 101, and the exposure period and image output period of the imaging unit 102. In Figure 2(a), the horizontal axis shows time. The vertical axis shows the light irradiation state, VD (Vertical Driving Pulse), exposure period, and image output period.

The light irradiation state indicates the irradiation state of the pattern light irradiated by the pattern light irradiation unit 101. The period indicated by the hatched line indicates the state in which the pattern light is irradiated, and the period indicated by the dashed line without hatching indicates the state in which the pattern light is not irradiated.

VD indicates the vertical synchronization signal of the imaging unit 102, and processing to acquire a new image signal begins when it switches from Low to High. In this embodiment, it indicates the VD from the Nth frame to the N+2th frame, that is, a period of three frames.

The exposure period refers to the period during which the imaging unit 102 exposes the subject to light. When the exposure period is Low, the photoelectric conversion unit 111 is in a reset state, and the subject is not exposed to light. When the exposure period is High, the photoelectric conversion unit 111 is released from reset, and the subject is exposed to light.

The image output period is a period during which the signal charge accumulated in the photoelectric conversion unit 111 in the imaging unit 102 in the previous frame is converted into an analog signal, and then converted into a digital signal, and the imaging unit 102 sequentially outputs the image signal to the brightness evaluation value calculation unit 103. In this embodiment, this process is executed in synchronization with the timing when VD changes from Low to High.

When the ranging system performs subject ranging in this embodiment, the system control unit 107 controls the timing so that the period during which the light irradiation state is in an irradiated state and the period during which the exposure period is High overlap. By doing so, the reflected light from the subject of the pattern light irradiated by the pattern light irradiation unit 101 can be reliably acquired as image information by the imaging unit 102.

In FIG. 2A, at timing t21, the light irradiation state of the pattern light irradiation unit 101 is switched from an irradiation state to a non-irradiation state. In addition, a VD is sent from the system control unit 107 to the imaging unit 102, and processing to acquire a new image signal is started within the imaging unit 102. At the same time, an exposure control signal is sent from the system control unit 107 to the imaging unit 102, and exposure of the subject is completed. The exposure control signal is specifically a reset signal for discharging all charges accumulated in the photoelectric conversion unit 111 within the imaging unit 102. By resetting the photoelectric conversion unit 111, all charges accumulated in the photoelectric conversion unit 111 until the exposure start period of the new frame are swept out.

At t22, the system control unit 107 sends a light emission control signal to the patterned light irradiation unit 101, and the light irradiation state of the patterned light irradiation unit 101 is switched from a non-irradiation state to an irradiation state. At the same time, the system control unit 107 sends an exposure control signal to the imaging unit 102, and the reset state of the photoelectric conversion unit 111 is released, thereby starting exposure. In other words, only the light reflected from the subject after t22 is exposed.

During the period from t22 to t23, light is emitted from the pattern light irradiation unit 101 and the imaging unit 102 is exposed to light.

At t23, the system control unit 107 transmits a light emission control signal to the patterned light irradiation unit 101, and the light irradiation state of the patterned light irradiation unit 101 is switched from an irradiation state to a non-irradiation state. At the same time, the system control unit 107 transmits a VD to the imaging unit 102, and an image output period is started in which the charge accumulated in the photoelectric conversion unit 111 in the imaging unit 102 is read, A/D conversion processing is performed, and an image signal is output. Then, an exposure control signal is transmitted from the system control unit 107 to the imaging unit 102, and the exposure period is ended.

During the period from t23 to t24, the image signals exposed during the period from t22 to t23 are sequentially output from the imaging unit 102 to the brightness evaluation value calculation unit 103.

At t24, the output of the image signal from the imaging unit 102 to the brightness evaluation value calculation unit 103 is completed.

As shown in FIG. 2(a), by synchronizing the timing at which the pattern light irradiation unit 101 irradiates the pattern light with the timing at which the imaging unit 102 performs exposure, it is possible to reliably obtain an image signal in a state in which the pattern light is irradiated onto the subject. In this embodiment, the period during which the pattern light is irradiated and the exposure period are set to be exactly the same period, but they do not necessarily have to be the same period, and it is sufficient if they include a state in which the pattern light is irradiated at any timing during the exposure period.

Figure 2(b) shows a subject image exposed during the period from t22 to t23 of the Nth frame, and output by the imaging unit 102 to the brightness evaluation value calculation unit 103 during the period from t23 to t24 of the N+1th frame. The subject image acquired when no pattern light is irradiated by the pattern light irradiation unit 101 is as shown in Figure 2(c). By performing the timing control of Figure 2(a), it is possible to reliably acquire a subject image on which dot-shaped pattern light is superimposed, as shown in Figure 2(b). As a result, it becomes possible to reliably acquire the measurement results of the subject distance for each frame.

This concludes the explanation of the high-speed distance measurement mode in this embodiment.

<High-precision distance measurement mode>
The high-precision distance measurement mode prioritizes distance measurement accuracy over distance measurement speed, and is used when the influence of ambient light is large and distance measurement accuracy is insufficient, etc. A specific control method in the high-precision distance measurement mode will be described with reference to FIG.

Similar to FIG. 2(a), FIG. 3(a) is a graph showing the light irradiation state of the pattern light irradiation unit 101, and the exposure period and image output period of the imaging unit 102. The items in the graph shown in FIG. 3(a) are the same as those in FIG. 2(a), so their explanation will be omitted.

In FIG. 3(a), the control method of the imaging unit 102, specifically, the timing control of the VD, exposure period, and image output period, is the same as in FIG. 2(a), so the description will be omitted. FIG. 3(a) differs from FIG. 2(a) in the light irradiation state of the pattern light irradiation unit 101. In FIG. 2(a), pattern light is irradiated every frame using the pattern light irradiation unit 101, such as the Nth frame, the N+1th frame, the N+2nd frame, and so on. In contrast, in FIG. 3(a), pattern light is irradiated every other frame, such as the Nth frame, the N+2nd frame, the N+4th frame, and so on. In other words, exposure is performed in a state where the N+1st frame, the N+3rd frame, the N+5th frame, and so on are not irradiated with pattern light.

Figure 3(b) shows a subject image exposed during the period from t32 to t33 of the Nth frame, and output by the imaging unit 102 to the brightness evaluation value calculation unit 103 during the period from t33 to t35 of the N+1th frame. The high-precision distance measurement mode is used in anticipation of a situation in which distance measurement accuracy is insufficient due to the effects of disturbance light, so an image is shown in which the subject is illuminated by both disturbance light and pattern light.

Figure 3(c) shows a subject image exposed during the period from t34 to t36 of the N+1th frame and output from the imaging unit 102 to the brightness evaluation value calculation unit 103 during the period from t36 to t37 of the N+2th frame. This image was acquired without irradiation with pattern light, and is therefore an image in which only ambient light is irradiated onto the subject.

The purpose of alternately acquiring these two images is to extract only the pattern light components by generating the difference between Figure 3(b) and Figure 3(c) for each pixel. Figure 3(d) is the difference image between Figure 3(b) and Figure 3(c). As shown in Figure 3(d), by removing the disturbance light components of Figure 3(c) from the image of Figure 3(b), it is possible to extract only the pattern light components. In the image of Figure 3(b), the pattern light is buried in the strong disturbance light, making it difficult to detect the characteristics of the pattern light. However, by generating the difference image of Figure 3(d) using the image acquired when only the disturbance light of Figure 3(c) is irradiated, it is possible to detect the characteristics of the pattern light with high accuracy.

This concludes the explanation of the high-precision distance measurement mode in this embodiment.

As explained above, when the high-speed distance measurement mode is used, distance measurement information can be acquired for each frame, allowing distance measurement to be performed at high speed. However, when strong disturbance light is irradiated on the subject, the distance measurement accuracy decreases. On the other hand, the high-precision distance measurement mode allows the characteristics of the pattern light to be acquired with high precision even when strong disturbance light is irradiated on the subject. However, since a difference image must be generated from an image with the pattern light irradiated and an image with the pattern light not irradiated, the load of image processing increases. In addition, the frequency of acquiring distance measurement information decreases, causing the response speed of distance measurement to decrease. In this embodiment, distance measurement control is performed to optimize distance measurement speed and distance measurement accuracy by appropriately switching the distance measurement mode according to the intensity of disturbance light.

The method for switching distance measurement modes in this embodiment is explained using Figure 4.

Figure 4(a) is a flowchart showing the distance measurement control method in this embodiment. This flowchart is executed by a CPU (not shown) in the system control unit 107 reading a program corresponding to the processing content from a ROM (not shown) and loading it into the storage unit 105.

In FIG. 4A, in S4a01, the distance measurement mode is initially set. In this embodiment, the high-speed distance measurement mode is set as the initial setting for the distance measurement mode. After the high-precision distance measurement mode is set in this flow, the process proceeds to S4a02.

In S4a02, a subject image is acquired. Specifically, an image signal is output from the imaging unit 102 to the brightness evaluation value calculation unit 103 using the control method described in FIG. 2(a) and FIG. 3(a). After the image signal is acquired in this flow, the process proceeds to S4a03.

In S4a03, the luminance evaluation value calculation unit 103 is used to calculate the luminance integrated value. Specifically, as shown in FIG. 5, a region to be measured is specified in the image signal output from the imaging unit 102, and the luminance values of all pixels present in the specified region are integrated. When specifying the luminance integrated region, it may be limited to a field of view region in which distance information is required when a mobile object equipped with this distance measurement system travels, or the entire pixel region may be specified. Alternatively, it may be limited to a range in which pattern light can be irradiated by the pattern light irradiation unit 101. In this flow, after the luminance integrated value is calculated, the flow proceeds to S4a04.

In S4a04, the system control unit 107 determines whether the luminance integrated value calculated in the flow of S4a03 is equal to or greater than a threshold value Th. If the luminance integrated value is equal to or greater than the threshold value Th, it is determined that the influence of disturbance light is large and the distance measurement accuracy is insufficient, and if the luminance integrated value is less than the threshold value Th, it is determined that the influence of disturbance light is small and the distance measurement accuracy is sufficient.

Here, an example of how to determine the threshold Th is explained. If you want to prioritize distance measurement accuracy over distance measurement speed and minimize degradation of distance measurement accuracy due to ambient light, set the threshold Th low. Specifically, you can set the threshold Th to a value close to the integrated brightness value when the brightness values generated by irradiating the pattern light are integrated over a specified area. Since the size of the dot pattern changes depending on the subject distance, you can calculate the threshold Th assuming the brightness value when the subject distance is close and the dot pattern is at its maximum. By doing so, if even a small amount of ambient light other than the pattern light is irradiated onto the subject, you can immediately switch to high-precision distance measurement mode.

On the other hand, if you want to prioritize distance measurement speed over distance measurement accuracy, set the threshold value Th higher. This can be set by adding the increase in brightness due to disturbance light to an acceptable range to the integrated brightness value obtained by integrating the brightness values generated by irradiating the aforementioned pattern light for a specified area. Since the acceptable intensity of disturbance light also changes depending on the detection accuracy of the distance measurement algorithm of the distance measurement unit 104, it is preferable to determine the threshold value taking into consideration the detection accuracy of the distance measurement algorithm. Setting the threshold value Th higher may result in detection errors, but in cases where, for example, not being able to obtain distance measurement information once every several frames does not affect the control of the moving object, the threshold value Th can be set higher.

The threshold Th may also be changed depending on the operation mode of the mobile body equipped with the ranging system. For example, assume that the mobile body in this embodiment is an automatic delivery robot, and the operation mode is "urgent mode" during delivery and "normal mode" after delivery. Since it is necessary to obtain ranging information as quickly as possible during delivery, it is desirable to perform ranging in the fastest possible ranging mode. Therefore, the threshold Th for changing to the high-precision ranging mode is set high. On the other hand, since there is more time after delivery, the threshold Th for changing to the high-precision ranging mode is set low. By performing such control, it is possible to optimize which of ranging speed and ranging accuracy is prioritized depending on the purpose of the mobile body.

The threshold Th may also be changed according to the emission intensity of the pattern light emitted by the pattern light irradiation unit 101. For example, as shown in FIG. 5(b), the threshold Th may be determined according to the magnitude of the drive current flowing through the light source unit 108 in the pattern light irradiation unit 101. When the emission intensity of the pattern light is strong, the detection accuracy is unlikely to decrease even if some disturbance light is superimposed, so the threshold Th is set high. By performing such control, it is possible to switch to the optimal distance measurement mode according to the distance measurement accuracy based on the intensity of the pattern light.

Returning to the flowchart of FIG. 4(a), in the flow of S4a04, if it is determined that the luminance integrated value calculated in the flow of S4a03 is equal to or greater than the threshold value Th, proceed to S4a05, and if it is not determined to be equal to or greater than the threshold value Th, return to S4a02.

In S4a05, the distance measurement mode is switched from the high-speed distance measurement mode to the high-precision distance measurement mode. Specifically, the method of controlling the light irradiation state by the pattern light irradiation unit 101 is switched from FIG. 2(a) to FIG. 3(a). Also, when performing distance measurement calculation using the distance measurement unit 104, the method is switched from using the image of FIG. 2(b) output from the brightness evaluation value calculation unit 103 to using the difference image of FIG. 3(d) generated by the system control unit 107.

Next, the distance measurement control method when switching the distance measurement mode from the high-precision distance measurement mode to the high-speed distance measurement mode will be explained using the flowchart in FIG. 4(b).

S4b01 and S4b02 in FIG. 4(b) correspond to S4a02 and S4a03 in FIG. 4(a), respectively, and perform similar processing, so explanation is omitted.

In S4b03, the system control unit 107 determines whether the luminance integrated value calculated in the flow of S4b02 is less than the threshold value Th. If the luminance integrated value is less than the threshold value Th, it is determined that the disturbance light is small and the distance measurement accuracy is sufficient even in the high-speed distance measurement mode, and if the luminance integrated value is not less than the threshold value Th, it is determined that the disturbance light is large and the distance measurement accuracy is low, so that the high-precision distance measurement mode needs to be continued.

In this embodiment, the setting of the threshold Th has been described under the assumption that the judgment threshold is the same when switching from the high-speed ranging mode to the high-precision ranging mode and when switching from the high-precision ranging mode to the high-speed ranging mode. However, a different threshold Th may be set for each state of the current ranging mode. For example, the high-precision ranging mode uses a differential image to suppress the effects of ambient light, so that only the pattern light components can be extracted with high precision. Therefore, for the purpose of power saving, etc., the emission intensity of the pattern light and the detection threshold of the pattern light can be set low. In other words, the threshold Th can be set lower than when switching from the high-speed ranging mode to the high-precision ranging mode. In that case, it is desirable to set the threshold Th for switching the ranging mode low as well. By performing such control, the ranging mode can be appropriately switched even if the luminance integrated value changes in conjunction with the ranging mode.

Returning to the flowchart of FIG. 4B, in the flow of S4b03, if it is determined that the luminance integrated value calculated in the flow of S4b02 is less than the threshold value Th, proceed to S4b04, and if it is not determined that it is less than the threshold value Th, return to S4b01.

In S4b04, the distance measurement mode is switched from the high-precision distance measurement mode to the high-speed distance measurement mode. Specifically, the control method of the pattern light irradiation unit 101 is switched from FIG. 3(a) to FIG. 2(a). Also, when performing distance measurement calculation using the distance measurement unit 104, the method is switched from using the difference image of FIG. 3(d) generated by the system control unit 107 to using the image of FIG. 2(b) output from the brightness evaluation value calculation unit 103.

As explained above, when the effect of ambient light on distance measurement accuracy is small and the distance measurement accuracy is sufficient, distance measurement is performed in high-speed distance measurement mode, and only when the effect of ambient light on distance measurement accuracy is large and accurate distance measurement is no longer possible, the mode is switched to high-precision distance measurement mode. This allows the distance measurement speed and accuracy to be optimized.

Second Embodiment
A second embodiment of the present invention will now be described.

In the second embodiment, the travel control method of the moving body is switched depending on the distance measurement mode. Specifically, in the high-speed distance measurement mode, no restrictions are imposed on the travel control method, and in the high-precision distance measurement mode, restrictions are imposed on the travel speed, turning, and other operations. The purpose is to suppress the occurrence of brightness differences in the difference image when generating a difference image in the high-precision distance measurement mode, which is caused by differences in the subject position between the image of disturbance light when pattern light is irradiated and the image of disturbance light when pattern light is not irradiated. Ideally, it is desirable that the position of the subject affected by disturbance light be consistent between the two frames for generating the difference image.

Figure 6 is a graph explaining the specific driving control method in the high-precision distance measurement mode in this embodiment. The high-speed distance measurement mode is controlled in the same way as in embodiment 1, so the explanation is omitted.

Figure 6 is a graph showing the light irradiation state of the pattern light irradiation unit 101, the control method of the imaging unit 102 (VD, exposure period, image output period), and the travel control method of the moving object. The items in the graph other than the travel control method are the same as those in Figures 2(a) and 3(a), so their explanations are omitted.

In FIG. 6, driving control is performed normally during the period from t61 to t63, i.e., the Nth frame period. Driving control is then restricted during the period from t63 to t65, i.e., the N+1th frame period. Specifically, high-speed driving and turning that would change the angle of view are not performed. Alternatively, restrictions are placed on the driving speed and turning speed. As mentioned above, if there is a significant change between the subject image exposed during the period from t62 to t63 of the Nth frame and the subject image exposed during the period from t64 to t65, a luminance difference other than that of the pattern light will occur when generating the difference image. To suppress this, driving control is performed to prevent the angle of view from changing as much as possible.

After the N+2 frame, normal driving is performed in the N+2, N+4, N+6, etc. frames, but high-speed driving and turning are prohibited or restricted in the N+3, N+5, N+7, etc. frames. By performing this type of control, it is possible to detect the components of only the pattern light with high accuracy, even when the influence of external light is large.

As described above, in frames where ambient light is captured in high-precision distance measurement mode, by restricting or prohibiting driving that changes the angle of view, such as high-speed driving or turning, it is possible to detect pattern light with high accuracy.

As described above in the first and second embodiments, the priority of distance measurement speed and distance measurement accuracy can be switched depending on the intensity of ambient light, thereby optimizing the distance measurement speed and distance measurement accuracy. Furthermore, when distance measurement accuracy is prioritized, driving control is performed so that the angle of view does not change, making it possible to measure the subject distance with high accuracy.

The present invention has been described in detail above based on preferred embodiments, but the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the invention are also included in the present invention. Parts of the above-mentioned embodiments may be combined as appropriate.

The present invention also includes cases where a software program that realizes the functions of the above-mentioned embodiments is supplied to a system or device having a computer capable of executing the program directly from a recording medium or via wired/wireless communication, and the program is executed.

Therefore, the program code that is supplied to and installed on a computer to realize the functional processing of the present invention also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

In this case, as long as it has the functionality of a program, the form of the program is not important, such as object code, a program executed by an interpreter, or script data supplied to the OS.

The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical/magnetic optical storage medium, or a non-volatile semiconductor memory.

Another possible method of supplying the program is to store the computer program forming the present invention on a server on a computer network, and have connected client computers download and program the computer program.

<Configuration 1>
a luminance evaluation value calculation means for calculating a luminance evaluation value indicating an intensity of disturbance light based on first image information obtained by an imaging means of reflected light when a pattern light having a predetermined characteristic is irradiated by a pattern light irradiation means onto a subject;
a control means for controlling the distance measuring means so as to calculate the distance to the subject in a predetermined distance measuring mode based on the characteristic information of the pattern light,
The predetermined distance measurement mode is
a first distance measurement mode in which the distance is calculated based on the first image information;
a second distance measurement mode in which the distance is calculated based on a difference image generated based on the first image information and second image information acquired by the imaging means in a state in which the pattern light is not irradiated,
The information processing device characterized in that the control means controls the distance measurement means as to whether to calculate the distance using a first distance measurement mode or a second distance measurement mode based on the luminance evaluation value calculated by the luminance evaluation value calculation means.

<Configuration 2>
The control means controls the distance measuring means to calculate the distance in a first distance measuring mode when the brightness evaluation value calculated by the brightness evaluation value calculation means is less than a predetermined threshold, and controls the distance measuring means to calculate the distance in a second distance measuring mode when the brightness evaluation value calculated by the brightness evaluation value calculation means is equal to or greater than the predetermined threshold, and the control means determines the predetermined threshold in accordance with the light emission intensity of the pattern light irradiation means.

<Configuration 3>
An information processing device mounted on a moving object,
2. The information processing apparatus according to configuration 1, wherein the control means determines the predetermined threshold value in accordance with an operation mode of the moving object.

<Configuration 4>
4. The information processing apparatus according to configuration 3, wherein the operation modes of the moving body include a first operation mode in which priority is given to distance measurement speed, and a second operation mode in which priority is given to distance measurement accuracy.

<Configuration 5>
The information processing device described in configuration 1, characterized in that, during execution of the second distance measurement mode, the control means determines the predetermined threshold value based on the intensity of the pattern light irradiated by the pattern light irradiating means and a setting of a threshold value for detecting the pattern light.

<Configuration 6>
a travel control means for starting and stopping the travel of the moving body, controlling the travel speed, and turning the moving body;
The information processing device described in configuration 1, characterized in that the driving control means suppresses the driving speed and turning of the moving body during a period from when an image is acquired while irradiating a pattern light by the light irradiation means while the second distance measurement mode is being executed and until an image is acquired while not irradiating the pattern light.

<Configuration 7>
2. The information processing apparatus according to configuration 1, wherein the luminance evaluation value includes an integrated luminance value and an average luminance value.

<Configuration 8>
a luminance evaluation value calculation step of calculating a luminance evaluation value indicating the intensity of disturbance light based on first image information acquired by an imaging step of reflected light when a pattern light having a predetermined characteristic is irradiated on a subject by the pattern light irradiation step;
a control step of controlling the distance measurement step so as to calculate the distance to the subject in a predetermined distance measurement mode based on the characteristic information of the pattern light,
The predetermined distance measurement mode is
a first distance measurement mode in which the distance is calculated based on the first image information;
a second distance measurement mode in which the distance is calculated based on a difference image generated based on the first image information and second image information acquired in the imaging step without irradiating the pattern light,
An information processing method characterized in that the control process controls the distance measurement process as to whether to calculate the distance using a first distance measurement mode or a second distance measurement mode based on the brightness evaluation value calculated by the brightness evaluation value calculation process.

<Configuration 9>
A computer program for controlling each means of the information processing device according to any one of configurations 1 to 7 by a computer.

REFERENCE SIGNS LIST 101 Pattern light irradiation unit 102 Imaging unit 103 Brightness evaluation value calculation unit 104 Distance measurement unit 105 Storage unit 106 Travel control unit 107 System control unit 108 Light source unit 109 Pattern generation unit 110 Lens 111 Photoelectric conversion unit 112 A/D conversion unit

Claims (9)

a calculation means for calculating a luminance evaluation value indicating an intensity of disturbance light based on first image information obtained by an imaging means of reflected light when a pattern light having a predetermined characteristic is irradiated onto a subject by a pattern light irradiating means;
a control means for controlling the distance measuring means so as to calculate the distance to the subject in a predetermined distance measuring mode based on the characteristic information of the pattern light,
The predetermined distance measurement mode is
a first distance measurement mode in which the distance is calculated based on the first image information;
a second distance measurement mode in which the distance is calculated based on a difference image generated based on the first image information and second image information acquired by the imaging means in a state in which the pattern light is not irradiated,
The information processing device characterized in that the control means controls the distance measurement means to determine whether to calculate the distance using a first distance measurement mode or a second distance measurement mode based on the luminance evaluation value calculated by the calculation means. The information processing device described in claim 1, characterized in that when the brightness evaluation value calculated by the calculation means is less than a predetermined threshold, the control means controls the distance measurement means to calculate the distance in a first distance measurement mode, and when the brightness evaluation value calculated by the calculation means is equal to or greater than the predetermined threshold, the control means controls the distance measurement means to calculate the distance in a second distance measurement mode, and the control means determines the predetermined threshold according to the light emission intensity of the pattern light irradiation means. An information processing device mounted on a moving object,
3. The information processing apparatus according to claim 2, wherein the control means determines the predetermined threshold value in accordance with an operation mode of the mobile unit. 4. The information processing apparatus according to claim 3, wherein the operation modes of the mobile unit include a first operation mode in which priority is given to distance measurement speed, and a second operation mode in which priority is given to distance measurement accuracy. 3. The information processing apparatus according to claim 2, wherein the control means determines the predetermined threshold value based on a setting of an intensity of the pattern light irradiated by the pattern light irradiating means and a threshold value for detecting the pattern light while the second distance measurement mode is being executed. a travel control means for starting and stopping the travel of the moving body, controlling the travel speed, and turning the moving body;
The information processing device according to claim 3, characterized in that the driving control means suppresses the driving speed and turning of the moving body during a period from when an image is acquired while irradiating a pattern light by the pattern light irradiation means while the second distance measurement mode is being executed, to when an image is acquired while not irradiating the pattern light. The information processing apparatus according to claim 1 , wherein the luminance evaluation value includes an integrated luminance value and an average luminance value. a calculation step of calculating a luminance evaluation value indicating an intensity of disturbance light based on first image information obtained by an imaging step of reflecting light when a pattern light having a predetermined characteristic is irradiated on a subject by the pattern light irradiation step;
a control step of controlling the distance measurement step so as to calculate the distance to the subject in a predetermined distance measurement mode based on the characteristic information of the pattern light,
The predetermined distance measurement mode is
a first distance measurement mode in which the distance is calculated based on the first image information;
a second distance measurement mode in which the distance is calculated based on a difference image generated based on the first image information and second image information acquired in the imaging step without irradiating the pattern light,
An information processing method characterized in that the control step controls the distance measurement step as to whether to calculate the distance using a first distance measurement mode or a second distance measurement mode based on the brightness evaluation value calculated by the calculation step. A computer program for controlling each means of the information processing device according to any one of claims 1 to 7 by a computer.

Images