TWM520146U - Spatial information extractor - Google Patents

Abstract

The present invention discloses a spatial information extractor. The spatial information extractor comprises a structure light generating unit, a camera module and a processing and controlling unit. The structure light generating unit emits a structure light to an object and thereby a measuring pattern formed thereon. The camera module comprises an optically coded element and a sensor, the optically coded element has a reference pattern, wherein the measuring pattern and the reference pattern are patterns with features at least partially corresponding with each other. The measuring pattern and the reference pattern are both projected onto the sensor for the processing and controlling unit to calculate a spatial information, e.g., distance and measure, of the object by comparing the difference of the patterns with features between the measuring pattern and the reference pattern.

Description

Spatial information acquisition device

The present invention relates to a spatial information capture device, and more particularly to a spatial information capture device that senses a spatial information by transmitting a pattern through a structured light generating module and intercepting a pattern by a camera module.

In recent years, with the evolution of the electronics industry and the rapid development of industrial technology, the design and development of various electronic devices have gradually evolved in a light and portable manner, so that users can use it for mobile commerce, entertainment or leisure whenever and wherever. . For example, a wide variety of image capturing devices are widely used in various fields, such as portable electronic devices such as smart phones and wearable electronic devices, which have the advantages of small size and convenient carrying.

Moreover, with the improvement of the quality of life, people have more demands on the images captured by the image capturing device. For example, it is hoped that the obtained image can be a 3D stereoscopic image, and the 3D stereoscopic image includes Accurate depth information. For example, it is desirable for portable electronic devices to have a distance measurement function for gesture recognition. The measurement of the depth information or the measurement of the distance is currently available through Time of Flight (TOF), dual camera distance measurement, and the like.

However, although the measurement results obtained by the time-of-flight distance measurement method have better accuracy, if it is to be extended to the surface or multi-point mirror application, the software operation is extremely complicated, and it is necessary to introduce a specific operation chip and product. The body circuit (IC) has a large power consumption and a high computational cost. In addition, the time-of-flight distance method is also susceptible to environmental brightness, making it less accurate in the case of large external light damage. As for the dual-lens spanning method, although it has a considerable degree of complexity in software computing, it is not simple, and the power consumption and computing cost are better than the time-of-flight method because of the advantages of using the dual mirror. The surface distance is poorly expressed, that is, there is a disadvantage that the accuracy of the measurement results obtained on the smooth surface is low.

In view of the above, it is an object of the technical field to provide a spatial information capturing device that is miniaturized and can obtain a spatial distance information of a target object.

The main purpose of the present invention is to provide a spatial information capturing device. The spatial information capturing device includes a structured light generating module and a camera module. The camera module is specifically provided with an optical encoder, wherein the optical encoder can project A reference pattern, and an estimated pattern of the structured light generating module illuminating the target object, has at least part of a corresponding pattern content relationship. And by comparing the difference between the reference pattern and the content of the pattern between the estimated patterns, it is possible to determine a spatial distance information of the target object.

A preferred embodiment of the present invention provides a spatial information capture device including a structured light generating module, a camera module, and a processing control unit. The structured light The generating module emits a structured lighting pattern to a target object such that the target object presents an estimated pattern. The camera module includes a lens group, an optically coded element, and a sensing unit. The optical encoder has a reference pattern, and the reference pattern and the estimated pattern have at least a portion therebetween. Corresponding pattern features, and the reference pattern and the estimated pattern are projected to the sensing unit. The processing control unit signal is connected to the sensing unit, and the processing control unit determines a spatial distance information of the estimated target object by comparing a difference in pattern characteristics between the reference pattern and the estimated pattern.

In a preferred embodiment, the lens group has an optical axis and a plurality of lenses, and the optical encoder and the lenses are sequentially arranged along the optical axis, so that the estimated pattern is positioned on the lenses of the optical axis. And projecting on the sensing unit located on the optical axis, and projecting the reference pattern located on the optical axis to the sensing unit located on the optical axis; wherein the optical encoder is attached to the lens group, or The optical encoder is separated from the lens group.

In a preferred embodiment, the lens group has a plurality of lenses, and the estimated pattern is projected by the lenses on a first viewpoint position of the sensing unit, and the reference pattern is projected on a second viewpoint position of the sensing unit. The first viewpoint position is different from the second aperture position (viewing aperture for viewing); wherein the optical encoder is attached to the lens group, or the optical encoder is separated from the lens group.

In a preferred embodiment, the lens group has a first optical axis, a second optical axis, a prism, and a plurality of lenses, and the prism refracts the first optical axis and the second optical axis to make the estimation Measuring a pattern of the plurality of lenses on the first optical axis and projecting on the sensing unit located on the first optical axis, and causing the reference pattern located on the second optical axis to be projected through the prism in the first The sensing unit of an optical axis; wherein the optical encoder is attached to the lens group, or the optical encoder Separated from the lens group.

In a preferred embodiment, the optical encoder is a self-illuminating component; or the optical encoder is an optically encoded film.

In a preferred embodiment, the optical encoder is assembled from a plurality of liquid crystal structures, and a stylizing unit controls the liquid crystal structures to cause the liquid crystal structures to exhibit the reference pattern.

In a preferred embodiment, the optical encoder has a different wavelength of light than the structured light emitted by the structured light generating module.

In a preferred embodiment, the camera module further includes a housing and an adjustment mechanism, the adjustment mechanism is partially exposed outside the housing, wherein the adjustment mechanism is coupled to the optical encoder to enable the optical encoder It can be driven by the adjusting mechanism for a user to adjust the relative position between the optical encoder and the lens group.

In a preferred embodiment, the structured light generating module includes a light source, a collimating lens, and a diffractive optical element, and the structured light pattern emitted by the structured light generating module and the object presented on the target Corresponding to the estimated pattern; wherein the light source comprises a laser diode (LD), a light emitting diode (LED), an organic light emitting diode (OLED), and the output has a thermal induction At least one of a light emitting unit of the light beam of the wavelength interval.

In a preferred embodiment, the estimated pattern includes at least one of a grid pattern, a divergent radial pattern, a multi-dot pattern, and a symmetric/asymmetrical pattern.

In a preferred embodiment, the structured light generating module and the processing control unit are dynamically linked to each other, so that the processing control unit can perform regulation on the structured light generating module according to the dynamic change of the estimated pattern.

In a preferred embodiment, the illumination source of the structured light generating module is adjusted and varied, so that the estimated pattern can be changed accordingly.

In a preferred embodiment, the diffractive optical component of the structured light generating module is adjusted to be varied so that the estimated pattern can be changed accordingly.

1‧‧‧Space information acquisition device

11‧‧‧Structural light generation module

111‧‧‧Light source

112‧‧‧ Collimating lens

113‧‧‧Diffractive optical components

113a‧‧‧Structural light pattern

113b‧‧‧ Estimated pattern

12‧‧‧ camera module

121‧‧‧ lens group

122‧‧‧ optical coded parts

122a‧‧‧ benchmark pattern

123‧‧‧Sensor unit

13‧‧‧Processing Control Unit

22‧‧‧ camera module

221‧‧‧ lens group

222‧‧‧ optical coded parts

222a‧‧‧ benchmark pattern

223‧‧‧Sensor unit

225‧‧‧ convex lens

23‧‧‧Processing Control Unit

32‧‧‧ camera module

321‧‧‧ lens group

322‧‧‧ optical coded parts

322a‧‧‧ benchmark pattern

323‧‧‧Sensor unit

325‧‧‧ convex lens

33‧‧‧Processing Control Unit

43‧‧‧Processing Control Unit

42‧‧‧ camera module

421‧‧‧ lens group

422‧‧‧ optical coded parts

422a‧‧‧ benchmark pattern

423‧‧‧Sensor unit

425‧‧‧ convex lens

43‧‧‧Processing Control Unit

9‧‧‧ Subjects

X‧‧‧ optical axis

X1’‧‧‧first optical axis

X2’‧‧‧second optical axis

X1’‧‧‧first optical axis

X2’‧‧‧second optical axis

L‧‧‧ structured light

Figure 1 is a conceptual diagram of the object of the space information acquisition device for estimating the object to be estimated.

FIG. 2 is a conceptual diagram of a camera module according to a first embodiment of the present invention.

FIG. 3 is a conceptual diagram of a camera module according to a second embodiment of the present invention.

4 is a conceptual diagram of a camera module of a third embodiment of the present invention.

Please refer to FIG. 1. FIG. 1 is a conceptual diagram of the object of the space information acquisition device for estimating the object to be estimated.

The spatial information capturing device 1 of the present invention can capture the spatial information of a target object, and the spatial information referred to herein includes: the relative height and depth of the surface of the target object, the distance between the target object and the spatial information capturing device, and the like. Spatial information, which is very helpful for creating 3D stereoscopic images. As for the detailed calculation method for establishing 3D stereoscopic images on the back end, it is not mentioned in this case because it is not the focus of this case.

The spatial information capturing device 1 of the present invention comprises a structured light generating module 11, a camera module 12 and a processing control unit 13. When a user wants to capture the spatial information of the target object 9, the structured light patterning module L of the spatial information capturing device 1 can be used to emit a structured lighting pattern L to the target object 9. An estimated pattern 113b is presented on the target object 9. As for the estimation pattern 113b, it is used as the spatial information of the object 9 to be detected, as will be described later.

First, the structured light generating module 11 will be described. The structured light generating module 11 includes a light source 111, a collimating lens 112, and a diffractive optical element 113. The light source 111 can output a plurality of beams, and the collimating lens 112 is disposed between the light source 111 and the diffractive optical element 113. Its function is to collimate a plurality of beams, and the plurality of beams are incident on the diffractive optical element 113. The diffractive optical element 113 has a pattern formed thereon, and when the plurality of beams pass through the collimator lens 112 and pass through the diffractive optical element 113, the structured light generating unit 1 can output the structured light pattern 113a.

In detail, the light source 111 may be selected from a laser diode (LD), a light emitting diode (LED), an organic light emitting diode (OLED), and a light beam having a wavelength range of thermal sensing. One of the lighting units. The structured light pattern 113a may be at least one of a grid pattern, a divergent radial pattern, a multi-dot pattern, and a symmetric/asymmetric pattern, but this is only an enumeration, and is not limited thereto. .

However, when the structured light L is irradiated to the target object 9, the structured light L is irradiated by the structured light L due to the relative illumination angle between the structured light generating module 11 and the target object 9, or due to the undulation of the surface of the target object 9. Presented on the surface of the object 9 An estimated pattern 113b has been slightly deformed to be different from the original structured light pattern 113a, but still retains at least some corresponding content between the two, such as: offset or tilt angle, corresponding point or line position, phase The size of the corresponding point, the thickness of the corresponding line, the length of the corresponding line, the direction of the corresponding line, the curvature of the corresponding line, and so on. Taking the structured light pattern 113a and the estimated pattern 113b depicted in FIG. 1 as an example, the parallel grid-like lines of the original structured light pattern 113a are deformed to the left narrow and right width on the estimated pattern 113b due to the illumination angle relationship. Not parallel lines. However, this is only one of the examples for convenience of explanation, and is not limited thereto.

Next, the camera module 12 will be described. The camera module 12 of the present invention includes a lens group 121, an optically coded element 122, and a sensing unit 123. One of the features of the present invention is that the optical encoder 122 has a reference pattern 122a. The design of the reference pattern 122a is preferably the same or similar to the structured light pattern 113a. The optical encoder 122 is provided for the purpose of providing the reference pattern 122a and The reference pattern 122a is projected toward the sensing unit of the camera module, so that the reference pattern 122a (this is the first pattern projected on the sensing unit 123) is sensed on the sensing unit 123 as a reference for comparison. On the other hand, on the outer surface of the target object 9 described in the preceding paragraph, the estimated pattern 113b is also captured by the sensing unit 123 of the camera module 12, and thus the sensing unit 123 senses The pattern 113b is estimated (this is the second pattern projected onto the sensing unit 123). Thereafter, the processing control unit 13 connected to the sensing unit 123 is configured to perform pattern feature comparison on the reference pattern 122a (the first pattern) and the estimation pattern 113b (the second pattern), so that the target object can be calculated. 9 spatial information, the main concept of the above implementation.

It should be noted that the reference pattern 122a and the estimated pattern 113b are not necessarily identical, and only a part of the same portion is required, that is, at least a part of the corresponding pattern features between the reference pattern 122a and the estimated pattern 113b. , the part of the same pattern can be used as the feature point of the comparison. In addition, the structured light generating module 11 and the processing control unit 13 are dynamically linked to each other, so that the processing control unit 13 can automatically or passively perform the regulation on the structured light generating module 11 in response to the dynamic change of the estimated pattern 113b. For example, the illumination source 111 of the structured light generating module 11 is adjusted to be changed, so that the estimated pattern 113b can be changed accordingly; or the diffractive optical component 113 of the structured light generating module 11 is adjusted and changed to make an estimate. The pattern 113b can be changed accordingly. The details of the operation mechanism and the contents of the camera module are detailed below.

The following three different camera modules are embodiments of the present invention, so that the reference pattern of the optical encoder and the estimated pattern presented on the target object are well captured by the sensing unit of the camera module for processing the control unit. The comparison is performed to calculate the spatial information of the target object.

FIG. 2 is a conceptual diagram of a camera module according to a first embodiment of the present invention. In the first embodiment, the camera module 22 includes a lens group 221, an optically coded element 222, and a sensing unit 223, wherein the lens group 221, the optical encoder 222, and the sensing unit 223 are along an optical axis. X is arranged in order. The lens group 221 includes a complex lens 221a, and the estimated pattern 113b presented on the target object 9 can be projected and imaged by the sensing unit 223 located on the optical axis X through the complex lens 221a, and the image is captured by the sensing unit 223. On the other hand, the reference pattern 222a on the optical encoder 222 can be projected to the sensing unit 223 through a convex lens 225, and then induced. Unit 223 captures the image. In addition, since the lens group 221, the optical encoder 222, and the sensing unit 223 are arranged along the optical axis X in this embodiment, the estimated pattern 113b and the captured reference pattern 222a may be At least some of the images are stacked, and then processed by the processing control unit 23 to obtain the spatial information of the target object 9.

In this embodiment, in order to enable the reference pattern 222a to be clearly imaged on the sensing unit 223, the convex lens 225 is disposed between the optical encoder 222 and the sensing element 223, preferably, the distance between the optical encoder 222 and the convex lens 225. The distance between the same convex lens 225 and the sensing unit 223. The arrangement of the optical encoder 222 may be attached to the lens group 221, or may be arranged such that the optical encoder 222a is separated from the lens group 221, which is not limited herein.

Further, the optical encoder 222 of the present invention has the following two implementations. In the first embodiment, the optical encoder 222 is a self-illuminating component, for example, which is composed of a plurality of liquid crystal structures, and the liquid crystal structures can be controlled by a stylized unit (not shown). The management controls the patterns that they can present. Moreover, for the purpose of obtaining spatial information, the stylizing unit controls the optical encoder 222 to cause the reference pattern 222a presented on the optical encoder 222 to correspond to the structured light pattern generated by the structured light generating module. Therefore, the reference pattern 222a will also correspond to the estimated pattern 113b on the target object 9. In the second embodiment, the optical encoder 222 is an optically encoded film. After the external light enters the camera module 22 and passes through the optically encoded film, the reference pattern 222a on the optically encoded film can be projected and imaged to the sensing unit. 223.

Furthermore, in order to make the overlapping measurement patterns captured by the sensing unit 223 The distinguishability between the 113b and the reference pattern 222a is increased, and the identifiability between the two projection patterns on the sensing unit 223 is further increased by designing the optical encoder 222 and the structured light L to have different wavelengths of light. Improve the accuracy of the calculation.

On the other hand, the amount of deformation of the reference pattern 222a projected on the optical encoder 222 onto the sensing unit 223 is easily adjusted. The camera module 22 further includes a housing 224 and an adjustment mechanism 226. It is exposed outside the housing 224. The adjusting mechanism 226 is coupled to the optical encoder 222. Therefore, when a user moves the adjusting mechanism 226, the optical encoder 222 can be moved up and down/left/right/back and forth along with the action of the adjusting mechanism 226, thereby adjusting the optical encoding. The relative position between the member 222 and the lens group 221 / sensing unit 223.

FIG. 3 is a conceptual diagram of a camera module according to a second embodiment of the present invention. The component composition of the second embodiment is similar to that of the first embodiment, which is different from the first embodiment in that the relative position between the optical encoder 322 and the lens group 321 is disposed. In the second embodiment, the camera module 32 includes a lens group 321, an optical encoder 322, and a sensing unit 323. The lens group 321 includes a complex lens 321a, and the estimated pattern 113b presented on the target object 9 can be projected and imaged to the first viewpoint position P1 of the sensing unit 323 through the complex lens 321a located on the first optical axis X1, and is further induced by the sensing unit. 323 capture images. On the other hand, the reference pattern 322a on the optical encoder 322 is projected onto the second viewpoint position P2 of the sensing unit 323 through a convex lens 325 located on the second optical axis X2, and is further captured by the sensing unit 323.

Wherein, the first optical axis X1 is preferably disposed parallel to or close to the second optical axis X2, so the first viewpoint position P1 is different from the second viewpoint position. P2 (different aperture for viewing). And by the juxtaposed arrangement of the lens group 321 and the optical encoder 322, the optical encoder 322 will not interfere with the imaging process of the estimation pattern 113b on the target object 9 to the sensing unit 323. The captured estimation pattern 113b and the captured reference pattern 222a are further transmitted through the processing control unit 33, and the captured images are superimposed on each other, or the similarities and differences between the patterns are directly calculated, thereby acquiring the target. The spatial information of the object 9.

In the second embodiment, in order to enable the reference pattern 322a to be clearly imaged on the sensing unit 323, the convex lens 325 is also disposed between the optical encoder 322 and the sensing unit 323, and the distance between the optical encoder 322 and the convex lens 325. Preferably, the distance between the same convex lens 325 and the sensing unit 323. In order to facilitate the adjustment of the amount of deformation of the reference pattern 322a on the optical encoder 322 onto the sensing unit 323, the camera module 32 is also provided with an adjustment mechanism 326 for adjusting the relative position between the optical encoder 322 and the sensing unit 323. .

4 is a conceptual diagram of a camera module of a third embodiment of the present invention. The component composition of the third embodiment is similar to that of the first embodiment, and is different from the first embodiment mainly except that the relative position between the optical encoder and the lens group is different from the first two embodiments, and an additional addition is made. A prism is configured. In detail, in the embodiment, the camera module 42 includes a lens group 421, an optical encoder 422, and a sensing unit 423, wherein the lens group 421 has a first optical axis X1' and a second optical axis. X2', the complex lens 421a and a prism 421b, the prism 421b refracts the first optical axis X1' and the second optical axis X2', so that the estimated pattern 113b presented on the target object 9 can pass through the first optical axis. X1' complex lens 421a and prism 421b are well imaged at first The sensing unit 423 of the optical axis X1' is further captured by the sensing unit 423. On the other hand, the reference pattern 422a on the optical encoder 422 disposed on the second optical axis X2' is reflected by the prism 421b to the sensing unit 423 positioned on the first optical axis X1'.

Wherein, the first optical axis X1' is preferably disposed perpendicular to the second optical axis X2' or is disposed perpendicular to the vertical direction, and the optical encoder 422 is not disposed by the optical encoder 422 being offset from the first optical axis X1'. It will interfere with the imaging process of the estimation pattern 113b on the target object 9 projected onto the sensing unit 423. Through the relative positions of the prism 421b, the lens group 421 and the optical encoder 422 which are arranged in this embodiment, the estimated pattern 113b captured and the reference pattern 222a captured may be at least partially overlapped, and thereafter The retransmission processing control unit 23 calculates the spatial information of the target object 9.

In the third embodiment, in order to facilitate the adjustment of the amount of deformation of the reference pattern 422a projected onto the sensing unit 423 on the optical encoder 422, the camera module 42 may be provided with an adjustment mechanism 426 for adjusting the optical encoder 422 and the prism. The relative position between 421b and 421b.

In summary, the spatial information capture device of the present invention is specially equipped with an optical encoder, and the reference pattern of the optical encoder and the estimated pattern on the target object can be simultaneously projected to the sensing unit for processing control unit processing. The spatial distance information of the object to be marked, and it is no longer necessary to pre-position the content of the reference pattern in the processing control unit, which not only saves the construction time and labor cost, but also reduces the calculation of the control unit and improves the operation. The speed of the comparison.

The above embodiments are merely illustrative of the principles and effects of the present invention, as well as the technical features of the present invention, and are not intended to limit the scope of protection of the present invention. Any familiar with this technology Any change or equality of arrangements that can be easily accomplished by a person skilled in the art without departing from the technical principles and spirit of the present invention is within the scope of the present invention. Therefore, the scope of protection of the present invention should be as set forth in the scope of the patent application described later.

1‧‧‧Space information acquisition device

11‧‧‧Structural light generation module

111‧‧‧Light source

112‧‧‧ Collimating lens

113‧‧‧Diffractive optical components

113a‧‧‧Structural light pattern

113b‧‧‧ Estimated pattern

12‧‧‧ camera module

121‧‧‧ lens group

122‧‧‧ optical coded parts

122a‧‧‧ benchmark pattern

123‧‧‧Sensor unit

13‧‧‧Processing Control Unit

9‧‧‧ Subjects

L‧‧‧ structured light

Claims (13)

A spatial information capture device includes: a structured light generating module that emits a structured lighting pattern to a target object such that the target object presents an estimated pattern; and a camera module including a lens group An optically coded element having a reference pattern, the reference pattern and the estimated pattern having at least a portion of corresponding pattern features, and the sensing element The reference pattern and the estimated pattern are projected to the sensing unit; and a processing control unit, the signal is connected to the sensing unit, and the processing control unit compares the pattern feature between the reference pattern and the estimated pattern to determine A spatial distance information of the estimated object. The spatial information capturing device of claim 1, wherein the lens group has an optical axis and a plurality of lenses, and the optical encoder and the lenses are sequentially arranged along the optical axis, so that the estimated pattern is The lenses positioned on the optical axis are projected on the sensing unit located on the optical axis, and the reference pattern positioned on the optical axis is projected on the sensing unit located on the optical axis; wherein the optical encoding The member is attached to the lens group, or the optical encoder is separated from the lens group. The spatial information capturing device of claim 1, wherein the lens group has a plurality of lenses, and the estimated pattern is projected by the lenses on a first viewpoint position of the sensing unit, and the reference pattern is projected on the lens a second viewpoint position of the sensing unit, the first viewpoint position being different from the second aperture position (the different aperture for viewing); wherein the optical encoder is attached to the lens group, or the optical encoder is separated from the lens group . The spatial information capturing device of claim 1, wherein the lens group has a first optical axis, a second optical axis, a prism, and a plurality of lenses, the prism refracting the first optical axis and The second optical axis causes the estimated pattern to be projected on the sensing unit of the first optical axis by a plurality of lenses positioned on the first optical axis, and the reference pattern positioned on the second optical axis The sensing unit is projected by the prism on the first optical axis; wherein the optical encoder is attached to the lens group, or the optical encoder is separated from the lens group. The spatial information capture device of claim 1, wherein the optical encoder is a self-illuminating component; or the optical encoder is an optically encoded film. The spatial information capturing device according to claim 1, wherein the optical encoder is composed of a plurality of liquid crystal structures, and a stylized unit controls and manages the liquid crystal structures, so that the liquid crystal structures are presented. The reference pattern. The spatial information capture device of claim 1, wherein the optical encoder has a different wavelength of light than the structured light emitted by the structured light generating module. The space information capture device of claim 1, wherein the camera module further includes a housing and an adjustment mechanism, the adjustment mechanism being partially exposed outside the housing, wherein the adjustment mechanism is coupled to the The optical encoder enables the optical encoder to be driven by the adjustment mechanism for a user to adjust the relative position between the optical encoder and the lens assembly. The spatial information capturing device of claim 1, wherein the structured light generating module comprises a light source, a collimating lens and a diffractive optical element, and the structured light generating module emits the The structured light pattern corresponds to the estimated pattern presented on the target object; wherein the light source comprises a laser diode (LD), a light emitting diode (LED), and an organic light emitting diode ( An OLED) and at least one of a light emitting unit for outputting a light beam having a thermally induced wavelength interval. The spatial information capturing device of claim 9, wherein the structured light pattern comprises at least one of a grid pattern, a divergent radial pattern, a multi-dot pattern, and a symmetric/asymmetrical pattern. By. The spatial information capturing device of claim 9, wherein the structured light generating module and the processing control unit are dynamically linked to each other, so that the processing control unit can estimate the dynamic change of the pattern. The structured light generating module performs regulation. The spatial information capturing device of claim 11, wherein the illumination source of the structured light generating module is adjusted and varied, so that the estimated pattern can be changed accordingly. The spatial information capturing device of claim 11, wherein the diffractive optical component of the structured light generating module is adjusted to be varied, so that the estimated pattern can be changed accordingly.