TWI604196B - An optical flow speed measuring module and the method thereof - Google Patents

Description

Optical flow speed measuring module and speed measuring method thereof

The invention relates to an optical flow speed measuring module and a speed measuring method thereof, and more particularly to an optical flow speed measuring module and a speed measuring method thereof for taking a plurality of images by using a camera array to calculate a statistical speed.

The optical flow speed measurement module is a set of modules used to calculate its own speed. Conventionally, it is often installed on the UAV. By taking a camera shot on the ground, the image of the ground is obtained, and the displacement of the image pixels is calculated to obtain the speed information. However, in practice, the flow of computing light is often susceptible to light reflections from ambient light sources or ground materials that affect the operation.

This is because the optical flow speed measurement module calculates the relative speed by continuously shooting the ground and comparing the movement of the image texture. If the texture of the image is clearly identifiable, it is easy to estimate the speed of the optical flow speed measuring module by the optical flow algorithm. However, the conventional optical flow speed measuring module usually only carries one camera. If the ground reflection is quite serious or the ambient light source is too bright when shooting, it will affect the optical flow calculation.

In view of the foregoing problems, one aspect of the present invention is to provide an optical flow speed measuring module, which can take multiple images of multiple groups by using a camera array with multiple cameras, and then perform image analysis to calculate the image presented by each image. After the speed and standard deviation, perform a statistical operation In order to take out a statistical optical speed measurement module speed.

The optical flow speed measuring module provided by the present invention is mounted on a mobile device for measuring a moving speed of the mobile device, comprising: a camera array, an arithmetic unit and a statistical unit. The camera array includes N×M cameras for capturing multiple sets of N×M ambient images of the mobile device; the computing unit is electrically connected to the camera array, and is calculated according to multiple sets of N×M surrounding images. N×M camera speeds and N×M standard deviations; the statistical unit is electrically connected to the arithmetic unit for calculating the moving speed of the mobile device according to N×M camera speeds and N×M standard deviations, wherein N and M is a natural number greater than one.

In an embodiment of the invention, wherein the statistical unit calculates the speed of movement of the mobile device by the following formula: Where K=N×M, V _s is each of the N×M camera speeds, σ _s is each of the N×M standard deviations, and V _out is the motion speed of the mobile device.

In an embodiment of the present invention, the operation unit includes an image processing unit for digitizing pixels and brightness of the plurality of sets of N×M surrounding image images, and correspondingly generating numerical values of the N×M surrounding environment images. Results and pixel number - luminance analysis histogram.

In an embodiment of the present invention, the arithmetic unit includes a speed computing device, and the speed computing device calculates an N×M camera speeds by using an optical flow algorithm based on the numerical results of the N×M ambient images.

In an embodiment of the present invention, the operation unit includes a standard deviation operation device for performing data according to each N×M pixel number-luminance analysis histogram. An operation of N × M standard deviations.

Another aspect of the present invention is to provide a speed measuring method for an optical flow speed measuring module for measuring a moving speed of a mobile device, wherein the mobile device is equipped with an optical flow speed measuring module, and the optical flow speed measuring module includes a camera. An array, an arithmetic unit and a statistical unit, wherein the camera array comprises N×M cameras, and N and M are both natural numbers greater than 1. The speed measuring method of the optical flow speed measuring module comprises the following steps: S1: using a camera array N×M cameras are taken to obtain a plurality of sets of N×M ambient images of the mobile device; S2: using the computing unit to calculate N×M camera speeds according to multiple sets of N×M surrounding environment images; S3: Using the computing unit to calculate N×M standard deviations according to multiple sets of N×M surrounding environment images; and S4: using the statistical unit to calculate the mobile device based on N×M camera speeds and N×M standard deviations Movement speed.

In another embodiment of the invention, wherein the statistical unit calculates the speed of movement of the mobile device by the following formula: Where K=N×M, V _s is each of the N×M camera speeds, σ _s is each of the N×M standard deviations, and V _out is the motion speed of the mobile device.

In another embodiment of the present invention, the computing unit includes an image processing unit, and the optical flow speed measuring module speed measuring method further comprises the step of: digitizing pixels and brightness of the plurality of sets of N×M surrounding images using the image processing unit. And corresponding to the numerical results of the N × M ambient image, and the number of pixels - brightness analysis histogram.

In another embodiment of the present invention, wherein the arithmetic unit includes a speed computing device The method is used to calculate N×M camera speeds by an optical flow algorithm according to the numerical result of the N×M surrounding images.

In another embodiment of the present invention, the arithmetic unit includes a standard deviation operation device for performing an operation of each N×M standard deviations according to the data presented by each pixel number-luminance analysis histogram.

Compared with the prior art, the present invention utilizes a camera array composed of a plurality of cameras to take a plurality of images to reduce the unfavorable factors that are difficult to identify due to ground reflection or ambient light source, and can improve the amount of the optical flow measuring module. Speed reliability.

1‧‧‧ optical flow speed module

10‧‧‧ camera array

102‧‧‧Base

104‧‧‧ camera

12‧‧‧ arithmetic unit

122‧‧‧Image processing device

124‧‧‧Speed computing device

126‧‧‧Standard Difference Operator

14‧‧‧Statistics unit

2‧‧‧Measurement method of optical flow speed measuring module

V _s ‧‧‧ camera speed

V _out ‧‧‧Moving speed of mobile equipment

σ _s ‧ ‧ standard deviation

1 is a functional block diagram of an embodiment of the present invention.

Figure 2 depicts a top view of the camera array of the present invention.

Figure 3 depicts a side view of a camera array of the present invention.

4A-4B illustrate a surrounding environment image and its corresponding number of pixels-luminance analysis histograms in accordance with an embodiment of the present invention.

5A-5B illustrate still another ambient image and its corresponding number of pixels-luminance analysis histograms in accordance with an embodiment of the present invention.

6A-6B illustrate another ambient image and its corresponding number of pixels-luminance analysis histograms in accordance with an embodiment of the present invention.

7 is a schematic diagram of ambient image, its corresponding number of pixels - luminance analysis histogram, and weights thereof, in accordance with an embodiment of the present invention.

Figure 8 is a block diagram showing the functional blocks of another aspect of the present invention.

Please refer to FIG. 1 to FIG. 3 . FIG. 1 is a schematic diagram of functional blocks of an embodiment of the present invention. Figure 2 depicts a top view of the camera array of the present invention. Figure 3 depicts a side view of a camera array of the present invention.

The present invention provides an optical flow speed measuring module 1 adapted to be mounted on a mobile device and to calculate a moving speed of the mobile device, comprising: a camera array 10, an arithmetic unit 12, and a statistical unit 14.

The camera array 10 includes a pedestal 102 and N×M cameras 104 for capturing multiple sets of N×M ambient images of the mobile device; the computing unit 12 is electrically connected to the camera array 10, and according to N×M The ambient image calculates N×M camera speeds and N×M standard deviations; N standard deviation standard deviation indicates that N images taken by the N-subject cameras 104 are taken to calculate a surrounding environment image, and a set is calculated. Each surrounding environment image in N×M ambient images corresponds to a standard deviation. These standard deviations are N×M, collectively referred to as N×M standard deviations. The statistical unit 14 is electrically connected to the computing unit 12, and is configured to calculate the moving speed of the mobile device according to the N-connection, the camera speed and the N-camera standard deviation, wherein N and M are both natural numbers greater than 1.

The term "image" is used to describe a still image or a moving image, but in the present embodiment, it is solved by a still image. However, the term "multiple groups" is used for static images. Those of ordinary skill in the art can also know that N×M motion pictures are captured by N×M cameras, and then N×M motion pictures are The lattice is decomposed into a plurality of N x M still images.

In this embodiment, the camera array 10 shown in FIG. 2 is a 4×4 square matrix, that is, a camera array with N=M=4. However, this embodiment is for convenience of description, and the present invention does not limit the camera. The array 10 must form a camera array in a square matrix, and each camera 104 in the camera array 10 of the present invention is not limited to being photographed in the same direction, and the individual cameras 104 are phased. Shooting in different directions to further reduce environmental factors that affect speed measurement.

The motorized device installed in the optical flow speed measuring module 1 of the present invention comprises a mobile device such as an unmanned aerial vehicle, a bicycle, a motor vehicle, etc., and N x M cameras of the camera array 10 when the mobile device of the present invention is installed in motion 104 will capture multiple sets of N x M ambient images around the mobile device for speed measurement. The N×M ambient images are further calculated by the arithmetic unit 12 to calculate N×M camera speeds and N×M standard deviations, respectively.

The computing unit 12 includes an image processing device 122, a speed computing device 124, and a standard deviation computing device 126. The image processing device 122 performs pixel and brightness analysis on the ambient image captured by each camera 104, thereby correspondingly generating numerical results of N×M ambient images, and generating N×M surroundings according to those numerical results. The number of pixels of the environmental image - the luminance analysis histogram.

Please refer to FIG. 4A to FIG. 4B, FIG. 5A to FIG. 5B, and FIG. 6A to FIG. 6B and the like. 4A to 4B, 5A to 5B, and 6A to 6B respectively illustrate a surrounding environment image and its corresponding pixel number-luminance analysis histogram according to an embodiment of the present invention.

4A is an ambient environment experimental image taken by the camera 104 of the camera array 10 under normal ambient brightness, FIG. 5A is an ambient environment experimental image taken by the camera 104 under high ambient brightness, and FIG. 6A is a low ambient brightness. The surrounding environment experimental image taken by the camera 104. 4B, FIG. 5B and FIG. 6B are respectively obtained by computerizing the numerical results of FIG. 4A, FIG. 5A and FIG. 6A via the image processing device 122, and then counting the relationship between the number of pixels and the brightness. The number of pixels of the surrounding experimental image - the brightness analysis histogram.

Referring to FIG. 4A to FIG. 4B, in the background of the general ambient brightness, the texture of the image in FIG. 4A can be clearly seen, and the corresponding FIG. 4B can be seen in FIG. 4A. Pixels are rendered at high brightness, but there are also a certain number of pixels distributed in the middle and low brightness portions. The tendency is that the image brightness distribution is averaged and the standard deviation is high.

Referring to FIG. 5A to FIG. 5B, in the background of higher ambient brightness, only about half of the image texture portion in FIG. 5A is still visible, and the corresponding pixel of FIG. 5A can be seen in the corresponding FIG. 5B. They are all presented in high brightness, and few pixels are distributed to the middle and low brightness parts. The image brightness concentration region has a high luminance and the standard deviation is low, and the average luminance value tends to be high.

Finally, referring to FIG. 6A to FIG. 6B, in the background of lower ambient brightness, most of the image texture parts in FIG. 6A are hardly visible, and corresponding pixels in FIG. 6B can be seen in the corresponding pixels in FIG. 6A. Presented at low brightness, and few pixels are distributed to the medium and high brightness parts. The image brightness concentration region tends to be low in brightness, the standard deviation is low, and the average brightness value tends to be low.

Next, the numerical result of the N×M surrounding environment images is output to the speed computing device 124, and the speed computing device 124 calculates each camera 104 by the optical flow algorithm based on the numerical results of the N×M surrounding environment images. The moving speeds of the captured N×M surrounding images, and the number of pixels of the N×M surrounding images-luminance analysis histograms are transmitted to the standard deviation computing device 126 to analyze the number of pixels according to the number of pixels. The data presented by the histogram computes the standard deviation of each ambient image.

Please refer to FIG. 7. FIG. 7 is a schematic diagram of ambient image, corresponding pixel number-luminance analysis histogram and weights thereof, according to an embodiment of the present invention. It should be understood that if there is a number of pixels of the surrounding environment image - the luminance analysis histogram shows that the luminance distribution is concentrated in the high luminance interval, and the standard deviation value or the weight value is low, as shown in FIG. 7(C). , which means that the surrounding image is too much affected by the light refraction, resulting in overexposure and insufficient Enough image texture to determine speed. Conversely, if there is a number of pixels of the surrounding environment image - the luminance analysis histogram shows that the luminance distribution is concentrated in the low luminance interval, and the standard deviation value or the weight value is low, as shown in FIG. 7(A), On behalf of the surrounding environment, the image is underexposed and too dark, and it is impossible to provide enough image texture to judge the speed. If there is a pixel number of the surrounding environment image - the luminance analysis histogram shows that the luminance distribution has a pixel distribution in each luminance interval, and the standard deviation value or the weight value is not low, as shown in FIG. 7(B). , it means that the surrounding environment image is hardly affected by light refraction. Without overexposure, sufficient image texture should be provided to judge the speed.

The method for digitizing the image and the method for calculating the image speed and the standard deviation using the optical flow algorithm should be a conventional technique, and those skilled in the art should naturally know the relevant technical content when reading the present specification.

After calculating the standard speed difference between the moving speed V _{s of} each camera 104 and each surrounding environment image, the statistical unit 14 performs the maneuver with the camera array 10 installed according to the standard deviation of the moving speed and each surrounding environment image in the following formula 1. The calculation of the speed of movement of the device.

Where K=N×M, V _s is each of the N×M camera speeds, σ _s is each of the N×M standard deviations, and V _out is the motion speed of the mobile device. In the first formula, the standard deviation σ _s represents the weight, and the standard deviation σ _s value is higher, which represents that the ambient image corresponding to the standard deviation can provide sufficient image texture, and the camera that photographs the surrounding environment image is taken. The speed has a higher reference value; if the standard deviation σ _s is lower, the ambient image corresponding to the standard deviation cannot provide sufficient image texture, and the camera speed of the surrounding image is taken. Lower reference value. The standard deviation σ _s has NxM, which is collectively referred to as N standard deviation, which respectively correspond to each surrounding image of a group of N sets of surrounding environment images taken by N standard cameras.

Through the calculation of the statistical unit 14, a reliable measurement of the speed of the mobile device can be obtained, which removes the adverse effects such as ground reflection or ambient light source or insufficient ambient light source.

Referring to FIG. 8, FIG. 8 is a schematic diagram of functional blocks of another scope of the present invention. Another aspect of the present invention is to provide a speed measuring method for an optical flow speed measuring module for measuring a moving speed of a mobile device, wherein the mobile device is equipped with an optical flow speed measuring module, and the optical flow speed measuring module includes a camera. The array 10, an arithmetic unit 12, and a statistical unit 14, wherein the camera array 10 includes a base 102 and N×M cameras 104, and N and M are both natural numbers greater than 1, and the speed measuring method of the optical flow speed measuring module The method comprises the following steps: S1: using N×M cameras of the camera array to obtain multiple sets of N×M surrounding environment images of the mobile device; S2: calculating, by using the computing unit, according to multiple sets of N×M surrounding environment images N×M camera speeds; S3: using the arithmetic unit to calculate N×M standard deviations according to multiple sets of N×M surrounding environment images; and S4: using statistical units according to N×M camera speeds, and N× The M standard deviations calculate the speed of movement of the mobile device.

In step S1, a plurality of sets of N x M ambient images around the mobile device are captured using a camera array having N x M cameras mounted on a mobile device, such as an unmanned aerial vehicle. Wherein, each camera 104 in the camera array 10 of the present invention is not limited to shooting in the same direction, and the individual cameras 104 are photographed in different directions to further reduce the environmental factors affecting the speed measurement.

Next, in step S2, the arithmetic unit 12 calculates the speed of each of the cameras 104 based on a plurality of sets of N x M surrounding image images. First, the arithmetic unit 12 includes an image processing device 122, a speed computing device 124, and a standard deviation computing device 126. The image processing device 122 first analyzes the surrounding environment image captured by each camera 104 by pixel and brightness, thereby correspondingly generating numerical results of N×M surrounding environment images, and generating N×M according to those numerical results. The number of pixels in the surrounding image - the brightness analysis histogram. Therefore, between steps S1 and S2, step S15 is further included: using the image processing unit to quantize the pixels and brightness of the N×M surrounding image images, and correspondingly generating numerical results of the N×M surrounding environment images, and pixels. Number - Brightness analysis histogram.

After the numerical result of the N×M surrounding images is generated, the speed computing device 124 executes step S2 to calculate the moving speed included in the surrounding image captured by each camera 104 by the optical flow algorithm.

Next, the pixel number-luminance analysis histogram of the N×M surrounding images is transmitted to the standard deviation operation device 126, and the standard deviation operation device 126 performs step S3 to perform data calculation according to the pixel number-luminance analysis histogram. The standard deviation of each ambient image.

Finally, in step S4, the statistical unit 14 performs the motorized device in which the camera array 10 is mounted according to the following formula based on the moving speed of each camera calculated via step S2 and the standard deviation of each surrounding environment image calculated in step S3. Calculation of the speed of movement.

Where K=N×M, V _s is each of the N×M camera speeds, σ _s is each of the N×M standard deviations, and V _out is the motion speed of the mobile device. In the first formula, the standard deviation σ _s represents the weight, and the standard deviation σ _s value is higher, which represents that the ambient image corresponding to the standard deviation can provide sufficient image texture, and the camera that photographs the surrounding environment image is taken. The speed has a higher reference value; if the standard deviation σ _s is lower, the ambient image corresponding to the standard deviation cannot provide sufficient image texture, and the camera speed of the surrounding image is taken. Lower reference value.

By using the optical flow speed measuring module 1 of the present invention through the above steps, a reliable motorized device speed measurement value can be obtained which removes unfavorable factors such as ground reflection or ambient light source or insufficient ambient light source.

In summary, the present invention provides an optical flow speed measuring module and a speed measuring method thereof. The optical flow speed measuring module includes a camera array, an arithmetic unit and a statistical unit. The camera array comprises N×M cameras for capturing a plurality of sets of N×M ambient images of a mobile device, and the computing unit respectively calculates N×M surrounding environment images according to the plurality of sets of N×M surrounding environment images. The camera speed is different from its standard, and the statistical unit calculates the motion speed of the mobile device based on N×M camera speeds and N×M standard deviations.

Compared with the prior art, the present invention utilizes a camera array composed of a plurality of cameras to take a plurality of images to reduce the unfavorable factors that are difficult to identify due to ground reflection or ambient light source, and can improve the amount of the optical flow measuring module. Speed reliability.

The features and spirit of the present invention will be more apparent from the detailed description of the preferred embodiments. On the contrary, its purpose is to cover various changes and equal arrangements. Within the scope of the patent scope of the invention as claimed. Therefore, the scope of the patented scope of the invention should be construed as broadly construed in the

1‧‧‧ optical flow speed module

10‧‧‧ camera array

12‧‧‧ arithmetic unit

122‧‧‧Image processing device

124‧‧‧Speed computing device

126‧‧‧Standard Difference Operator

14‧‧‧Statistics unit

Claims (10)

An optical flow speed measuring module is mounted on a mobile device for measuring a moving speed of the mobile device. The optical flow speed measuring module comprises: a camera array, the camera array comprises N×M cameras, Obtaining a plurality of sets of N×M ambient images of the mobile device by shooting; an computing unit electrically connecting with the camera array, and calculating N×M camera speeds according to the plurality of sets of N×M surrounding environment images And a plurality of sets of N×M standard deviations; and a statistical unit electrically connected to the arithmetic unit for calculating the mobile device according to the N×M camera speeds and any set of N×M standard deviations Movement speed; where N and M are both natural numbers greater than one. The optical flow speed measuring module according to claim 1, wherein the statistical unit calculates the moving speed of the mobile device by the following formula: Where K=N×M, V _s is each of the N×M camera speeds, σ _{s is} each standard deviation of any one of N×M standard deviations, and V _out is the moving speed of the mobile device . The optical flow speed measuring module of claim 1, wherein the computing unit comprises an image processing unit for digitizing pixels and brightness of the plurality of sets of N×M surrounding image images, and correspondingly generating the image Multiple sets of N×M numerical results of multiple sets of N×M surrounding images, and multiple sets of N×M number of pixels-luminance analysis histograms. The optical flow speed measuring module according to claim 3, wherein the arithmetic unit comprises a speed computing device, and the speed computing device uses an optical flow algorithm to calculate according to the plurality of sets of N×M numerical results. The N x M camera speeds are output. The optical flow speed measurement module according to claim 3, wherein the operation unit comprises a standard deviation operation device, configured to analyze data represented by the histogram according to each of the plurality of sets of N×M pixels. And performing an operation for each standard deviation of each of the plurality of sets of N × M standard deviations. A speed measuring method for an optical flow speed measuring module for measuring a moving speed of a mobile device, wherein the mobile device is provided with an optical flow speed measuring module, the optical flow speed measuring module comprising a camera array, an arithmetic unit and a statistical unit, wherein the camera array comprises N×M cameras, and N and M are both natural numbers greater than 1. The speed measuring method of the optical flow speed measuring module comprises the following steps: using the N×M of the camera array Taking a camera to obtain a plurality of sets of N×M ambient images of the mobile device; using the computing unit to calculate N×M camera speeds according to the plurality of sets of N×M surrounding environment images; using the computing unit according to Equivalent NxM standard deviations corresponding to the plurality of sets of N×M surrounding environment images of any one of the groups of N×M surrounding environment images; and using the statistical unit according to the N×M cameras The speed, in contrast to the N x M standard deviations, is calculated for the speed of movement of the powered device. The speed measuring method of the optical flow speed measuring module according to claim 6, wherein the statistical unit calculates the moving speed of the mobile device by the following formula: Where K=N×M, V _s is each of the N×M camera speeds, σ _s is each of the N×M standard deviations, and V _out is the motion speed of the mobile device. The method for measuring the speed of the optical flow speed measuring module according to claim 6, wherein the computing unit comprises an image processing unit, and the speed measuring method of the optical flow speed measuring module further comprises the step of: using the image processing unit to use the image processing unit × pixels of the surrounding image are quantized and corresponding to the plurality of sets of N×M numerical results of the plurality of sets of N×M ambient images, and a plurality of sets of N×M pixels-luminance analysis histogram Figure. The speed measuring method of the optical flow speed measuring module according to the eighth aspect of the invention, wherein the calculating unit comprises a speed calculating device, configured to calculate an optical flow algorithm according to the plurality of sets of N×M numerical results The N x M camera speeds. The method for measuring the speed of the optical flow speed measuring module according to claim 8, wherein the calculating unit comprises a standard deviation calculating device, which is configured according to each of the plurality of sets of N×M pixels-brightness analysis histogram The data is calculated for each of the N x M standard deviations.