WO2022027444A1 - Event detection method and device, movable platform, and computer-readable storage medium - Google Patents

Abstract

An event detection method and device, a movable platform, and a computer-readable storage medium. The method comprises: obtaining at least two images acquired by an industrial camera; performing image processing according to the at least two images, so as to obtain a processing result, wherein the processing result is used to indicate changes in pixel brightness in the images; performing rendering according to the processing result, so as to obtain an event image, wherein the event image is used to indicate a brightness change event; and outputting the event image. In this way, event detection can be performed by directly using images acquired by a conventional industrial camera without needing to specifically provide an event camera, thereby reducing detection costs to a certain extent.

Description

Event detection method, apparatus, removable platform, and computer-readable storage medium technical field

The present invention belongs to the technical field of machine vision, and in particular, relates to an event detection method, a device, a movable platform and a computer-readable storage medium.

Background technique

In tasks in the field of machine vision, such as high-speed perception of moving objects, high dynamic range scene perception, real-time robot localization and map reconstruction (SLAM), and target recognition and tracking of moving objects, it is often necessary to detect changes in the external environment. , that is, to detect events in the external environment that cause changes in brightness.

In the prior art, an event camera is often installed, and event detection is implemented through the event camera. Specifically, the event camera is a machine vision image sensor. The event camera can directly output the change value of the light intensity through its internal special pixel circuit when sensing the change of light intensity in the external environment, thereby realizing event detection. However, since the structure of the pixel circuit in the event camera is complex and the process is difficult, the cost is often high. In this way, the detection cost will be higher when the event camera is used for detection.

SUMMARY OF THE INVENTION

The present invention provides an event detection method, a device, a movable platform and a computer-readable storage medium, so as to solve the problem of high detection cost by using an event camera for detection.

In order to solve the above-mentioned technical problems, the present invention is achieved in this way:

In a first aspect, an embodiment of the present invention provides an event detection method, which includes:

Acquire at least two frames of images captured by an industrial camera;

Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

The event image is output.

In a second aspect, an embodiment of the present invention provides an event detection apparatus, the apparatus includes a memory and a processor;

the memory for storing program codes;

The processor calls the program code, and when the program code is executed, is configured to perform the following operations:

Acquire at least two frames of images captured by an industrial camera;

Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

The event image is output.

In a third aspect, an embodiment of the present invention provides a movable platform, where the movable platform includes an industrial camera and the above-mentioned event detection device; the event detection device is configured to perform the following operations:

Acquire at least two frames of images captured by an industrial camera;

Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

The event image is output.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the following operations are implemented:

Acquire at least two frames of images captured by an industrial camera;

Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

The event image is output.

In the embodiment of the present invention, at least two frames of images collected by an industrial camera may be acquired, and image processing is performed according to the at least two frames of images to obtain a processing result, wherein the processing result is used to characterize the brightness change of the pixels in the image, and then according to The processing result is rendered to obtain an event image, wherein the event image is used to represent the brightness change event, and finally, the event image is output. In this way, event detection can be realized directly based on images collected by conventional industrial cameras without the need to specially set up an event camera, so the detection cost can be saved to a certain extent.

Description of drawings

1 is a flowchart of steps of an event detection method provided by an embodiment of the present invention;

2 is a schematic diagram of an output comparison provided by an embodiment of the present invention;

3 is a schematic diagram of an output provided by an embodiment of the present invention;

4 is a schematic diagram of a processing process of a specific example provided by an embodiment of the present invention;

5 is a block diagram of an event detection apparatus provided by an embodiment of the present invention;

6 is a schematic diagram of a hardware structure of a device implementing various embodiments of the present invention;

FIG. 7 is a block diagram of a computing processing device according to an embodiment of the present invention;

FIG. 8 is a block diagram of a portable or fixed storage unit according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

FIG. 1 is a flowchart of steps of an event detection method provided by an embodiment of the present invention. As shown in FIG. 1 , the method may include:

Step 101: Acquire at least two frames of images collected by an industrial camera.

The event detection method provided in the embodiment of the present invention may be performed by an event detection device. For example, the event detection device may be an unmanned aerial vehicle, an autonomous vehicle, an automatic guided transport vehicle, a mobile robot, and the like.

Industrial cameras can shoot continuously to capture a stream of image frames. The at least two frames of images may be images included in the image frame data stream continuously captured by the industrial camera, and the capture moments corresponding to these images may be different. The industrial camera can be mounted on the event detection device, and accordingly, the acquisition can be achieved by directly reading the image captured by the industrial camera. The industrial camera can also be set independently from the event detection device, and accordingly, the acquisition can be achieved by receiving images transmitted by the industrial camera through wireless transmission technology. Further, the industrial camera may use a general-purpose image sensor chip, for example, the industrial camera may be a camera based on a charge coupled device (Charge Coupled Device, CCD) or a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) chip. Since the industrial camera does not require special pixel circuits, that is, no special design and process are required at the sensor chip level, the structure is simple, and the manufacturing process is more mature, so the cost is lower. In this way, detection costs can be saved to a certain extent by using images captured by industrial cameras for event detection.

Step 102: Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image.

Since the events to be detected are events that cause changes in light intensity, changes in brightness often cause changes in light intensity. At the same time, since each frame of image contains the brightness of the frame of image, the embodiment of the present invention may first perform image processing through at least two frames of images to obtain a processing result that can characterize the brightness change of pixels in the image. Wherein, the processing result may represent the brightness change of one frame of the at least two frames of images relative to the pixels of the remaining frame of images. For example, the at least two frames of images may be two frames of images, and the processing result may represent the brightness changes of the pixels of the previous frame of images relative to the next frame of images in the two frames of images. The processing result may also represent brightness changes of the multiple frames of images in the at least two frames of images relative to the pixel points of the remaining frame images. For example, at least two frames of images may be three frames of images, and the processing result may represent the brightness changes of pixels in the first two frames of images relative to the last frame of images.

Step 103: Render according to the processing result to obtain an event image; the event image is used to represent a brightness change event.

In the embodiment of the present invention, since the processing result represents the brightness change of a pixel, and the number of pixels is often large, rendering can be performed according to the brightness change of multiple pixels to summarize the brightness changes of these pixels. , which visualizes the overall brightness change relative to other images. Since the brightness change between images is often caused by the brightness change event, the brightness change event can be represented by the event image obtained after rendering.

Step 104, outputting the event image.

In this embodiment of the present invention, outputting the event image may be directly displaying the event image, or may be sending the event image to the display terminal. Since the event image can represent the brightness change event, and the image is more intuitive, the way of outputting the event by outputting the event image can make the user know the event more intuitively and conveniently to a certain extent, thereby improving the user's viewing efficiency.

To sum up, the event detection method provided by the embodiment of the present invention can acquire at least two frames of images collected by an industrial camera, perform image processing according to the at least two frames of images, and obtain a processing result, wherein the processing result is used to characterize the pixels in the image The brightness change of the point is then rendered according to the processing result to obtain an event image, wherein the event image is used to represent the brightness change event, and finally, the event image is output. In this way, event detection can be realized directly based on images collected by conventional industrial cameras without the need to specially set up an event camera, so the detection cost can be saved to a certain extent.

Optionally, the above operation of performing image processing according to at least two frames of images may be implemented through the following sub-steps (1) to (2):

Sub-step (1): for each frame of the image, obtain the value of each color channel of the image respectively.

In this embodiment of the present invention, the color spaces to which the images belong are different, and the color channels included in the images may be different. For example, if the image is an image in the red-green-blue (RGB) color space, the image can contain three color channels R, G, and B. Accordingly, in this step, R, G, and B in the image can be extracted. Channel values for the G and B color channels. If the image is an image in the Red-Yellow-Blue (RYB) color space, the image can contain three color channels R, Y, and B. Correspondingly, in this step, R, Y, and B in the image can be extracted. The channel value of the color channel. Specifically, the color space to which the image belongs may be the arrangement of color pixels adopted by the photosensitive element in the industrial camera. For example, in the case of using the RGGB arrangement, an RGB image can be obtained, and in the case of the RYYB arrangement, an RYB image can be obtained.

Sub-step (2): Perform image difference processing respectively according to the values of each color channel of the at least two frames of images.

In this step, you can first determine the corresponding gray value according to the value of each color channel in the image, and then perform differential processing according to the gray value of the image on each color channel according to the time sequence of the image, so as to obtain the corresponding gray value of each color channel. Corresponding difference result. Finally, these difference results can be aggregated, for example, added to obtain a final difference result, and the final difference result can be used as a processing result. Alternatively, the difference result corresponding to each color channel may also be directly used as the processing result, which is not limited in this embodiment of the present invention. Since the brightness change event will cause the gray value change, in the embodiment of the present invention, the difference processing is performed according to the gray value, so that the finally obtained difference result can reflect the brightness change event.

Specifically, when performing image difference processing, the grayscale value of the image with the later timing can be subtracted from the grayscale value of the image with the earlier timing according to the image timing, and the difference between the two can be calculated. For example, it is assumed that at least two frames of images are the first frame image and the third frame image in the data stream collected by the industrial camera, and the difference can be performed by subtracting the first frame image from the third frame image. Or, assuming that at least two frames of images are the first frame image, the second frame image, the third frame image and the fourth frame image in the data stream collected by the industrial camera, the difference can be performed by first comparing the third frame image and the fourth frame image. The 4 frames are added, and then the first frame image and the second frame image are subtracted, that is, the difference result is used to represent the brightness change of the pixels of the last two frames of images relative to the first two frames of images. Of course, other differential algorithms may also be used to implement differential processing, which is not limited in this embodiment of the present invention.

Further, compared to not processing each color channel differently, the overall grayscale value is directly determined according to multiple color channels and differential processing is performed. In the embodiment of the present invention, by extracting the values of each color channel separately, performing differential processing according to the grayscale values of each color channel, and finally summarizing, it is possible to avoid multiple color channels being combined into one channel for processing, resulting in The channels are coupled with each other, and the differential error increases, which in turn can improve the accuracy of differential processing.

It should be noted that, limited by the spectral response of the sensor in the camera and the characteristics of the filter array, the image collected by the industrial camera can also be a grayscale image. Correspondingly, in this case, the difference processing can be performed directly according to the gray value of each pixel in the image, so that the value of each color channel does not need to be extracted separately, the operation of the difference processing can be simplified to a certain extent, and the difference processing can be improved. s efficiency.

Optionally, the difference result may be a difference image, and the value of each pixel in the difference image may be the difference between grayscale values. Further, there is often noise interference in the circuit of the image sensor. For example, there may be noise such as shot noise, thermal noise, and fixed pattern noise that fluctuate over time. And noise may lead to inaccurate gray value, there are errors. In order to improve the accuracy of the difference result, in this embodiment of the present invention, noise filtering processing may also be performed. In an implementation manner, the following step A may be performed after image processing is performed according to at least two frames of images and a processing result is obtained:

Step A: Perform noise filtering processing on the differential image.

Specifically, various types of filters can be introduced in the frequency domain or wavelet domain of the differential image, and noise filtering is performed through the filters. In the embodiment of the present invention, after the image processing is completed, the processing result, that is, the difference image, can be processed, so that the noise interference included in the processing result can be removed, thereby ensuring the accuracy of the difference result.

In another implementation manner, before image processing is performed according to at least two frames of images and a processing result is obtained, noise filtering processing may be implemented through the following steps B to D:

Step B: Obtain m frames of images from the at least two frames of images; the m is a positive integer.

In this embodiment of the present invention, the specific value of m may be determined according to the actual situation. Optionally, the m frames of images may be images that participate in image difference processing among at least two frames of images. That is, the difference processing can be performed once every time m frames of images are acquired. Correspondingly, in this embodiment of the present invention, noise filtering processing may be performed on the m frames of images before each differential processing is performed, that is, noise filtering processing and differential processing are performed alternately, so as to maintain the event output while reducing the noise. Noise interference in the differential processing performed each time, thereby ensuring the effect of the differential processing.

Step C: Perform weighted average processing on the m frames of images to collect noise information in the m frames of images.

In the actual scene, due to noise interference, there will be noise information in each image. In this step, by performing weighted average processing on the m frames of images, the noise information in the m frames of images is collected, which can facilitate the subsequent steps to centrally process the noise information. Specifically, when performing the weighted average processing, the preset weight corresponding to each frame of image may be determined according to the preset weight and the weight distribution rule. For example, it can be assigned in such a way that the lower the image quality, the higher the weight. Next, a weighted sum corresponding to each pixel in the m frames of images may be calculated according to the preset weight corresponding to each frame of the image and the pixel value of each pixel.

Step D, performing noise filtering processing on the noise information.

In the embodiment of the present invention, since the weighted sum corresponding to all the pixels calculated in the above steps collects all the noise information, therefore, in the embodiment of the present invention, the image composed of the weighted sum corresponding to these pixels can be filtered for noise processing, and then realize the noise filtering processing on the noise information. For a specific implementation manner of noise filtering processing, refer to the implementation manner in step A, or, spatial domain filtering may also be performed by using the temporal domain characteristics of m frames of images, which is not limited in this embodiment of the present invention. In an implementation manner, noise filtering may be performed on each frame of image separately. However, it needs to perform m times of noise filtering processing to achieve filtering out noise interference. In the embodiment of the present invention, the noise information in the multi-frame images is collected first, and then the noise information is filtered as a whole. In this way, only one noise filtering process needs to be performed, thereby improving the noise filtering efficiency to a certain extent.

Optionally, the processing result may specifically include a difference image and a difference value corresponding to each pixel in the difference image. Correspondingly, in an implementation manner, the operation of rendering according to the processing result to obtain the event image can be implemented through the following sub-steps (3) to (4):

Sub-step (3): according to the sign of the difference value corresponding to the pixel point in the difference image, determine the rendering color corresponding to the pixel point, and determine the color of the rendering color according to the absolute value of the difference value concentration.

In this embodiment of the present invention, when the difference value is a positive number, the sign of the difference value may be a positive sign, and when the difference value is a negative number, the sign of the difference value may be a negative sign. When the rendering color is determined according to the sign of the difference value, the rendering color corresponding to the sign of the difference value can be searched according to the preset correspondence between the sign and the rendering color, and then the rendering color corresponding to the sign is determined as the rendering color of the pixel. The preset correspondence between symbols and rendering colors may be set according to actual requirements. For example, the rendering color corresponding to the positive sign may be set to be red, and the rendering color corresponding to the negative sign may be set to green. Correspondingly, if the sign of the difference value corresponding to a pixel point is a positive sign, it can be determined that the rendering color corresponding to the pixel point is red; if the sign of the difference value corresponding to a pixel point is a negative sign, it can be determined that the pixel point corresponding to the The rendering color is green.

Further, the absolute value of the difference value can be positively correlated with the color density. When determining the color density according to the absolute value of the difference value, the larger the absolute value, the higher the color density, and the smaller the absolute value, the higher the setting. Low color intensity. For example, the absolute value of the difference value may be used as the input of the first preset generating function, and the output of the first preset function may be used as the color density. Wherein, the first preset function may be a function in which the dependent variable and the independent variable are positively correlated. Alternatively, the absolute value of the difference value can also be negatively correlated with the color density. When determining the color density according to the absolute value of the difference value, the smaller the absolute value, the higher the color density. Low color density, which is not limited in this embodiment of the present invention. Of course, there may be a case where the difference value is 0. In this case, the pixel point may not be rendered, thereby saving processing resources to a certain extent.

Sub-step (4): Render the pixel points according to the rendering color and the color density.

In the embodiment of the present invention, the pixel point may be rendered according to the rendering color and the color density, so that the rendered pixel point presents the rendering color, and the density of the rendered color matches the color density. The rendered differential image is the event image, and the event image may be a matrix image. Since the difference value of a pixel point is often caused by a brightness change event that occurs, in the embodiment of the present invention, the pixel point is rendered into a rendering color corresponding to the sign of the difference value and a color density corresponding to the absolute value, which is obtained after rendering. The demarcation between colors in the event image can indicate the direction of the event's change. For example, the direction with the clearest color demarcation in the event image may be used as the event change direction. The specific change of the density at the boundary can reflect the starting and ending directions of the change direction. For example, when the absolute value of the difference value is positively correlated with the color density, the direction with the clearest color boundary has a lower density. The end of the concentration is used as the starting position, and the end with the higher concentration is used as the ending position. In this way, the brightness change event can be displayed more clearly and clearly through the event image.

In another implementation manner, the operation of rendering according to the processing result to obtain the event image may be implemented through the following sub-steps (5) to (6):

Sub-step (5): Determine the rendering color corresponding to the pixel point according to the corresponding relationship between the preset value and the rendering color and the difference value corresponding to the pixel point in the difference image.

In the embodiment of the present invention, the corresponding relationship between the preset numerical value and the rendering color may be set according to actual requirements. As an example, one value can be set to correspond to one rendering color. Of course, in order to avoid excessive visual burden caused by too many colors, the same rendering color can also be set for values within the same value range. This is not limited.

Specifically, the rendering color corresponding to the difference value may be searched in the corresponding relationship, and the corresponding rendering color may be determined as the rendering color corresponding to the pixel point.

Sub-step (6): Render the pixel points according to the rendering color.

In the embodiment of the present invention, the pixels can be rendered into corresponding rendering colors according to the default color density, so that the uniformity of the color density of the pixels can be ensured, thereby facilitating viewing by the user. Since the difference value of the pixel point is often caused by the occurrence of the brightness change event, in the embodiment of the present invention, the pixel point is rendered as the rendering color corresponding to the difference value, and the color distribution change in the difference image obtained after rendering can be reflected. The direction of the event's change. For example, the direction with the highest regularity of color distribution change may be used as the change direction of the event. In the direction with the highest regularity, the end with a lower degree of regularity is taken as the starting position, and the end with a higher degree of regularity is regarded as the end position. In this way, the brightness change event can be displayed more clearly and clearly through the event image.

It should be noted that, after the event image is generated, the event image can be output through a preset data interface. The image frame rate of the output event image may not be higher than the frame rate of the image frame data stream collected by the industrial camera.

Optionally, the brightness change event in this embodiment of the present invention may be an object motion. Accordingly, after rendering is performed according to the processing result to obtain the event image, the following steps E to G may be performed, so as to realize the execution according to the event detection result. Avoidance:

Step E: Acquire a high dynamic range HDR image according to the at least two frames of images, and determine the moving direction of the object according to the event image.

In the embodiment of the present invention, when the movement direction of the object is determined according to the event image, the event change direction may be read according to the event image, and the event change direction is determined as the movement direction of the object. Specifically, for an implementation manner of determining the direction of an event change, reference may be made to the description in the sub-step (4) or sub-step (6) above, which is not repeated in this embodiment of the present invention.

Further, when obtaining a high dynamic range (High-Dynamic Range, HDR) image according to at least two frames of images, an operation may be performed: according to the gray value of each pixel in the image, calculate each of the at least two frames of images. The sum of the grayscale values of the pixel points; normalize the sum of each of the grayscale values to obtain the HDR image.

Specifically, according to each color channel in the image, the value of each color channel in the image can be read into the operation of calculating the sum of grayscale values. In this way, it is possible to avoid the problem that multiple color channels are combined into one channel for processing, resulting in the mutual coupling of the channels, which increases the calculation error, thereby improving the calculation accuracy. For the specific way of reading in separately, please refer to the relevant descriptions in the above sub-steps (1) to (2). Of course, this step can also be directly multiplexed and read separately when executing sub-steps (1) to (2). The value of each color channel entered is not limited in this embodiment of the present invention. Of course, when the image captured by the industrial camera itself is a grayscale image, the sum of the grayscale values can be directly performed according to the grayscale values of each pixel in the image to improve the calculation efficiency.

When calculating the sum of grayscale values, the integration operation may be performed on the preceding and following frame images in the at least two frame images according to the time sequence of the images, that is, the sum of the grayscale values of the pixels in the preceding and following frame images is calculated. Specifically, the sum of the gray values on each color channel can be calculated, and finally the sum of the gray values on the color channels is used as the sum of the gray values of the pixels.

Because the sum of grayscale values after adding multiple frames of images may exceed the bit depth and display range adopted by common display devices. Therefore, in this embodiment of the present invention, the sum of gray values can be normalized through a normalization operation, so that the HDR image can be displayed normally by a common display device, thereby improving the applicable range of the HDR image. For example, the bit depth used by a common display device may be 8 bits (bit). Correspondingly, in the embodiment of the present invention, the sum of gray values may be normalized to an 8-bit bit depth in the numerical range of 0-255. Further, the bit depth of the image can also be increased according to the number of frames of the image participating in the calculation, so as to ensure that the sum of the calculated gray values has enough bit depth for use. For example, the bit depth can be increased to 10-bit when the number of frames of the image involved in the calculation is 4. For example, an 8-bit image with a grayscale value in the range of 0-255 will obtain a 10-bit image in the range of 0-1023 through the integration of 4 frames of images. By increasing it to 10bit, it can ensure that the bit depth can meet the needs of use. need.

In practical applications, the noise of an image fluctuates randomly, while weak signals in an image, such as darker parts, are relatively stable. Therefore, in the embodiment of the present invention, after the sum operation of multiple frames of images, the increase of the noise intensity will be lower than the increase of the weak signal, so that the darker part of the image can be enhanced from the noise, and the signal-to-noise ratio in the dark area can be improved. , which in turn provides more imaging details. Further, when there are high-brightness and low-brightness areas in the imaging scene at the same time, due to the random fluctuation of noise, it is possible to achieve enhanced perception of low-brightness areas on the premise of keeping high-brightness areas without overexposure to a certain extent. , and then realize HDR imaging, that is, obtain an HDR image. When the industrial camera used is a high-speed industrial camera, since the exposure time of the high-speed industrial camera is relatively short and the number of photons collected per unit time is small, the summation calculation method can also improve the image signal-to-noise ratio to a certain extent. .

It should be noted that, after the generated HDR image, the HDR image can be output through the preset data interface for use in subsequent steps. The output image may be a color image, which can help determine the material and texture information of the object, thereby improving the efficiency of subsequent image recognition. Further, there may be some noise interference in the HDR image. In order to improve the image quality of the HDR image, the HDR image may also be subjected to noise filtering processing. For specific noise filtering methods, refer to the noise filtering processing performed in the above-mentioned related steps, for example, spatial domain filtering, frequency domain filtering, and wavelet domain filtering can be adopted.

Step F: Identify the position information of the object in the external environment according to the HDR image.

In this embodiment of the present invention, the position information of an object in the external environment may be the coordinates of the object in the external environment, or may be the relative positional relationship, relative distance, etc. between the object and other objects in the external environment. Specifically, a preset image recognition algorithm may be used to recognize the HDR image, for example, operations such as target object recognition, image semantic segmentation, etc. may be performed to determine the location information. It should be noted that, in order to improve the accuracy of recognition, the embodiment of the present invention may also compare the features of the recognized object with the features of the event in the event image, and if the two match, it is confirmed that the object is a moving object , and then further determine the position information of the object.

Step G: Perform obstacle avoidance on the object according to the movement direction and the position information.

In the embodiment of the present invention, the movement change that will occur after the object can be estimated according to the movement direction and position information, and the obstacle avoidance strategy is determined according to the movement change. For example, assuming that the movement direction is horizontal movement from left to right, the object is currently in the middle position, and will then move to the far right of the viewing range, then the device can be controlled to move away from the right position to avoid the object.

In scenes with drastic changes in light intensity, such as entering and exiting tunnels, and switching between indoor and outdoor, when avoiding obstacles for moving objects, if obstacles are avoided only by analyzing the images collected by industrial cameras, due to the dynamic nature of the images Insufficient range can cause image analysis to fail, resulting in poor obstacle avoidance. In the embodiment of the present invention, the HDR image is further generated according to the image collected by the industrial camera, and the moving direction of the object is determined in combination with the event image. In the method of obstacle avoidance, since the HDR image has a high dynamic range, it can provide sufficient information for image recognition and analysis. information, which can ensure that the position information of the object can be identified, thereby ensuring the effect of obstacle avoidance.

Optionally, when there is a moving object in the scene, the position of the object on the front and rear frame images may change, so that motion blur may be formed in the HDR image obtained by the operation to a certain extent. Therefore, in this embodiment of the present invention, after acquiring the HDR image according to at least two frames of images, the following steps may be performed:

Step H: Determine a motion blur kernel according to the motion information of the object in the event image.

In this embodiment of the present invention, the motion information may be related information of pixels in the direction of event change. Since the blurring is caused by the motion of the object, the blurring of the image can also be called image degradation. Therefore, these pixel points can be considered as points causing degradation. The motion blur kernel can be a convolution kernel used to describe the motion trajectory of these pixels.

Optionally, when the motion blur kernel is determined, a motion blur model may be estimated first according to the motion information of the object in the event image. For example, since there is motion blur in the HDR image, the HDR image may be used as the degraded image, the image used for generating the HDR image may be used as the original clear image, and the motion information may be used as the input of the preset degraded point spread function. The image degradation model is constructed according to the degraded image, the original clear image and the degraded point spread function, and the final image degradation model is used as the motion blur model.

Next, the motion blur kernel may be estimated according to the motion blur model. The motion blur kernel may be a space-invariant blur kernel or a space-variant blur kernel. Specifically, the space-invariant blur kernel assumes that the overall blur degree of the HDR image is the same, and when used, it is equivalent to performing an overall deblurring operation on the image. The space-invariant blur kernel is a global calculation. When the motion blur kernel is a space-invariant blur kernel, the overall estimation algorithm can be used according to the motion blur model. Principal component analysis and other overall estimation algorithms to determine the motion blur kernel. Since the operation of the overall estimation is relatively simple, the use of a space-invariant blur kernel can improve the computational efficiency to a certain extent. However, since it is equivalent to performing an overall deblurring operation on the image, it may cause defects such as ringing effects, which in turn lead to poor deblurring effects.

In the case where the motion blur kernel is a spatially variable blur kernel, the motion blur kernel can be determined by a local estimation algorithm according to the motion blur model. The spatial variation blur kernel assumes that the object motion in the three-dimensional space has different blurring degrees on the image. The spatial variation blur kernel can be used to reflect the difference of this blurring degree in the local area. Therefore, it can often get better deblurring when using it. Effect. However, the computational complexity of local estimation is high. Therefore, the specific use of the space-invariant blur kernel or the space-varying blur kernel can be selected according to actual needs.

Step 1. Perform deblurring on the HDR image according to the motion blur check.

In this embodiment of the present invention, a motion blur kernel may be used to perform a deconvolution operation on the HDR image, thereby implementing deblurring processing.

Of course, deblurring can also be implemented in other ways. Illustratively, the image convolution kernel for motion blur can be directly estimated based on the prior knowledge of the image, for example, the statistical correlation of the previous and subsequent frame images and other data features, as well as the preset convolution kernel function, and finally the image convolution kernel is used to check the HDR. The image is deconvolved.

Further, in this embodiment of the present invention, after acquiring at least two frames of images collected by an industrial camera, a grayscale image or a color image corresponding to the at least two frames of images may be output. For example, a grayscale image may be output when the collected image is a grayscale image, and a color image may be output when the collected image is a color image. Among them, the color map refers to a map belonging to a color space, such as an RGB map. When outputting an image, it can be output through a preset data interface. In the scenario of continuously acquiring the images in the image frame data stream collected by the industrial camera, the images in the data stream can be continuously output. The output image frame rate may be the same as the frame rate of the image frame data stream.

Illustratively, taking the brightness change event as a motion event as an example, FIG. 2 is a schematic diagram of an output comparison provided by an embodiment of the present invention. Among them, for the brightness change event caused by object motion or other factors, the event camera can output the event at a higher time sampling frequency, but can only output the event and cannot output the image. Illustratively, the output of the event camera can be as shown in the first line. In the embodiment of the present invention, using an industrial camera, that is, a classic imaging camera, can output the image frames collected at each moment shown in the second row, and determine the event image through inter-frame operations, and output the event image by outputting the event image (only shown in FIG. 2 ). out the event change direction in the event image) to realize the event output.

Optionally, the frame rate of the industrial camera in the embodiment of the present invention may be not less than 100 frames per second (frame per second, fps), that is, the industrial camera may be a high-speed industrial camera. In this way, by using an industrial camera with a frame rate of not less than 100, the problems of poor real-time detection and low practicability due to too low frame rate can be avoided. For example, in a practical application scenario, for example, in a common robot application scenario, when the robot moves at a low speed, the frame rate of the perception of the environment is similar to that of the human eye, about 20fps. Although the time sampling frequency of the event camera can reach 1 kilohertz (kHz) or even 100 kHz, which meets the application requirements of robots to a large extent, it only provides event information. However, in the embodiment of the present invention, an industrial camera with a frame rate greater than 100 fps is used, and the inter-frame operation is used to realize the event output satisfying the application of the robot, and at the same time, information such as images and HDR images can be further provided, thereby providing more comprehensive information for the robot perception. Live image data.

Further, FIG. 3 is a schematic diagram of an output provided by an embodiment of the present invention. As shown in FIG. 3 , the embodiment of the present invention can directly output a grayscale image or a color image. Alternatively, the HDR image and the event image are output through the inter-frame operation (only the event change direction is shown in FIG. 3 ).

FIG. 4 is a schematic diagram of a processing process of a specific example provided by an embodiment of the present invention. As shown in Figure 4, for the image frame data stream output by the industrial camera, the image output type can be selected according to actual needs. In the case where direct output is selected, the original image, that is, the image in the image frame data stream, can be directly output. In the case of selecting the output event image, the operation of reading in the raw data by channel may be performed first, that is, the operation shown in the sub-step (1) above. Then, after performing the inter-frame difference operation and the noise filtering process, the operation of visualizing the difference numerical value symbol, that is, the rendering operation shown in the above steps is performed. Finally, the event image can be output. In the case of selecting the output HDR image, the operation of reading in the raw data of the channels may be performed first, and then the inter-frame integration operation, that is, the operation of acquiring the HDR image shown in the above step E may be performed. Next, noise filtering and deblurring may be performed, wherein deblurring may be performed in conjunction with the event image. Finally, an HDR image can be output.

Further, compared with the method of using the event camera, since the event camera is only sensitive to the change value of the light intensity and is not sensitive to the light intensity distribution, the output information is often sparse, and the image cannot be output. In the embodiment of the present invention, an HDR image can be generated and output, and an image collected by an industrial camera can be output, thereby improving the richness of output data. At the same time, compared with event cameras, industrial cameras have higher compatibility with various computing platforms, smaller pixel sizes, larger fill factors and larger array scales. Therefore, higher-resolution images can be output to further improve image information. richness. Illustratively, through the design of signal processing transmission circuits, industrial cameras can achieve high acquisition and transmission speeds (eg, using 5 switching bandwidth (Gbps) to achieve speeds of about 200 million pixels/sec), in high-speed imaging modes greater than 200fps The Video Graphics Array (VGA) in the camera can output images at higher image resolutions such as 720p.

It should be noted that the image processing process in the embodiment of the present invention may be a field programmable gate array (Field Programmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU) carried in the event detection device. , Graphics Processing Unit (GPU), embedded neural network processor (Neural-network Processing Unit, NPU) and other high-speed image computing processing units. The specific number of images used when generating the event image may be determined according to the acquisition and data transmission speed of the industrial camera, the computing power of the processor, and the actual output frame rate requirements of the event image. Not limited.

FIG. 5 is a block diagram of an event detection apparatus provided by an embodiment of the present invention. The apparatus 50 may include: a memory 501 and a processor 502 .

The memory 501 is used to store program codes.

The processor 502 calls the program code, and when the program code is executed, is configured to perform the following operations:

Acquire at least two frames of images captured by an industrial camera;

Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

The event image is output.

Optionally, the processor 502 is specifically configured to:

For each frame of the image, obtain the value of each color channel of the image respectively;

Image difference processing is performed respectively according to the values of the respective color channels of the at least two frames of images.

Optionally, the processing result includes a difference image; the processor 502 is further configured to: perform noise filtering processing on the difference image.

Optionally, the processor 502 is further configured to: acquire m frames of images from the at least two frames of images; the m is a positive integer; and perform weighted average processing on the m frames of images to collect the m frames Noise information in the image; perform noise filtering processing on the noise information.

Optionally, the brightness change event is movement of an object; the processor 502 is further configured to: acquire a high dynamic range HDR image according to the at least two frames of images, and determine the movement direction of the object according to the event image ; Recognize the position information of the object in the external environment according to the HDR image; perform obstacle avoidance for the object according to the movement direction and the position information.

Optionally, the processor 502 is further configured to: determine a motion blur kernel according to the motion information of the object in the event image; and perform deblurring processing on the HDR image according to the motion blur kernel.

Optionally, the processor 502 is specifically configured to: estimate a motion blur model according to the motion information of the object in the event image; estimate the motion blur kernel according to the motion blur model; wherein, the motion The blur kernel is a space-invariant blur kernel or a space-variant blur kernel.

Optionally, the processor 502 is specifically configured to: calculate the sum of the grayscale values of each pixel point in the at least two frames of images according to the grayscale value of each pixel point in the image; The sum of the values is normalized to obtain the HDR image.

Optionally, the processing result includes a difference image and a difference value corresponding to each pixel in the difference image; the processor 502 is specifically configured to: according to the difference value corresponding to the pixel in the difference image , determine the rendering color corresponding to the pixel point, and determine the color density of the rendering color according to the absolute value of the difference value; and render the pixel point according to the rendering color and the color density. Optionally, the processing result includes a difference image and a difference value corresponding to each of the pixels in the difference image; the processor 502 is specifically configured to: according to the preset value and the corresponding relationship between the rendering color and the corresponding difference value. The difference value corresponding to the pixel point in the difference image determines the rendering color corresponding to the pixel point; the pixel point is rendered according to the rendering color.

Optionally, the frame rate of the industrial camera is not less than 100 frames per second. Optionally, the processor 502 is further configured to: output a grayscale map or a color map corresponding to the at least two frames of images. To sum up, the event detection device provided by the embodiment of the present invention can acquire at least two frames of images collected by an industrial camera, perform image processing according to the at least two frames of images, and obtain a processing result, wherein the processing result is used to characterize the pixels in the image The brightness change of the point is then rendered according to the processing result to obtain an event image, wherein the event image is used to represent the brightness change event, and finally, the event image is output. In this way, event detection can be realized directly based on images collected by conventional industrial cameras without the need to specially set up an event camera, so the detection cost can be saved to a certain extent.

Further, an embodiment of the present invention further provides a movable platform, where the movable platform includes an industrial camera and any of the above-mentioned event detection devices; the event detection device is configured to execute each step in the above-mentioned event detection method, And can achieve the same technical effect, in order to avoid repetition, it is not repeated here.

Optionally, the movable platform includes a powered propeller and a drive motor for driving the powered propeller.

Further, an embodiment of the present invention also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, each step in the above-mentioned event detection method is implemented, and can To achieve the same technical effect, in order to avoid repetition, details are not repeated here.

6 is a schematic diagram of the hardware structure of a device implementing various embodiments of the present invention. The device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, User input unit 607 , interface unit 608 , memory 609 , processor 610 , and power supply 611 and other components. Those skilled in the art can understand that the device structure shown in FIG. 6 does not constitute a limitation on the device, and the device may include more or less components than the one shown, or combine some components, or arrange different components. In this embodiment of the present invention, the devices include but are not limited to mobile phones, tablet computers, notebook computers, handheld computers, vehicle-mounted devices, wearable devices, and pedometers. It should be understood that, in this embodiment of the present invention, the radio frequency unit 601 may be used for receiving and sending signals during sending and receiving of information or during a call. Specifically, after receiving the downlink data from the base station, it is processed by the processor 610; The uplink data is sent to the base station. Generally, the radio frequency unit 601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 601 can also communicate with the network and other devices through a wireless communication system. The device provides the user with wireless broadband Internet access through the network module 602, such as helping the user to send and receive emails, browse web pages, access streaming media, and the like. The audio output unit 603 may convert audio data received by the radio frequency unit 601 or the network module 602 or stored in the memory 609 into audio signals and output as sound. Also, the audio output unit 603 may also provide audio output related to a specific function performed by the device 600 (eg, call signal reception sound, message reception sound, etc.). The audio output unit 603 includes a speaker, a buzzer, a receiver, and the like. The input unit 604 is used to receive audio or video signals. The input unit 604 may include a graphics processor (Graphics Processing Unit, GPU) 6041 and a microphone 6042, and the graphics processor 6041 is used for still pictures or video images obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode data is processed. The processed image frames may be displayed on the display unit 606 . The image frames processed by the graphics processor 6041 may be stored in the memory 609 (or other storage medium) or transmitted via the radio frequency unit 601 or the network module 602 . The microphone 6042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 601 for output in the case of a telephone call mode. Device 600 also includes at least one sensor 605, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 6061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 6061 and/or when the device 600 is moved to the ear. or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes), and can detect the magnitude and direction of gravity when stationary, and can be used to identify the device posture (such as horizontal and vertical screen switching, related games, The sensor 605 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared Sensors, etc., will not be repeated here. The display unit 606 is used to display information input by the user or information provided to the user. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. The user input unit 607 may be used to receive input numerical or character information, and generate key signal input related to user settings and function control of the device. Specifically, the user input unit 607 includes a touch panel 6041 and other input devices 6072 . The touch panel 6041, also referred to as a touch screen, can collect the user's touch operations on or near it (such as the user's finger, stylus, etc., any suitable object or accessory on or near the touch panel 6041). operate). The touch panel 6041 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 610, the command sent by the processor 610 is received and executed. In addition, the touch panel 6041 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 6041 , the user input unit 607 may also include other input devices 6072 . Specifically, other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described herein again. Further, the touch panel 6041 can be covered on the display panel 6061. When the touch panel 6041 detects a touch operation on or near it, it transmits it to the processor 610 to determine the type of the touch event, and then the processor 610 determines the type of the touch event according to the touch The type of event provides a corresponding visual output on the display panel 6061. Although the touch panel 6041 and the display panel 6061 are used as two independent components to realize the input and output functions of the device, in some embodiments, the touch panel 6041 and the display panel 6061 can be integrated to realize the input and output functions of the device. The output function is not limited here. The interface unit 608 is an interface for connecting an external device to the device 600 . For example, external devices may include wired or wireless headset ports, external power (or battery charger) ports, wired or wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input/output (I/O) ports, video I/O ports, headphone ports, and more. Interface unit 608 may be used to receive input (eg, data information, power, etc.) from an external device and transmit the received input to one or more elements within device 600 or may be used between device 600 and an external device. transfer data between.

The memory 609 may be used to store software programs as well as various data. The memory 609 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data created by the use of the mobile phone (such as audio data, phone book, etc.), etc. Additionally, memory 609 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The processor 610 is the control center of the device, uses various interfaces and lines to connect various parts of the entire device, and executes by running or executing the software programs and/or modules stored in the memory 609, and calling the data stored in the memory 609. Various functions of the equipment and processing data, so as to monitor the equipment as a whole. The processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem The processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 610. The device 600 may also include a power supply 611 (such as a battery) for supplying power to various components. Preferably, the power supply 611 may be logically connected to the processor 610 through a power management system, so as to manage charging, discharging, and power consumption management through the power management system. Features. In addition, the device 600 includes some unshown functional modules, which will not be repeated here. The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor may be used in practice to implement some or all of the functions of some or all of the components in the computing processing device according to the embodiments of the present invention. The present invention can also be implemented as apparatus or apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form. For example, FIG. 7 is a block diagram of a computing and processing device provided by an embodiment of the present invention. As shown in FIG. 7 , FIG. 7 shows a computing and processing device that can implement the method according to the present invention. The computing processing device traditionally includes a processor 710 and a computer program product or computer readable medium in the form of a memory 720 . The memory 720 may be electronic memory such as flash memory, EEPROM (electrically erasable programmable read only memory), EPROM, hard disk, or ROM. The memory 720 has storage space 730 for program code for performing any of the method steps in the above-described methods. For example, the storage space 730 for program codes may include various program codes for implementing various steps in the above methods, respectively. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are typically portable or fixed storage units as described with reference to FIG. 8 . The storage unit may have storage segments, storage spaces, etc. arranged similarly to the memory 720 in the computing processing device of FIG. 7 . The program code may, for example, be compressed in a suitable form. Typically, the storage unit includes computer readable code, ie code readable by a processor such as 710 for example, which when executed by a computing processing device, causes the computing processing device to perform each of the methods described above. step. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments may be referred to each other. Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Also, please note that instances of the phrase "in one embodiment" herein are not necessarily all referring to the same embodiment. In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (26)

1. An event detection method, characterized in that the method comprises:

   Acquire at least two frames of images captured by an industrial camera;

   Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

   Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

   The event image is output.
2. The method according to claim 1, wherein the performing image processing according to the at least two frames of images comprises:

   For each frame of the image, obtain the value of each color channel of the image respectively;

   Image difference processing is performed respectively according to the values of the respective color channels of the at least two frames of images.
3. The method according to claim 1, wherein the processing result comprises a differential image; after performing image processing according to the at least two frames of images to obtain a processing result, the method further comprises:

   Perform noise filtering processing on the difference image.
4. The method according to claim 1, wherein, before the image processing is performed according to the at least two frames of images and a processing result is obtained, the method further comprises:

   Obtain m frames of images from the at least two frames of images; the m is a positive integer;

   performing weighted average processing on the m frames of images to collect noise information in the m frames of images;

   Perform noise filtering processing on the noise information.
5. The method according to any one of claims 1 to 4, wherein the brightness change event is object motion; after the rendering according to the processing result to obtain the event image, the method further comprises:

   Acquiring a high dynamic range HDR image according to the at least two frames of images, and determining the movement direction of the object according to the event image;

   Identify the position information of the object in the external environment according to the HDR image;

   According to the movement direction and the position information, the object is avoided from obstacles.
6. The method according to claim 5, wherein after acquiring the high dynamic range HDR image according to the at least two frames of images, the method further comprises:

   determining a motion blur kernel according to the motion information of the object in the event image;

   The HDR image is deblurred according to the motion blur check.
7. The method according to claim 6, wherein the determining a motion blur kernel according to the motion information of the object in the event image comprises:

   Estimating a motion blur model according to the motion information of the object in the event image;

   estimating the motion blur kernel according to the motion blur model;

   Wherein, the motion blur kernel is a space-invariant blur kernel or a space-variant blur kernel.
8. The method according to claim 5, wherein the acquiring a high dynamic range HDR image according to the at least two frames of images comprises:

   According to the gray value of each pixel in the image, calculate the sum of the gray value of each pixel in the at least two frames of images;

   The sum of each of the gray values is normalized to obtain the HDR image.
9. The method according to claim 1, wherein the processing result comprises a difference image and a difference value corresponding to each of the pixel points in the difference image;

   The rendering according to the processing result to obtain the event image includes:

   Determine the rendering color corresponding to the pixel point according to the sign of the difference value corresponding to the pixel point in the difference image, and determine the color density of the rendering color according to the absolute value of the difference value;

   The pixel points are rendered according to the rendering color and the color density.
10. The method according to claim 1, wherein the processing result comprises a difference image and a difference value corresponding to each of the pixel points in the difference image;

    The rendering according to the processing result to obtain the event image includes:

    Determine the rendering color corresponding to the pixel point according to the corresponding relationship between the preset value and the rendering color and the difference value corresponding to the pixel point in the difference image;

    The pixel points are rendered according to the rendering color.
11. The method according to claim 1, wherein the frame rate of the industrial camera is not less than 100 frames per second.
12. The method according to claim 1, wherein the method further comprises:

    A grayscale map or a color map corresponding to the at least two frames of images is output.
13. An event detection device, characterized in that the device includes a memory and a processor;

    the memory for storing program codes;

    The processor calls the program code, and when the program code is executed, is configured to perform the following operations:

    Acquire at least two frames of images captured by an industrial camera;

    Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

    Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

    The event image is output.
14. The apparatus according to claim 13, wherein the processor is specifically configured to:

    For each frame of the image, obtain the value of each color channel of the image respectively;

    Image difference processing is performed respectively according to the values of the respective color channels of the at least two frames of images.
15. The apparatus according to claim 13, wherein the processing result comprises a differential image; the processor is further configured to:

    Perform noise filtering processing on the difference image.
16. The apparatus according to claim 13, wherein the processor is further configured to:

    Obtain m frames of images from the at least two frames of images; the m is a positive integer;

    performing weighted average processing on the m frames of images to collect noise information in the m frames of images;

    Perform noise filtering processing on the noise information.
17. The device according to any one of claims 13 to 16, wherein the brightness change event is object motion; the processor is further configured to:

    Acquiring a high dynamic range HDR image according to the at least two frames of images, and determining the movement direction of the object according to the event image;

    Identify the position information of the object in the external environment according to the HDR image;

    According to the movement direction and the position information, the object is avoided from obstacles.
18. The apparatus of claim 17, wherein the processor is further configured to:

    determining a motion blur kernel according to the motion information of the object in the event image;

    The HDR image is deblurred according to the motion blur check.
19. The apparatus according to claim 18, wherein the processor is specifically configured to:

    Estimating a motion blur model according to the motion information of the object in the event image;

    estimating the motion blur kernel according to the motion blur model;

    Wherein, the motion blur kernel is a space-invariant blur kernel or a space-variant blur kernel.
20. The apparatus according to claim 17, wherein the processor is specifically configured to:

    According to the gray value of each pixel in the image, calculate the sum of the gray value of each pixel in the at least two frames of images;

    The sum of each of the gray values is normalized to obtain the HDR image.
21. The device according to claim 13, wherein the processing result comprises a difference image and a difference value corresponding to each of the pixel points in the difference image;

    The processor is specifically used for:

    Determine the rendering color corresponding to the pixel point according to the sign of the difference value corresponding to the pixel point in the difference image, and determine the color density of the rendering color according to the absolute value of the difference value;

    The pixel points are rendered according to the rendering color and the color density.
22. The device according to claim 13, wherein the processing result comprises a difference image and a difference value corresponding to each of the pixel points in the difference image;

    The processor is specifically used for:

    Determine the rendering color corresponding to the pixel point according to the corresponding relationship between the preset value and the rendering color and the difference value corresponding to the pixel point in the difference image;

    The pixel points are rendered according to the rendering color.
23. The device according to claim 13, wherein the frame rate of the industrial camera is not less than 100 frames per second.
24. The apparatus according to claim 13, wherein the processor is further configured to:

    A grayscale map or a color map corresponding to the at least two frames of images is output.
25. A movable platform, characterized in that the movable platform comprises an industrial camera and the event detection device according to any one of claims 13 to 24; the event detection device is used to perform the following operations:

    Acquire at least two frames of images captured by an industrial camera;

    Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

    Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

    The event image is output.
26. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the following operations are implemented:

    Acquire at least two frames of images captured by an industrial camera;

    Perform image processing according to the at least two frames of images to obtain a processing result; the processing result is used to characterize the brightness change of the pixels in the image;

    Rendering according to the processing result to obtain an event image; the event image is used to represent a brightness change event;

    The event image is output.

Images