WO2021102928A1

WO2021102928A1 - Image processing method and apparatus

Info

Publication number: WO2021102928A1
Application number: PCT/CN2019/122034
Authority: WO
Inventors: 张青涛
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-11-29
Filing date: 2019-11-29
Publication date: 2021-06-03

Abstract

An image processing method and apparatus. The method comprises: recognizing a target object from an image in an infrared video frame; performing image enhancement processing on the image on the basis of the target object, the image enhancement processing comprising one or more processing of histogram statistics and stretching, medium and high frequency detail enhancement or pseudo-color mapping, and obtaining an image subjected to the image enhancement processing, so that the image subjected to the image enhancement processing can be outputted. Hence, according to the embodiments of the present application, the image enhancement processing can be performed on the image on the basis of the target object, local details of the image can be more accurately processed, thereby facilitating better displaying the details and content of the target object.

Description

Image processing method and device

Technical field

This application relates to the field of image processing technology, and in particular to an image processing method and device.

Background technique

Images are the main source for humans to obtain and exchange information, and the application fields of image processing technology also involve all aspects of human life and work.

As people's requirements for image recognizability increase, how to process images so that the quality of the images can be greatly improved is becoming more and more important in the field of image processing. For example, the histogram of the image output by the infrared sensor is often very concentrated, and other image processing operations need to be performed on the image in order to display the scene content in the image.

However, this image processing method still cannot better present the image content that the user is interested in.

Summary of the invention

The present application provides an image processing method and device, which can better present the image content that users are interested in, and can more accurately process the local details of the image.

In the first aspect, an embodiment of the present application provides an image processing method, and the method includes:

Identify the target object from the image in the infrared video frame;

Performing image enhancement processing on the image based on the target object;

An image after image enhancement processing is output, and the image enhancement processing includes one or more processing of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo-color mapping.

In a second aspect, an embodiment of the present application provides an image processing device, the image processing device includes: a processor and a memory;

Program instructions are stored in the memory;

The processor calls the program instructions for

Identify the target object from the image in the infrared video frame;

In a third aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored in the computer-readable storage medium, and when the computer program instructions are executed by a processor, they are used to execute the above-mentioned first aspect. Image processing method.

The embodiment of the application can identify the target object from the image in the infrared video frame, and perform image enhancement processing on the image in the infrared video frame based on the target object. Compared with the global processing of the image, the image can be processed more accurately. Partial details, in turn, help to show the image content that users are interested in.

Description of the drawings

FIG. 1a is a schematic diagram of an image in an infrared video frame provided by an embodiment of the present application without image enhancement processing;

FIG. 1b is a schematic diagram of an image in the infrared video frame shown in FIG. 1a after image enhancement processing provided by an embodiment of the present application;

Fig. 2 is a schematic diagram of an image processing system provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of identifying the contour shape of a target object according to an embodiment of the present application;

FIG. 5 is a schematic diagram of temperature distribution of an image provided by an embodiment of the present application;

Fig. 6a is a schematic diagram of histogram statistics of an image provided by an embodiment of the present application;

FIG. 6b is another schematic diagram of histogram statistics of an image provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of an image processing method provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of another image processing method provided by an embodiment of the present application;

FIG. 9 is a schematic flowchart of another image processing method provided by an embodiment of the present application;

Fig. 10 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

The embodiments of the present application provide an image processing method that can identify a target object from an image in an infrared video frame; perform image enhancement processing on the image based on the target object; output the image enhanced by the image so that the image The details of the area where the target object is located are more obvious.

Taking Fig. 1a and Fig. 1b as examples, Fig. 1a is a schematic diagram of an image in an infrared video frame provided by an embodiment of the present application without image enhancement processing, and Fig. 1b is an image in the infrared video frame shown in Fig. 1a after image enhancement Schematic diagram after processing. The target object 11 is included in FIGS. 1a and 1b. Figure 1a is subjected to image enhancement processing to obtain Figure 1b, where the image enhancement processing refers to performing image enhancement processing on local details in the image. The image shown in FIG. 1b can show the details of the target object 11. For example, in FIG. 1b, the clothes of the target object 11 may display the number of buttons, and the handbag carried by the target object 11 may display wrinkles and the brand label of the handbag.

In the embodiment of the present application, the operation of image enhancement processing may include one or more of histogram statistics and stretching, medium and high frequency detail enhancement, and pseudo-color mapping. Among them, histogram statistics and stretching are methods of adjusting the gray value of the image by improving the number of pixels with the same gray value (or the frequency of the pixels with the same gray value) in the image. Medium and high frequency detail enhancement is an image processing method that removes the low frequency part of the image through a filter and highlights the high frequency part. Pseudo-color mapping is a method of assigning color values to grayscale values based on certain criteria.

In the embodiments of the present application, histogram statistics and stretching can be referred to as a contrast stretching algorithm; medium and high frequency detail enhancement can be referred to as a detail enhancement algorithm; pseudo-color mapping can be referred to as a pseudo-color enhancement.

The image processing methods mentioned in the embodiments of this application can be applied to unmanned aerial vehicles, unmanned vehicles, or mobile terminals. Mobile terminals may include: mobile phones, notebooks, tablet computers, and so on.

The image processing method described in the embodiment of the present application can be applied to various image processing systems. For example, the image processing system may include control equipment and a camera. Optionally, the camera may be located on a movable platform. The movable platform may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, etc. The machine is described as an example. The camera may be an infrared camera for collecting infrared images. An infrared camera can also be called an infrared thermal imager, an infrared sensor, and so on. The camera transmits the acquired infrared image to the control device. For example, the camera can transmit the collected infrared image to the control device through a drone, or a communication module is installed on the camera to directly transmit the collected infrared image to the control device. In this case, the control device performs the related operations of the image processing method described in this application, and can obtain and output the processed image. The control device may be, for example, a remote control of a drone, a ground station, a terminal or a server, etc.

Another optional implementation is that the camera transmits the acquired infrared image to the drone, and the processor of the drone performs related operations of the image processing method described in this application to obtain and output the processed image. The image processing process in this case will be described below in conjunction with FIG. 2. Please refer to FIG. 2. FIG. 2 is a schematic diagram of an image processing system provided by an embodiment of the present application. The image processing system includes a drone and a control device. The drone is equipped with a camera, and the camera may include an infrared sensor. The infrared sensor is used to shoot infrared video frames. The drone can perform the image processing method mentioned in the embodiment of this application on the image in the infrared video frame to obtain an image after image enhancement processing. Furthermore, the UAV can send the image after image enhancement processing to the control device for display. Correspondingly, the control device can display the image after the image enhancement processing. For example, the control device may be a drone remote control or a terminal, and the remote control or the display screen on the terminal may display the image after the image enhancement processing.

Please refer to FIG. 3, which is a schematic flowchart of an image processing method provided by an embodiment of the present application. As shown in Figure 3, the method may include the following steps.

S301: Identify the target object from the image in the infrared video frame.

In an optional implementation manner, the infrared video frame may be an infrared video frame output by an infrared sensor.

In an optional implementation manner, step S301 may include: identifying the target object in the image according to the contour shape of the target object in the image and the temperature distribution of the image.

The method for obtaining the contour shape of the target object in the image may be: using object recognition and/or semantic segmentation algorithms to obtain the contour shape of the target object in the image. Among them, a characteristic triangle can be formed based on the contour points of the target object, and the contour information of the target object can be obtained according to the area and the characteristic angle of the characteristic triangle.

For example, FIG. 4 is a schematic diagram of identifying the contour shape of a target object according to an embodiment of the application. In the target object 400 shown in FIG. 4, the vertex 401, the vertex 402, and the vertex 403 are three contours of the target object 400. Point, the three contour points can form a characteristic triangle. The centroid of the characteristic triangle is the mark 404. The contour information of the target object is obtained by calculating the area and characteristic angle of the characteristic triangle, and the contour shape of the target object can be determined based on the contour information of the target object.

The method of obtaining the temperature distribution of the image may be: converting the pixel value of each pixel in the image into a temperature value, and the temperature distribution of the image can be obtained from the temperature value of each pixel. Generally, the temperature difference on the same object is not big, and the temperature difference between different objects is big. As shown in FIG. 5, FIG. 5 is a schematic diagram of the temperature distribution of an image provided by an embodiment of the present application.

S302: Perform image enhancement processing on the image based on the target object.

S303: Output an image after image enhancement processing.

In an optional implementation manner, step S302 may include: performing histogram statistics on the pixels in the area where the target object is located in the image to obtain statistical results; and performing stretching processing on the statistical results to obtain the stretched image . Among them, the pixels of the area where the target object is located can also be referred to as the pixels of the target object. When performing histogram statistics, statistics can be performed on the target object instead of global statistics, so that when the histogram is stretched, the contrast of the target object can be better displayed and the processing accuracy of the local details in the image can be improved. In this embodiment of the application, the target object may be a region of interest (ROI). The region of interest can be preset by the system or specified by the user during actual use. For example, the target object may be a human body, electric tower, sky, etc.

Among them, performing histogram statistics on the pixels in the area where the target object in the image is located refers to counting the occurrence frequency of each gray value according to the size of the gray value of the pixels in the area where the target object is located in the image. Stretching the statistical result refers to stretching the statistical result so that the gray values of the pixels in the area where the target object is located are distributed in the entire gray value range.

For example, please refer to FIGS. 6a and 6b. FIG. 6a is a schematic diagram of histogram statistics of an image provided by an embodiment of the present application, and FIG. 6b is another schematic diagram of histogram statistics of an image provided by an embodiment of the present application. In Fig. 6a, the dynamic range of the gray value of the pixel is small, only concentrated in the gray value range 1 to 2, and other areas are blank or have few pixels, resulting in the loss of image details. The gray value distribution range of the pixel in FIG. 6b is larger than the gray value distribution range of the pixel in FIG. 6a, which can reflect more image details.

In another optional implementation manner, step S302 may include: performing histogram statistics for the pixels in the area where the target object is located and the pixels in the area where the non-target object is located in the image, respectively, to obtain the first statistical result and the second statistical result. Result; the first statistical result and the second statistical result are respectively stretched, and the images after the respectively stretched processing are image fused to obtain the stretched image. It can be seen that performing histogram statistics and stretching for the area where the target object is located and the area where the non-target object is located in the image can better show the contrast between the area where the target object is located and the area where the non-target object is located, thereby obtaining the target object more accurately The details of the area. Wherein, performing stretching processing on the above-mentioned first statistical result includes: performing stretching processing on the first statistical result according to a first stretching interval (first gray value interval). Stretching the above-mentioned second statistical result includes: performing stretch processing on the second statistical result according to a second stretch interval (second gray value interval).

It can be understood that the first stretching interval is different from the second stretching interval. The situation where the first stretching interval and the second stretching interval are not in phase include but are not limited to the following methods: for example, the maximum value in the first stretching interval is greater than or equal to the maximum value in the second stretching interval, and The minimum value in the first stretching interval is less than or equal to the minimum value in the second stretching interval. For another example, the difference between the maximum value and the minimum value in the first stretching interval is greater than or equal to the difference between the maximum value and the minimum value in the second stretching interval. For another example, there is no common part between the interval range included in the first stretching interval and the interval range included in the second stretching interval. For another example, there is an overlap range between the interval range included in the first stretching interval and the interval range included in the second stretching interval, but the overlap range is different from the interval ranges of the first stretching interval and the second stretching interval. Wait.

The above-described execution of the two stretching processes of "stretching the first statistical result based on the first stretching interval" and "stretching the second stretching result based on the second stretching interval" does not Distinguish the order of execution. In an implementation manner, the first statistical result may be stretched according to the first stretch interval first, and then the second statistical result may be stretched according to the second stretch interval. In an implementation manner, the second statistical result may be stretched according to the second stretch interval first, and then the first stretch result may be stretched according to the first stretch interval. In an implementation manner, the two processes of "stretching the first statistical result based on the first stretching interval" and "stretching the second stretching result based on the second stretching interval" can be performed at the same time .

In an optional implementation manner, step S302 may further include: performing medium and high frequency detail enhancement processing on the pixels of the target object in the image to obtain an image after the medium and high frequency detail enhancement processing. Among them, the medium and high frequency detail enhancement processing refers to the use of mean filtering to eliminate shot noise in the image to obtain an image with prominent high frequency parts.

It can be seen that performing medium and high frequency detail enhancement processing for the pixels of the target object can improve the processing accuracy of the target object and show more details of the target object compared to performing the global mid and high frequency detail enhancement processing on the image.

In another optional implementation manner, step S302 may include: performing medium and high frequency detail enhancement processing on the pixel points of the target object and the pixel points of the non-target object in the image to obtain an image after the medium and high frequency detail enhancement processing. For example, the degree of detail enhancement is reduced in the area where the non-target object is located in the image, and the detail enhancement intensity of the area where the target object is correspondingly increased. Therefore, the image can be sharpened in the frequency domain, such as weakening the low-frequency component of the area where the non-target object is located, and enhancing the high-frequency component of the area where the target object is located. Thereby, an image with richer content and details of the target object can be obtained.

It can be seen that the medium and high-frequency detail enhancement processing of the image based on the target object can make the high-frequency component of the area where the target object is located stand out, and the low-frequency component of the area where the non-target object is located is weakened, thereby better showing the details and details of the target object in the image. content.

In an optional implementation manner, step S302 may include: using a target color palette to perform pseudo-color processing on the pixels of the target object in the image to obtain a pseudo-color processed image. For example, select the target palette corresponding to the target object from the preset palette library. Wherein, the preset color palette library includes one or more color palettes corresponding to one or more objects respectively.

Optionally, the color palettes corresponding to different target objects are different. For example, the color palette in the preset palette library can be generated according to the type of the target object. For example, if the type of the target object is a white cloud, the color palette can be color palette 1; if the type of the target object is a puppy, the color palette can be color palette 2. Palette 2 is a different palette. When the system detects a target object, it can match the corresponding color palette based on the type of the target object, and use the color palette to perform pseudo-color mapping on the target object. It should be noted that the color palette in the preset color palette library can also be generated according to other information of the target object, such as generating a color palette based on the number of pixels in the area where the target object is located, which is not limited in the embodiment of the application. .

It can be seen that performing pseudo-color mapping processing on the image based on the target object can convert the grayscale image into a color image, thereby improving the image display effect.

In the embodiment of the present application, the above-mentioned image enhancement processing on the image based on the target object may include one or more of the histogram statistics and stretching, mid- and high-frequency detail enhancement, and pseudo-color mapping described in the foregoing implementation manners. In addition, the execution sequence of histogram statistics and stretching, mid-high frequency detail enhancement, and pseudo-color mapping described in each implementation manner is not limited in the embodiment of the present application.

For example, as shown in Fig. 7, the histogram statistics and stretching processing can be performed on the image based on the target object first, and then the mid-high frequency detail enhancement processing and pseudo color mapping processing can be performed on the image based on the target object. Alternatively, as shown in the dotted line in FIG. 7, the histogram statistics and stretching processing are performed on the image based on the target object, and then the pseudo-color mapping processing is performed on the image based on the target object.

In the embodiment of the present application, the operation of identifying the target object in the image may be performed before one or more of the histogram statistics and stretching processing, the middle and high frequency detail enhancement processing, and the pseudo color mapping processing in the image enhancement processing.

For example, the operation of recognizing the target object in the image can be performed before the pseudo-color mapping process and after the mid-high frequency detail enhancement process. Furthermore, the target object is pushed to one or more of histogram statistics and stretching processing, middle and high frequency detail enhancement processing, and pseudo-color mapping processing, so that these image enhancement processing can be performed based on the target object.

For another example, the operation of recognizing the target object in the image can be performed before the middle and high frequency detail enhancement processing, and after the histogram statistics and stretching processing are performed on the image. Furthermore, the target object can be pushed to one or more of histogram statistics and stretching processing, medium and high frequency detail enhancement processing, and pseudo-color mapping processing, respectively, so that these image enhancement processing can be performed based on the target object.

For another example, the operation of identifying the target object in the image can be performed before the histogram statistics and stretching processing. Furthermore, the target object can be pushed to one or more of histogram statistics and stretching processing, medium and high frequency detail enhancement processing, and pseudo-color mapping processing, respectively, so that these image enhancement processing can be performed based on the target object.

As shown in FIG. 8, the operation of recognizing the target object in the image can be performed after the middle and high frequency detail enhancement processing and before the pseudo color mapping processing. In addition, the target object is identified from the image in the video frame, and then the target object is input into each operation of the image enhancement processing, so that each operation of the image enhancement processing is performed based on the target object.

It can be seen that the embodiment of the present application recognizes the target object from the image in the infrared video frame, and performs image enhancement processing on the image in the infrared video frame based on the target object. Compared with the global processing of the image, it can be processed more accurately. The local details in the image, so as to better show the content of the target object in the image.

Please refer to FIG. 9, which is a schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method shown in FIG. 9 is different from the image processing method shown in FIG. 3 in that identifying the target object in the image can be performed after image processing. Wherein, the image processing may include one or more operations in the image enhancement processing described above. As shown in Figure 9, the method may include the following steps.

S901: Perform image processing on an image to obtain an image after image processing.

Wherein, the image processing includes one or more of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo-color mapping. That is, before identifying the target object, you can perform these processing on the image first.

S902: Identify the target object from the image after the image processing.

In an optional implementation manner, if the target object is recognized after the above image enhancement processing, according to the contour shape of the target object in the image and the temperature distribution of the image, during the process of recognizing the target object in the infrared video frame, the image The temperature distribution of is the temperature distribution of the image before the image enhancement processing. As a result, it is beneficial to ensure the accuracy of the temperature distribution of the image, thereby making the recognized target object more accurate.

In an alternative implementation manner, step S902 may include: recognizing the target object from the image after image processing based on the target object preset by the system or specified by the user.

For example, the system can store at least one type of target object. Extract pixel point information of the object in the image after image processing; determine whether the object is the stored target object based on the pixel point information, and if so, identify the target object based on the pixel point information. Wherein, the pixel point information may include the gray value, temperature value, contour shape, etc. of each pixel point. For example, the target object stored by the system is a car, and the system extracts the pixel information of the object in the image after image processing; based on the pixel information, it determines whether the object is a stored car, and if so, the car is identified based on the pixel information .

For another example, the display device may display a label representing the target object for the user to select. Detect the user's click on the label; based on the label clicked by the user, determine that the object corresponding to the label is the target object in the image. For example, the display device can display the label of the white cloud, the label of the puppy, and the label of the car; if the user clicks the label of the white cloud, the target object in the image can be determined as the white cloud based on the label of the white cloud.

It can be understood that the process of identifying the target object from the image after image processing in step S902 is similar to the implementation process of step S301, and will not be repeated here.

S903: Perform image enhancement processing on the image based on the target object.

S904: Output an image after image enhancement processing.

The implementation process of step S902 and step S301 in the embodiment of the present application is similar, and will not be repeated here. Steps S903 to S904 in the embodiment of the present application are the same as steps S302 to S303, and will not be repeated here.

It can be seen that before performing target object recognition on infrared video frames, image processing can be performed on the image to obtain an image with better contrast. Recognizing the target object in the image based on the image after image processing, and performing image enhancement processing on the image based on the target object, can more accurately process the local details of the image, thereby helping to display the image content that the user is interested in.

Please refer to FIG. 10. FIG. 10 is a schematic structural diagram of an image processing device according to an embodiment of the present application. The image processing device 10 of the embodiment of the present application includes a memory 1001 and a processor 1002. The memory 1001 may include a volatile memory (volatile memory), such as random-access memory (RAM); the memory 1001 may also include a non-volatile memory (non-volatile memory), such as flash memory (flash memory), solid-state drive (solid-state drive, SSD), etc.; the memory 1001 may also include a combination of the foregoing types of memories.

The processor 1002 may be a central processing unit (CPU), and the processor 1002 may also be an image processing unit (GPU). The processor 1002 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The above-mentioned PLD may be a field-programmable gate array (FPGA), a general array logic (generic array logic, GAL), and the like.

In an embodiment, the image processing device may be a movable platform or a movable device, such as a drone. In this embodiment, the image processing device can be configured with a sensor (such as an infrared sensor) and a processor. The infrared sensor can obtain an infrared video frame and transmit the infrared video frame to the processor; the processor can analyze the image in the infrared video frame. Perform related operations of the image processing method described in this application. Optionally, the image processing device may also be equipped with a communication interface, which is used to send the processed image to the terminal, and the terminal will output, for example, display the processed image.

In another embodiment, the image processing device may be a control device, and the control device is configured with a processor and a memory as shown in FIG. 10, and in addition, a display and a communication interface may also be configured. The communication interface is used to receive infrared video frames sent by mobile platforms such as drones, and the processor is used to perform the relevant operations of the image processing method described in this application for the infrared video frames received by the communication interface, and output the processed images To the monitor, the processed image is displayed on the monitor. For example, the control device may be a terminal device, a remote control, a server, or a control device of a movable platform.

The image processing device of the embodiment of the present application can be used to implement the method implemented by each embodiment of the present application shown in FIG. 3 or FIG. 9 through the processor 1002. For ease of description, only the method implemented in the present application is shown. For related parts of the example, please refer to the embodiments of the present application shown in FIG. 3 or FIG. 9 for implementation.

In an optional implementation manner, the memory 1001 stores program instructions, and the processor 1002 calls the program instructions in the memory. When the program instructions are executed, the processor 1002 is used to: Identify the target object in the image; perform image enhancement processing on the image based on the target object, where the image enhancement processing may include one or more of histogram statistics and stretching, medium and high frequency detail enhancement, and pseudo-color mapping; output The image after image enhancement processing.

In an optional implementation manner, the processor 1002 is configured to: perform histogram statistics on the pixels in the area where the target object in the image is located, to obtain a statistical result;

Stretching processing is performed on the statistical result to obtain a stretched image.

In an optional implementation manner, the processor 1002 is configured to: perform the histogram statistics on the pixels of the target object and the pixels of the non-target object in the image, respectively, to obtain a first statistical result And the second statistical result;

The first statistical result and the second statistical result are respectively subjected to stretching processing, and the images after the stretching processing are subjected to image fusion to obtain the image after the stretching processing.

In an optional implementation manner, the processor 1002 is configured to: use a first stretch interval to stretch the first statistical result, and use a second stretch interval to stretch the second statistical result. For stretching treatment, the first stretching interval is different from the second stretching interval.

In an optional implementation manner, the maximum value in the first stretching interval is greater than or equal to the maximum value in the second stretching interval, and the minimum value in the first stretching interval is less than or equal to the second stretching interval The minimum value in.

In an optional implementation manner, the processor 1002 is configured to: perform medium and high frequency detail enhancement processing on the pixels of the target object in the image to obtain an image after the medium and high frequency detail enhancement processing.

In an optional implementation manner, the processor 1002 is configured to: perform medium and high frequency detail enhancement processing on the pixels of the target object and the pixels of the non-target object in the image, respectively, to obtain medium and high frequency detail enhancement The processed image.

In an optional implementation manner, the processor 1002 is configured to: use a target color palette to perform pseudo-color processing on the pixels of the target object in the image to obtain a pseudo-color processed image.

In an optional implementation manner, the processor 1002 is configured to: select the target palette corresponding to the target object from a preset palette library, and the preset palette library includes one or One or more color palettes corresponding to multiple objects.

In an optional implementation manner, the processor 1002 is configured to: identify the target object in the image according to the contour shape of the target object in the image and the temperature distribution of the image.

In an optional implementation manner, the processor 1002 is configured to use an object recognition and/or semantic segmentation algorithm to obtain the contour shape of the target object in the image.

In an optional implementation manner, the processor 1002 is configured to: perform image processing on the image to obtain an image after image processing;

The processor 1002 recognizes the target object from the image in the infrared video frame, including:

The target object is identified from the image after image processing.

In an optional implementation manner, the image processing includes one or more of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo-color mapping.

In an optional implementation manner, the infrared video frame is an infrared video frame obtained by an infrared sensor.

In an optional implementation manner, the target object is preset by the system or specified by the user.

The image processing device provided in this embodiment can execute the image processing method provided in the foregoing embodiment, and its execution mode and beneficial effects are similar, and will not be repeated here.

The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, they are used to execute any of the instructions described in FIGS. 3 to 9 Or multiple embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other divisions in actual implementation, for example, multiple modules or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or modules, and may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional modules in the various embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, or in the form of hardware plus software functional modules.

The above-mentioned integrated modules implemented in the form of software functional modules may be stored in a computer readable storage medium. The above-mentioned software function module is stored in a storage medium, and includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute the method described in each embodiment of the present application. Part of the steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program instructions .

Those skilled in the art can clearly understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit it; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. range.

Claims

An image processing method, characterized in that it comprises:

Identify the target object from the image in the infrared video frame;

Performing image enhancement processing on the image based on the target object;

An image after image enhancement processing is output, and the image enhancement processing includes one or more processing of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo-color mapping.
The method of claim 1, wherein the performing image enhancement processing on the image based on the target object comprises:

Performing histogram statistics on the pixels in the area where the target object in the image is located to obtain a statistical result;

Stretching processing is performed on the statistical result to obtain a stretched image.
The method of claim 1, wherein the performing image enhancement processing on the image based on the target object comprises:

Performing the histogram statistics on the pixels of the target object and the pixels of the non-target object in the image, respectively, to obtain a first statistical result and a second statistical result;

The first statistical result and the second statistical result are respectively subjected to stretching processing, and the images after the stretching processing are subjected to image fusion to obtain the image after the stretching processing.
The method according to claim 3, wherein said performing stretching processing on said first statistical result and said second statistical result respectively comprises:

The first stretching interval is used to perform stretching processing on the first statistical result, and the second stretching interval is used to perform stretching processing on the second statistical result. The first stretching interval is compared with the second stretching interval. The stretch range is different.
The method of claim 4, wherein the maximum value in the first stretching interval is greater than or equal to the maximum value in the second stretching interval, and the minimum value in the first stretching interval is The value is less than or equal to the minimum value in the second stretching interval.
The method according to any one of claims 1 to 5, wherein the performing image enhancement processing on the image based on the target object further comprises:

Medium and high frequency detail enhancement processing is performed on the pixel points of the target object in the image to obtain an image after the medium and high frequency detail enhancement processing.
The method according to any one of claims 1 to 6, wherein the performing image enhancement processing on the image based on the target object further comprises:

Medium and high frequency detail enhancement processing is performed on the pixel points of the target object and the pixel points of the non-target object in the image, respectively, to obtain an image after the medium and high frequency detail enhancement processing.
The method according to any one of claims 1 to 6, wherein the performing image enhancement processing on the image based on the target object further comprises:

A target color palette is used to perform pseudo-color processing on the pixels of the target object in the image to obtain a pseudo-color processed image.
8. The method according to claim 8, wherein said using a target color palette to perform pseudo-color processing on the pixels of the target object in the image, and before obtaining the pseudo-color processed image, the method further comprises:

The target color palette corresponding to the target object is selected from a preset color palette library, and the preset color palette library includes one or more color palettes respectively corresponding to one or more objects.
The method according to any one of claims 1 to 9, wherein the recognizing the target object from the image in the infrared video frame comprises:

Identify the target object in the image according to the contour shape of the target object in the image and the temperature distribution of the image.
The method according to claim 10, characterized in that, before recognizing the target object in the image according to the contour shape and temperature distribution of the target object in the image, the method further comprises:

An object recognition and/or semantic segmentation algorithm is used to obtain the contour shape of the target object in the image.
The method according to any one of claims 1 to 11, wherein before the identifying the target object from the image in the infrared video frame, the method further comprises:

Performing image processing on the image to obtain an image after image processing;

The identifying the target object from the image in the infrared video frame includes:

The target object is identified from the image after image processing.
The method according to claim 12, wherein the image processing includes one or more of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo color mapping.
The method according to any one of claims 1 to 13, wherein the infrared video frame is an infrared video frame obtained by an infrared sensor.
The method according to any one of claims 1 to 14, wherein the target object is preset by the system or specified by the user.
An image processing device, characterized in that the image processing device includes: a processor and a memory;

Program instructions are stored in the memory;

The processor calls the program instructions for

Identify the target object from the image in the infrared video frame;

Performing image enhancement processing on the image based on the target object;

An image after image enhancement processing is output, and the image enhancement processing includes one or more processing of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo-color mapping.
The device according to claim 16, wherein the image processing device further comprises a sensor;

The sensor is used to obtain infrared video frames.
The device according to claim 16, wherein the image processing device further comprises a display;

The display is used to display the image after image enhancement processing.
The device of claim 17, wherein the sensor comprises an infrared sensor.
The device according to any one of claims 16 to 19, wherein the image processing device is a movable device or a control device of the movable device.
The device according to any one of claims 16 to 19, wherein the processor performs image enhancement processing on the image based on the target object, comprising:

Performing histogram statistics on the pixels in the area where the target object in the image is located to obtain a statistical result;

Stretching processing is performed on the statistical result to obtain a stretched image.
The device according to any one of claims 16 to 19, wherein the processor performs image enhancement processing on the image based on the target object, comprising:

Performing the histogram statistics on the pixels of the target object and the pixels of the non-target object in the image, respectively, to obtain a first statistical result and a second statistical result;

The first statistical result and the second statistical result are respectively subjected to stretching processing, and the images after the stretching processing are subjected to image fusion to obtain the image after the stretching processing.
The device according to claim 22, wherein the processor performs stretching processing on the first statistical result and the second statistical result respectively, comprising:

The first stretching interval is used to perform stretching processing on the first statistical result, and the second stretching interval is used to perform stretching processing on the second statistical result. The first stretching interval is compared with the second stretching interval. The stretch range is different.
The device according to claim 23, wherein the maximum value in the first stretching interval is greater than or equal to the maximum value in the second stretching interval, and the minimum value in the first stretching interval is The value is less than or equal to the minimum value in the second stretching interval.
The device according to any one of claims 16 to 24, wherein the processor performs image enhancement processing on the image based on the target object, comprising:

Perform mid- and high-frequency detail enhancement processing on the pixels of the target object in the image to obtain an image after mid- and high-frequency detail enhancement processing.
The device according to any one of claims 16 to 24, wherein the processor performs image enhancement processing on the image based on the target object, comprising:

Medium and high frequency detail enhancement processing is performed on the pixel points of the target object and the pixel points of the non-target object in the image, respectively, to obtain an image after the medium and high frequency detail enhancement processing.
The device according to any one of claims 16 to 26, wherein the processor performs image enhancement processing on the image based on the target object, comprising:

A target color palette is used to perform pseudo-color processing on the pixels of the target object in the image to obtain a pseudo-color processed image.
The device of claim 27, wherein the processor uses a target color palette to perform pseudo-color processing on the pixels of the target object in the image, and before obtaining the pseudo-color processed image, the processor further comprises :

The target color palette corresponding to the target object is selected from a preset color palette library, and the preset color palette library includes one or more color palettes respectively corresponding to one or more objects.
The device according to any one of claims 16 to 28, wherein the processor recognizing the target object from the image in the infrared video frame comprises:

Identify the target object in the image according to the contour shape of the target object in the image and the temperature distribution of the image.
The device according to claim 29, wherein the processor is further configured to use an object before recognizing the target object in the image according to the contour shape and temperature distribution of the target object in the image A recognition and/or semantic segmentation algorithm is used to obtain the contour shape of the target object in the image.
The device according to any one of claims 16 to 28, characterized in that:

Before the processor recognizes the target object from the image in the infrared video frame, it is also used to perform image processing on the image to obtain an image after image processing;

The processor identifying the target object from the image in the infrared video frame includes:

The target object is identified from the image after image processing.
The device according to claim 31, wherein the image processing includes one or more of histogram statistics and stretching, medium and high frequency detail enhancement, or pseudo-color mapping.
The device according to any one of claims 16 to 32, wherein the infrared video frame is an infrared video frame obtained by an infrared sensor.
The method according to any one of claims 16 to 32, wherein the target object is preset by the system or specified by the user.
A computer-readable storage medium, characterized in that computer program instructions are stored in the computer-readable storage medium, and when the computer program instructions are executed by a processor, they are used to execute any one of claims 1 to 15 The image processing method described.