WO2020119454A1

WO2020119454A1 - Method and apparatus for color reproduction of image

Info

Publication number: WO2020119454A1
Application number: PCT/CN2019/121126
Authority: WO
Inventors: 孙超伟; 竺旭东
Original assignee: 华为技术有限公司
Priority date: 2018-12-12
Filing date: 2019-11-27
Publication date: 2020-06-18
Also published as: CN111311500A

Abstract

The embodiments of the present application provide a method and an apparatus for color reproduction of an image, relating to the field of image processing, and being able to perform accurate color reproduction of an overexposed object (for example, a traffic signal lamp) in an image. Said method comprises: acquiring a first image to be processed, the first image comprising an overexposed first target object; according to the saturation and luminance of pixels of the first image, determining a first region of the first image; performing binarization processing on the first region, to obtain a binary image corresponding to the first region; determining, in the binary image, at least one connected region of which the area is greater than or equal to a first threshold, the contour of the at least one connected region corresponding to the contour of the first target object; and reproducing the color of the at least one connected region. The embodiments of the present application are applicable to situations where color reproduction is performed on an overexposed image.

Description

Method and device for color restoration of image

Technical field

The present application relates to the field of image processing, and in particular to a method and device for color restoration of an image.

Background technique

Image overexposure is a relatively common phenomenon, which may cause a series of problems. For example, in a traffic video surveillance system, a surveillance camera working in an electronic police mode can simultaneously capture red light signals and illegal vehicles as evidence for violations by illegal personnel. However, in the case of relatively low background illumination (such as dusk or night scenes), when the surveillance camera takes a photo forensic against the signal red light, the camera often needs to increase the shutter, gain and aperture size. If the gain, shutter or aperture is increased too much, it will cause the traffic lights to be overexposed (for example, the red signal lights will become yellow or white), so that the captured images cannot be used as evidence of violations.

At present, there are several methods to realize the color restoration of signal lights, which can be divided into hardware methods and software methods. Hardware method: A camera with an ultra-wide dynamic range can be used to eliminate signal color distortion caused by strong light. This is because a camera with an ultra-wide dynamic range detects a large dynamic range of brightness, and can restore image details in a high-contrast brightness scene. However, on the one hand, the ultra-wide dynamic camera is relatively expensive; on the other hand, because the signal light area generally accounts for a small proportion of the entire screen, the ultra-wide dynamic range camera has a limited effect on the color reproduction of the over-exposed signal light under low illumination. Software method 1: You can identify the signal light area based on the red, green, blue (RGB) (color) space of the image, and then perform the color restoration of the signal light. However, the RGB color space does not reflect the brightness information of the signal light well, and may cause a discontinuous gradient phenomenon between the signal area after the color reproduction and the surrounding image. Software method 2: Based on the method of deep learning, the signal lights in the image can be identified and the color can be enhanced. However, the method based on deep learning does not have a high accuracy in recognizing the contour of the signal light, so the color reproduction area of the signal light is prone to appear discontinuous with the surrounding image.

Therefore, there is a need for a more accurate method for color reproduction of overexposed objects (such as overexposed traffic lights) in images.

Summary of the invention

Embodiments of the present application provide an image processing method, which can accurately reproduce color of overexposed objects (for example, traffic lights) in an image. Furthermore, it can solve a series of problems caused by overexposure of objects in the image (for example, the problem of difficulty in obtaining evidence caused by overexposure of signal lights in the current electric police monitoring scene).

In a first aspect, an embodiment of the present application provides a method for color restoration of an image, including: acquiring a first image to be processed, the first image including an overexposed first target object; and according to the saturation of the pixels of the first image And the brightness determine the first area of the first image, the saturation of the pixels in the first area is lower than the average value of the saturation of the pixels in the first image, and the brightness of the pixels in the first area is higher than the brightness of the pixels in the first image The average value of the first area; the first area corresponds to the area of the first target object (where); the first area is binarized to obtain the binary image corresponding to the first area; the area of the binary image is determined to be greater than or equal to the first threshold At least one connected area; the outline of at least one connected area corresponds to the outline of the first target object; the color of at least one connected area is restored.

Based on the method provided in the embodiments of the present application, after acquiring the first image to be processed, the first area of the first image may be determined according to the saturation and brightness of the pixels of the first image (the first area may be used as the first target of overexposure) The area where the object is located), and then the first area can be binarized to obtain a binary image corresponding to the first area; and then determine at least one connected area (at least one connected area) in the binary image whose area is greater than or equal to the first threshold The contour of the area corresponds to the contour of the first target object), and then, the color of the at least one connected area is restored (that is, the color of the over-exposed first target object is restored). It can be seen that the method provided by the embodiments of the present application can accurately reproduce color of an over-exposed object in the image (that is, the first target object, for example, can be a traffic signal). Furthermore, it can solve a series of problems caused by overexposure of objects in the image (for example, the problem of difficulty in obtaining evidence caused by overexposure of signal lights in the current electric police monitoring scene).

In a possible implementation, before determining the first region of the first image according to the saturation and brightness of the first image, the method further includes: converting the first image from the first space to hue saturation and brightness (hue saturation) Value (HSV) space, the first space is any one of the brightness and chroma YUV space, RGB space or hue saturation brightness hue saturation brightness (HSL) space. In other words, if the first image to be processed is in the first space, the first space may be YUV space or RGB space or HSL space. At this time, in order to obtain the saturation and lightness components of the pixels of the first image, you need to The first image is converted from the first space to the HSV space.

In a possible implementation manner, when the first target object is a traffic signal, restoring the color of at least one connected area includes: acquiring color information of the traffic signal; when the traffic signal is red, the pixel of at least one connected area Adjust the hue to the red range, and increase and decrease the saturation and lightness of the pixels of at least one connected area; when the traffic signal is yellow, adjust the hue of the pixels of at least one connected area to the yellow range, and adjust at least one The saturation and brightness of pixels in the connected area are increased and decreased respectively; when the traffic signal is green, the hue of the pixels of at least one connected area is adjusted to the green range, and the saturation and brightness of the pixels of at least one connected area are respectively adjusted Perform ascent and descent.

In the embodiment of the present application, increasing and decreasing the saturation and lightness of the pixels of at least one connected area may be linearly increasing and decreasing the saturation and lightness of the pixels of at least one connected area, respectively, or may be The saturation and lightness of the pixels of at least one connected area are nonlinearly improved and nonlinearly reduced, respectively.

In a possible implementation manner, when the first target object is a traffic signal, restoring the color of at least one connected area includes: converting the at least one connected area to an RGB space; acquiring color information of the traffic signal; when the traffic signal is red , Adjust the red component of the pixels of at least one connected area to the first preset range, and reduce the blue and green components of the pixels of at least one connected area; when the traffic signal is yellow, change the red color of the pixels of at least one connected area And the green component are adjusted to the second preset range to reduce the blue component of pixels in at least one connected area; when the traffic signal is green, the green component of the pixels in at least one connected area is adjusted to the third preset range and reduced by at least The red and blue components of pixels in a connected area.

Wherein, reducing the blue and green components of the pixels of at least one connected region may be a linear or non-linear reduction of the blue and green components of the pixels of at least one connected region.

In a possible implementation, before determining the first region of the first image according to the saturation and brightness of the first image, the method further includes: performing filtering processing on the saturation component and the brightness component of the pixels of the first image. In this way, the outline of the overexposed first object can be identified more stably and accurately.

In a possible implementation manner, acquiring the first image to be processed includes: using the second area selected by the user on the second image as the first image to be processed.

In this way, compared to processing the second image directly, selecting the first image from the second image and processing only the first image, on the one hand, it can reduce the amount of calculation in the subsequent steps, on the other hand, it can improve The recognition accuracy of the area where the first target object (for example, a signal light) is located.

In a second aspect, an embodiment of the present application provides an image processing device, including: an acquisition unit for acquiring a first image to be processed, the first image including an overexposed first target object; and a determination unit for The saturation and brightness of the pixels of the image determine the first area of the first image. The saturation of the pixels of the first area is lower than the average value of the saturation of the pixels of the first image, and the brightness of the pixels of the first area is higher than the first The average value of the lightness of pixels of an image; the first area corresponds to the first target object area; the processing unit is used to binarize the first area to obtain a binary image corresponding to the first area; the determination unit also uses To determine at least one connected region whose area is greater than or equal to the first threshold in the binary image; the contour of the first target object corresponds to the contour of the at least one connected region; the processing unit is also used to restore the color of the at least one connected region.

In a possible implementation manner, the processing unit is further configured to: convert the first image from the first space to the HSV space, and the first space is any one of YUV space, RGB space, or HSL space.

In a possible implementation, when the first target object is a traffic signal, the processing unit is used to: obtain the color information of the traffic signal through the acquisition unit; when the traffic signal is red, change the hue of the pixels of at least one connected area Adjust to the red range, and increase and decrease the saturation and brightness of the pixels of at least one connected area; when the traffic signal is yellow, adjust the hue of the pixels of at least one connected area to the yellow range, and connect at least one The saturation and brightness of the pixels in the area are increased and decreased respectively; when the traffic signal is green, the hue of the pixels of at least one connected area is adjusted to the green range, and the saturation and brightness of the pixels of at least one connected area are respectively adjusted Raise and lower.

In a possible implementation manner, when the first target object is a traffic signal, the processing unit is used to: convert at least one connected area to an RGB space; acquire the color information of the traffic signal through the acquisition unit; when the traffic signal is red , Adjust the red component of the pixels of the at least one connected area to the first preset range, and reduce the blue and green components of the pixels of the at least one connected area; when the traffic signal is yellow, the red sum of the pixels of the at least one connected area The green component is adjusted to the second preset range to reduce the blue component of pixels in at least one connected area; when the traffic signal is green, the green component of the pixels in at least one connected area is adjusted to the third preset range to reduce at least one The red and blue components of the pixels in the connected area.

In a possible implementation manner, the processing unit is further configured to filter the saturation component and the lightness component of the pixels of the first image.

In a possible implementation manner, the acquiring unit is configured to: use the second area selected by the user on the second image as the first image to be processed.

For the technical effects of the second aspect and its various possible implementation manners, refer to the technical effects of the first aspect and its various possible implementation manners, and details are not described herein again.

In a third aspect, an embodiment of the present application provides an apparatus that exists in the form of a chip product. The structure of the apparatus includes a processor and a memory, and the memory is used to couple with the processor to store necessary program instructions of the apparatus And data, the processor is used to execute the program instructions stored in the memory, so that the device performs the function of the image processing device in the above method.

According to a fourth aspect, an embodiment of the present application provides an image processing device that can implement the functions performed by the image processing device described above. The functions can be implemented by hardware, or can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

In a possible design, the structure of the image processing device includes a processor and a communication interface, and the processor is configured to support the image processing device to perform the corresponding function in the above method. The communication interface is used to support communication between the image processing device and other devices (e.g., a signal light detector on traffic lights). The image processing device may further include a memory for coupling with the processor, which stores necessary program instructions and data of the image processing device.

According to a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which, when run on a computer, cause the computer to execute any method provided in the first aspect.

According to a sixth aspect, an embodiment of the present application provides a computer program product containing instructions, which when run on a computer, causes the computer to execute any method provided in the first aspect.

BRIEF DESCRIPTION

FIG. 1 is a schematic diagram of a system architecture suitable for a method for color restoration of an image provided by an embodiment of the present application;

2 is a schematic structural diagram of an image processing device according to an embodiment of the present application;

FIG. 3 is a schematic flowchart of a method suitable for color restoration of an image provided by an embodiment of the present application;

4 is a schematic diagram of a user selecting a first image to be processed provided by an embodiment of the present application;

5 is a schematic diagram of a first area provided by an embodiment of this application;

6 is a schematic diagram of a binary image corresponding to a first area provided by an embodiment of the present application;

7 is a schematic diagram of a connected area provided by an embodiment of the present application;

8 is a schematic diagram of a first area, a binary image corresponding to the first area, and a connected area determined according to the binary image provided by an embodiment of the present application;

9 is a schematic structural diagram of yet another image processing device provided by an embodiment of the present application.

detailed description

Embodiments of the present application provide a method and device for color (color) restoration of an image, which are applied to a scene of color restoration of an overexposed image. It can be understood that the embodiments of the present application may also be applied to a scene where color restoration is performed on a video including one or more frames of overexposed images. Specifically, it can be applied to the process of color restoration of an overexposed object (or overexposed area) in an image. For example, in the process of color reproduction of overexposed traffic lights in images or videos taken by electronic police (cameras or monitors).

The embodiment of the present application takes the scene of an illegal image captured by an electronic police camera as an example. As shown in FIG. 1, a schematic diagram of a system architecture suitable for a method for performing color restoration on an image (an illegal image captured by an electronic police camera), including electronic A police camera and an image processing device connected to the electronic police camera (the image processing device may also be integrated in the electronic police camera), the image processing device may perform color restoration processing on the image or video taken by the electronic police camera. The image processing device can also be connected to a traffic signal lamp in order to obtain the (historical) color change of the signal lamp from the signal lamp detector on the traffic signal lamp.

The image processing device in FIG. 1 of the embodiment of the present application may be implemented by one device, or may be a functional module in a device, which is not specifically limited in the embodiment of the present application. It is understandable that the above-mentioned functions may be network elements in hardware devices, or software functions running on dedicated hardware, or virtualized functions instantiated on platforms (for example, cloud platforms), or chip systems. . In the embodiment of the present application, the chip system may be composed of a chip, or may include a chip and other discrete devices.

For example, the apparatus for implementing the functions of the image processing apparatus provided by the embodiments of the present application may be implemented by the apparatus 200 in FIG. 2. FIG. 2 is a schematic diagram of a hardware structure of an apparatus 200 provided by an embodiment of the present application. The apparatus 200 includes at least one processor 201, which is used to implement the functions of the image processing device provided in the embodiments of the present application. The device 200 may further include a bus 202 and at least one communication interface 204. The device 200 may further include a memory 203.

In the embodiment of the present application, the processor may be a central processing unit (central processing unit, CPU), a general-purpose processor, a network processor (NP), a digital signal processor (digital signal processing, DSP), or a micro processor Device, microcontroller, programmable logic device (PLD) or any combination of them. The processor may also be any other device with a processing function, such as a circuit, a device, or a software module.

The bus 202 can be used to transfer information between the aforementioned components.

The communication interface 204 is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 204 may be an interface, a circuit, a transceiver, or other devices capable of implementing communication, and the application is not limited. The communication interface 204 may be coupled with the processor 201. The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which may be in electrical, mechanical or other forms, and is used for information interaction between devices, units or modules.

In the embodiments of the present application, the memory may be read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), or may store Other types of dynamic storage devices for information and instructions can also be electrically erasable programmable read-only memory (electrically erasable programmable-read-only memory (EEPROM), compact-disc read-only memory (CD-ROM) or Other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired data in the form of instructions or data structures Program code and any other medium that can be accessed by the computer, but not limited to this. The memory may exist independently, or may be coupled with the processor, for example, through the bus 202. The memory can also be integrated with the processor.

The memory 203 is used to store program instructions, and can be controlled and executed by the processor 201, so as to implement the method for color restoration of an image provided by the following embodiments of the present application. The processor 201 is used to call and execute the instructions stored in the memory 203, so as to implement the method for color restoration of an image provided by the following embodiments of the present application.

Optionally, the computer execution instructions in the embodiments of the present application may also be called application program codes, which are not specifically limited in the embodiments of the present application.

Alternatively, the memory 203 may be included in the processor 201.

In a specific implementation, as an embodiment, the processor 201 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 2.

In a specific implementation, as an embodiment, the apparatus 200 may include multiple processors, such as the processor 201 and the processor 207 in FIG. 2. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

In a specific implementation, as an embodiment, the apparatus 200 may further include an output device 205 and an input device 206. The output device 205 and the processor 201 are coupled and can display information in various ways. For example, the output device 205 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. Wait. The input device 206 and the processor 201 are coupled and can receive user input in various ways. For example, the input device 206 may be a camera, a mouse, a keyboard, a touch screen device, or a sensing device.

The above apparatus 200 may be a general-purpose device or a dedicated device. In a specific implementation, the image processing device 200 may be a video camera, a camera, a monitor, a video display device, a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device , Embedded devices or devices with a similar structure in Figure 2. The embodiment of the present application does not limit the type of the device 200.

In order to describe the following embodiments clearly and concisely, a brief introduction of related concepts or technologies is first given:

Color space: Color is usually described by three independent attributes. Three independent variables work together to form a space coordinate, that is, color space. The color space can include RGB (color) space, YUV (color) space, HSV (color) space and HSL (color) space, etc. Different color spaces can measure the color of the same object from different angles. In different processing processes, the emphasis on color processing is different, so various color spaces can be converted to meet different processing requirements. among them:

RGB space: R represents the red component (red channel), G represents the green component (green channel), and B represents the blue component (blue channel). Various colors are obtained by changing the three color channels of red, green and blue and superimposing them. In each component, the smaller the value, the lower the brightness, and the larger the value, the higher the brightness. When the components are mixed, the brightness after mixing is equal to the sum of the brightness of the components.

YUV space: Y represents brightness, that is, gray scale value. U and V represent chroma, used to describe the image color and saturation, you can specify the color of the pixel. If there are only Y signal components and no U, V components, then the image represented in this way is a black and white grayscale image. YUV space is mainly used to optimize the transmission of color video signals, making it backward compatible with old-fashioned black and white TVs.

HSV space: HSV is a method of representing points in the RGB color space in an inverted cone. Among them, H represents color information, that is, the position of the spectral color, which can be measured by angle, and the value range is 0° to 360°. Red is 0°, green is 120°, and blue is 240°. S is expressed as the ratio between the saturation of the selected color and the maximum saturation of the color, ranging from 0 to 1. When S=0, there is only grayscale. V represents the brightness of the color, ranging from 0 to 1.

HSL space: HSL is similar to HSV, and the first two parameters in this model are the same as HSV. L represents the brightness of the color, and can be used to control the change of the light and dark of the color. The value of L ranges from 0% to 100%. The smaller the value, the darker the color and the closer to black; the larger the value, the brighter the color and the closer to white.

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, unless otherwise stated, “at least one” refers to one or more, and “plurality” refers to two or more than two. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish the same or similar items that have substantially the same functions and functions. Those skilled in the art may understand that the words "first" and "second" do not limit the number and execution order, and the words "first" and "second" do not necessarily mean different.

For ease of understanding, the method for color restoration of an image provided by embodiments of the present application will be specifically described below with reference to the drawings.

As shown in FIG. 3, an embodiment of the present application provides a method for color restoration of an image, including:

301: Acquire a first image to be processed, the first image including an overexposed first target object.

The first target object refers to one or more objects (or areas where the objects are located) that are overexposed in the first image. Among them, the object may be, for example, traffic lights (of various shapes and colors), vehicles, traffic signs, and so on.

In a possible design, the second area selected by the user on the second image may be used as the first image to be processed. Exemplarily, as shown in FIG. 4, when the user thinks that the traffic signal (first target object) in an image (second image) is overexposed, the user can use the input device (for example, mouse or touch screen) of the image processing device Perform the first operation on the second image. The first operation is used to frame the area (second area) where the overexposed traffic signal light is located. After recognizing the first operation of the second image by the user, the image processing device may cut the second area from the second image according to the coordinate information of the second area selected by the user to obtain the third image and the first image to be processed. The first image to be processed includes a second area, and the third image includes other areas than the second area. That is, the first image to be processed includes the second area selected by the user on the second image.

In this way, compared to processing the second image directly, selecting the first image from the second image and processing only the first image, on the one hand, it can reduce the amount of calculation in the subsequent steps, on the other hand, it can improve The recognition accuracy of the area where the first target object (for example, a traffic signal) is located.

302. Optionally, perform filtering processing on the saturation component and the lightness component of the pixels of the first image.

It should be noted that if the second image is an HSV format image, that is, the second image is in the HSV space, then the first image to be processed is also in the HSV space, and the saturation component and the lightness component of the pixels of the first image can be directly The filtering process is performed in order to more stably and accurately identify the outline of the overexposed first object (for example, the overexposed traffic signal light) in the subsequent steps.

If the second image is in the first space, and the first space may be YUV space or RGB space or HSL space, then the first image to be processed is also in the first space. At this time, in order to obtain the saturation component and the lightness component of the pixels of the first image, the first image needs to be converted from the first space to the HSV space. Then, the saturation component and the lightness component of the pixels of the first image are filtered, so as to identify the outline of the overexposed first object more stably and accurately in the subsequent steps.

303. Determine the first area of the first image according to the saturation and lightness of the pixels of the first image.

In the embodiment of the present application, the first area corresponds to the area of the first target object (where). The area where the overexposed first target object (eg, overexposed traffic signal light) in the first image is located may be determined according to the saturation and lightness of the pixels of the first image.

In a possible design, the saturation component and the lightness component of the pixel of the first image after the filtering process may be analyzed to determine that the saturation component is lower than the average value of the saturation of the pixel of the first image, and the brightness component ratio The area where the average value of the brightness of the pixels of the first image is high is the area where the first target object is located. That is, the saturation of the pixels in the first area is lower than the average of the saturation of the pixels in the first image, and the brightness of the pixels in the first area is higher than the average of the brightness of the pixels in the first image. As shown in FIG. 5, it is a schematic diagram of a first area.

304. Perform binarization processing on the first area to obtain a binary image corresponding to the first area.

Exemplarily, as shown in FIG. 6, after performing binarization processing on the first area shown in FIG. 5, a binary image corresponding to the first area may be obtained. The binary image corresponding to the first area may include three connected areas (a, b, and c, respectively). The connected area a is the area where the signal lamp is located, and the connected areas b and c may be the area where the halo generated by the signal lamp is located.

305. Determine at least one connected region whose area in the binary image is greater than or equal to the first threshold.

In a possible design, the contour of the at least one connected region whose area is greater than or equal to the first threshold in the binary image corresponds to the contour of the first target object. The contour of the at least one connected region with an area greater than or equal to the first threshold in the binary image may be used as the contour of the first target object. It should be noted that the outline of the at least one connected area whose area is greater than or equal to the first threshold may be larger or smaller than the outline of the actual (real) first target object. For example, when the first target object is a signal light, since the (lit) signal light generates halos, the at least one connected area may include not only the area where the signal light is, but also the area where the halo generated by the signal light is located, so The outline of the at least one connected area whose area is greater than or equal to the first threshold may be larger than the outline of the actual signal lamp. For another example, if a part of the signal lamp is damaged and cannot emit light, the at least one connected region may only include the region where the part where the signal lamp emits light. In this case, the outline of the at least one connected region is smaller than the actual signal lamp.

The first threshold may be determined according to the area of the connected area with the largest area in the binary image. For example, the first threshold may be equal to the area of the connected area with the largest area in the binary image, or the first threshold may be N% (eg, 30%) of the area of the connected area with the largest area in the binary image, N Is a positive number.

In the binary image corresponding to the first area, there are many connected areas with different areas. For example, as shown in FIG. 6, the binary image corresponding to the first area may include three connected areas (a, b, and c, respectively). Based on the binary image, find all connected regions in the binary image. Sort all the connected regions found according to the size of the area (for example, arrange all connected regions in order from large to small). Among these connected areas with different sizes, it is possible to filter out the connected areas with a smaller area, retain the connected areas with a larger area, and use at least one connected area with a larger area as the area of the first target object (such as a signal light) (for example, One connected area with the largest area may be used as the area where the signal lamp is located, or multiple connected areas with a larger area may be used as the area where the signal lamp is located).

For example, you can first filter out the connected area with the largest area in the binary image. Assuming its area is max, you can set the first threshold to max/(x+2) to filter out the area less than max/(x+2) Connected area. For example, as shown in FIG. 7, the connected regions (b and c) can be filtered out, and the connected region a is retained. The outline of the connected area a corresponds to the outline of the first target object.

The value range of the parameter x may be 0-9, and the parameter x may be used to adjust the size of the first threshold. The larger the value of x, the less the connected areas with a smaller area are filtered out, that is, the more connected areas with a smaller area are retained, so that the outline of the first target object corresponds to the outline of more connected areas. For example, if the first target object is a signal lamp, the area where the signal lamp is located spreads outward (that is, the area where the signal lamp is located includes not only the signal lamp but also the halo generated by the signal lamp). The smaller the value of x, the more connected areas with a smaller area are filtered out, that is, the fewer connected areas with a smaller area are retained, so that the outline of the first target object corresponds to the outline of the fewer connected areas. For example, if the first target object is a signal lamp, the area where the signal lamp is located shrinks inward (that is, the area where the signal lamp is located includes only the signal lamp, not the halo generated by the signal lamp).

For another example, if the first image to be processed includes an indicator light in the shape of an arrow, the image processing device determines that the first area of the first image is as shown in (a) in FIG. 8, and performs binarization processing on the first area After that, as shown in (b) of FIG. 8, the binary image corresponding to the first area may include four connected areas (respectively d, e, f, and g), as shown in (c) of FIG. 8 In this binary image, at least one connected region with an area greater than or equal to the first threshold may be connected regions d and e. That is, the connected regions (f and g) are filtered out, and the connected regions d and e are retained. The outline of the connected regions d and e can be used as the outline of the first target object.

306. Restore the color of at least one connected area.

In the embodiment of the present application, restoring the color of at least one connected region may also be regarded as correcting the color of at least one connected region. Restoring or correcting the color of at least one connected area means restoring or correcting the color of the overexposed first target object. For example, as shown in FIG. 7, restoring or correcting the color of the connected area a is restoring or correcting the color of the overexposed first target object.

Exemplarily, when the first target object is a traffic signal light, the color information of the traffic signal light may be acquired based on a signal light detector externally connected to the signal light or based on an image recognition status light.

In a possible design, the color of at least one connected area can be restored in the HSV space. Specifically, when (determine) the traffic signal is red, adjust the hue of the pixels of at least one connected area to the red range, and increase and decrease the saturation and brightness of the pixels of at least one connected area; when (OK) When the traffic signal is yellow, adjust the hue of the pixels of at least one connected area to the yellow range, and increase and decrease the saturation and brightness of the pixels of at least one connected area; when (OK) the traffic signal is green, change The hue of the pixels of at least one connected area is adjusted to the green range, and the saturation and lightness of the pixels of at least one connected area are increased and decreased, respectively.

In the embodiment of the present application, increasing and reducing the saturation and lightness of the pixels of at least one connected area may be linearly increasing and decreasing the saturation and lightness of the pixels of at least one connected area, respectively, or may be The saturation and lightness of the pixels of at least one connected area are nonlinearly improved and nonlinearly reduced, respectively. Linearly increasing the saturation can increase the saturation of each pixel in the pixels in at least one connected area by the same amount (value); non-linearly increasing the saturation can change the difference in the pixels in at least one connected area The saturation of the pixels is increased by different amounts (values). Linearly reducing the brightness can reduce the brightness of each pixel in at least one connected region by the same amount (value); non-linearly reducing the brightness can reduce the brightness of different pixels in the pixels in at least one connected region Reduce different amounts (values). It should be noted that, after the saturation and lightness of pixels in at least one connected area are respectively increased and decreased, there is a strong correlation between the saturation and lightness of at least one connected area. Before the saturation and lightness of pixels in at least one connected area are increased and decreased respectively, there is no strong correlation between the saturation and lightness of at least one connected area.

In another possible design, the color of at least one connected area can be restored in RGB space. Specifically, when (determining) the traffic signal light is red, the red component of the pixel of the at least one connected area may be adjusted to the first preset range, and the blue and green components of the pixel of the at least one connected area may be reduced.

For example, assuming color restoration of an 8-bit encoded RGB image, if the red component (R value) of a pixel is Rx, the R value of the pixel can be adjusted to R=(200+55*x/ 9+Rx)/2 (the first preset range). The value range of the parameter x can be 0-9, and the parameter x can be used to adjust the size of the R value.

Wherein, reducing the blue and green components of the pixels of at least one connected region may be a linear or non-linear reduction of the blue and green components of the pixels of at least one connected region. Linear reduction can reduce the blue and green components of each pixel in at least one connected region by the same amount (value); non-linear reduction can reduce the blue sum of different pixels in the pixels in at least one connected region The green component decreases by a different amount (value).

When (determine) the traffic signal is yellow, increase the red and green components of the pixels of at least one connected area, and reduce the blue component of the pixels of at least one connected area; when (determined) the traffic signal is green, increase at least one connected area The green component of the pixel reduces the red and blue components of the pixel in at least one connected area. For the specific process, reference may be made to the above related processing process when the traffic signal light is red, and details are not described here.

307: Combine the first image and the third image.

If the first image is converted from the first space to the HSV space in step 302, the color-reduced first image can be re-converted from the HSV space to the first space, so that the first image and the third image (i.e. The two images cut out the remaining part of the first image) and merge to obtain a complete image (it can be considered that the complete image is the second image after color restoration). If the operation of converting the first image from the first space to the HSV space is not performed in step 302, the first image and the third image after color restoration can be directly merged to obtain a complete image.

In a possible design, during the color restoration of the first target object that is overexposed in the video, the user can select the overexposed object before the video starts or when it is paused. In the subsequent video playback process, the user can use The method of the embodiment of the present application performs corresponding processing on each frame image of the video, thereby restoring the color of the over-exposed first target object in the entire video.

Based on the method provided in the embodiments of the present application, after acquiring the first image to be processed, the first area of the first image may be determined according to the saturation and lightness of the pixels of the first image (the first area may be used as the overexposed first The area where the target object is located), and then binarize the first area to obtain the binary image corresponding to the first area; then determine at least one connected area in the binary image whose area is greater than or equal to the first threshold (you can use The outline of at least one connected area is taken as the outline of the first target object), and then, the color of the at least one connected area is restored (that is, the color of the over-exposed first target object is restored). It can be seen that the method provided by the embodiments of the present application can accurately reproduce color of overexposed objects (for example, traffic lights) in the image. Furthermore, it can solve a series of problems caused by overexposure of objects in the image (for example, the problem of difficulty in obtaining evidence caused by overexposure of signal lights in the current electric police monitoring scene).

In the prior art, an ultra-wide dynamic range camera can be used to eliminate the color distortion of the signal light caused by strong light, and the ultra-wide dynamic camera is expensive and has a limited color reproduction effect on the over-exposed signal light under low illumination. The method provided by the embodiment of the present application does not require additional equipment, is inexpensive, and can achieve the effect of color reproduction, and has high environmental adaptability, and can perform color reproduction of over-exposed signal lights under low illumination. In the prior art, the color of the signal lights can be restored based on the RGB space of the image, or the area of the signal lights in the image can be identified and color enhanced based on the method of deep learning, but it may cause the color signal area and the surrounding image to be restored Discontinuous gradient phenomenon. The method provided in the embodiment of the present application is based on the HSV space, and better reflects the brightness information of the physical signal lamp, so that the identification of the signal lamp area is more stable and accurate, and the signal lamp area after color restoration is more continuous with the surrounding image, and the naked eye effect More realistic. In addition, the method provided by the embodiments of the present application is efficient and reliable, and can satisfy the color reproduction effect of the signal light in the video.

The above mainly introduces the solutions provided by the embodiments of the present application from the perspective of the image processing device. It can be understood that, in order to realize the above-mentioned functions, the image processing device includes a hardware structure and/or a software module corresponding to each function. Those skilled in the art should easily realize that, in conjunction with the algorithm steps described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and software. Whether a function is executed by hardware or software driven hardware depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

In the embodiments of the present application, the image processing device may be divided into function modules according to the above method examples. For example, each function module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or software function modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a division of logical functions. In actual implementation, there may be another division manner.

In the case where each functional module is divided corresponding to each function, FIG. 9 shows a possible structural schematic diagram of the image processing device 9 involved in the above embodiment. The image processing device includes: an acquisition unit 901, a determination unit 902, and Processing unit 903. The acquiring unit 901 is used to support the image processing device to perform the process 301 in FIG. 3. The determination unit 902 is used to support the image processing device to perform the processes 303 and 305 in FIG. 3. The processing unit 903 is used to support the image processing device to perform steps 302, 304, and 306 in FIG. Wherein, all relevant content of each step involved in the above method embodiments can be referred to the function description of the corresponding function module, which will not be repeated here.

The steps of the method or algorithm described in conjunction with the disclosure of the present application may be implemented by hardware, or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, which may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, mobile hard disk, read-only optical disk, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an application specific integrated circuit (application specific integrated circuit, ASIC). In addition, the ASIC may be located in the core network interface device. Of course, the processor and the storage medium may also exist as discrete components in the core network interface device.

Those skilled in the art should realize that in one or more of the above examples, the functions described in this application may be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in the image processing device readable medium or transmitted as one or more instructions or codes on the image processing device readable medium. The image processing device readable medium includes an image processing device storage medium and a communication medium, where the communication medium includes any medium that facilitates transferring the image processing device program from one place to another place. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose image processing device.

The specific embodiments described above further describe the purpose, technical solutions, and beneficial effects of this application in detail. It should be understood that the above descriptions are only specific implementations of this application, and are not intended to limit the scope of this application. The scope of protection, any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of this application, shall be included in the scope of protection of this application.

Claims

A method for color restoration of an image, characterized in that it includes:

Acquiring a first image to be processed, the first image including an overexposed first target object;

Determining the first area of the first image according to the saturation and lightness of the pixels of the first image, the saturation of the pixels of the first area is lower than the average value of the saturation of the pixels of the first image, And the brightness of the pixels of the first area is higher than the average value of the brightness of the pixels of the first image; the first area corresponds to the area of the first target object;

Performing binary processing on the first area to obtain a binary image corresponding to the first area;

Determining at least one connected region in the binary image whose area is greater than or equal to a first threshold; the contour of the at least one connected region corresponds to the contour of the first target object;

The color of the at least one connected area is restored.
The method for color restoring an image according to claim 1, wherein before determining the first area of the first image according to the saturation and lightness of the first image, the method further comprises:

Converting the first image from a first space to a hue saturation lightness HSV space, the first space is any one of a luma hue YUV space, a red-green-blue RGB space, or a hue saturation lightness HSL space.
The method for color restoring an image according to claim 1 or 2, wherein when the first target object is a traffic signal light, the restoring the color of the at least one connected area includes:

Acquiring color information of the traffic signal light;

When the traffic signal light is red, adjust the hue of the pixels of the at least one connected area to the red range, and respectively increase and decrease the saturation and brightness of the pixels of the at least one connected area;

When the traffic signal light is yellow, adjust the hue of the pixels of the at least one connected area to a yellow range, and increase and decrease the saturation and brightness of the pixels of the at least one connected area, respectively;

When the traffic light is green, adjust the hue of the pixels of the at least one connected area to the green range, and increase and decrease the saturation and brightness of the pixels of the at least one connected area, respectively.
The method for color restoring an image according to claim 1 or 2, wherein when the first target object is a traffic signal light, the restoring the color of the at least one connected area includes:

Convert the at least one connected region to RGB space;

Acquiring color information of the traffic signal light;

When the traffic signal light is red, adjust the red component of the pixels of the at least one connected area to the first preset range, and reduce the blue and green components of the pixels of the at least one connected area;

When the traffic signal light is yellow, adjust the red and green components of the pixels of the at least one connected area to a second preset range, and reduce the blue component of the pixels of the at least one connected area;

When the traffic signal light is green, adjust the green component of the pixels of the at least one connected area to a third preset range, and reduce the red and blue components of the pixels of the at least one connected area.
The method for color restoring an image according to any one of claims 1 to 4, wherein before determining the first area of the first image according to the saturation and lightness of the first image, all The method also includes:

Performing filtering processing on the saturation component and the lightness component of the pixels of the first image.
The method for color restoring an image according to any one of claims 1 to 5, wherein the acquiring the first image to be processed includes:

The second area selected by the user on the second image is used as the first image to be processed.
An image processing device, characterized in that it includes:

An obtaining unit, configured to obtain a first image to be processed, the first image including an overexposed first target object;

A determining unit for determining the first area of the first image according to the saturation and lightness of the pixels of the first image, the saturation of the pixels of the first area is lower than the saturation of the pixels of the first image An average value of degrees, and the brightness of the pixels of the first area is higher than the average value of the brightness of the pixels of the first image; the first area corresponds to the area of the first target object;

A processing unit, configured to perform binarization processing on the first area to obtain a binary image corresponding to the first area;

The determining unit is further configured to determine at least one connected region whose area in the binary image is greater than or equal to a first threshold; the contour of the at least one connected region corresponds to the contour of the first target object;

The processing unit is also used to restore the color of the at least one connected area.
The image processing apparatus according to claim 7, wherein the processing unit is further configured to:

Converting the first image from a first space to a hue saturation lightness HSV space, the first space is any one of a luma hue YUV space, a red-green-blue RGB space, or a hue saturation lightness HSL space.
The image processing device according to claim 7 or 8, wherein when the first target object is a traffic signal, the processing unit is configured to:

Acquiring the color information of the traffic signal light through the acquiring unit;

When the traffic signal light is red, adjust the hue of the pixels of the at least one connected area to the red range, and respectively increase and decrease the saturation and brightness of the pixels of the at least one connected area;

When the traffic signal light is yellow, adjust the hue of the pixels of the at least one connected area to a yellow range, and increase and decrease the saturation and brightness of the pixels of the at least one connected area, respectively;

When the traffic light is green, adjust the hue of the pixels of the at least one connected area to the green range, and increase and decrease the saturation and brightness of the pixels of the at least one connected area, respectively.
The image processing device according to claim 7 or 8, wherein when the first target object is a traffic signal, the processing unit is configured to:

Convert the at least one connected region to RGB space;

Acquiring the color information of the traffic signal light through the acquiring unit;

When the traffic signal light is red, adjust the red component of the pixels of the at least one connected area to the first preset range, and reduce the blue and green components of the pixels of the at least one connected area;

When the traffic light is yellow, adjust the red and green components of the pixels of the at least one connected area to a second preset range, and reduce the blue component of the pixels of the at least one connected area;

When the traffic signal light is green, adjust the green component of the pixels of the at least one connected area to a third preset range, and reduce the red and blue components of the pixels of the at least one connected area.
The image processing apparatus according to any one of claims 7-10, wherein the processing unit is further configured to:

Performing filtering processing on the saturation component and the lightness component of the pixels of the first image.
The image processing device according to any one of claims 7-11, wherein the acquisition unit is configured to:

The second area selected by the user on the second image is used as the first image to be processed. 13. A computer-readable storage medium, including instructions, characterized in that, when the instructions are run on a computer, the computer is caused to perform the method for color restoration of an image according to any one of claims 1 to 6.