WO2024060360A1 - 图像融合方法、装置、电子设备及存储介质 - Google Patents

图像融合方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2024060360A1
WO2024060360A1 PCT/CN2022/129361 CN2022129361W WO2024060360A1 WO 2024060360 A1 WO2024060360 A1 WO 2024060360A1 CN 2022129361 W CN2022129361 W CN 2022129361W WO 2024060360 A1 WO2024060360 A1 WO 2024060360A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
display device
value
pixels
environment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/129361
Other languages
English (en)
French (fr)
Inventor
李宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen TCL New Technology Co Ltd
Original Assignee
Shenzhen TCL New Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen TCL New Technology Co Ltd filed Critical Shenzhen TCL New Technology Co Ltd
Publication of WO2024060360A1 publication Critical patent/WO2024060360A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present invention relates to the field of image processing technology, and in particular, to an image fusion method, device, electronic equipment and storage medium.
  • Display devices such as TVs, monitors, billboards, etc., usually appear as a black screen when they are not working, and cannot integrate with the surrounding environment.
  • Environment integration means that the display device obtains content from the surrounding environment, generates pictures close to the environment, and displays them on the display device, integrating the display device and the surrounding environment to enhance the user's residential visual experience.
  • the present invention provides an image fusion method, device, electronic equipment and storage medium to solve the problem in the prior art that environmental images can only be obtained from above the TV and the fusion effect is not good.
  • the present invention provides an image fusion method, which method includes:
  • the environment image is an image representing the display device and the surrounding environment of the display device;
  • the brightness of the overlay area is adjusted to generate a target image that blends with the surrounding environment of the display device.
  • the step of performing image filling from at least two directions of the display device in the environment image to obtain the overlay area includes:
  • the target direction is determined according to the total number of pixels, and image filling is performed on the target direction to obtain an overlapping area.
  • the at least two directions include four directions: up, down, left, and right, and the step of calculating the total number of pixels corresponding to each direction includes:
  • the number of probing rectangles is configured based on the ratio of the length and width of the display device to obtain a first number of probing rectangles in the up and down directions and a second number of probing rectangles in the left and right directions.
  • the probing rectangles are used to obtain RGB values. Pixels smaller than the preset value;
  • the number of pixels detected in the four directions of up, down, left, and right by the detection rectangle is accumulated and saved to obtain the total number of pixels in the corresponding directions.
  • the step of configuring the number of probing rectangles based on the ratio of the length and width of the display device includes:
  • the width of each exploration rectangle is 8 to 10 pixels
  • the lengths of the exploration rectangles in the left and right directions are W/2 ⁇ 3W/4
  • the heights of the exploration rectangles in the up and down directions are H/2 ⁇ 3H/4
  • W represents the number of lengths of pixels occupied by the display device in the environment image
  • H represents the number of widths of pixels occupied by the display device in the environment image.
  • the step of accumulating and saving the number of pixels detected in the four directions of up, down, left and right by the detection rectangle to obtain the total number of pixels in the corresponding directions includes:
  • variables corresponding to the upper, lower, left and right are top_count, bottom_count, left_count and right_count respectively, and the pixels whose RGB value is smaller than the preset value represent pixels with a darker color.
  • the step of determining a target direction based on the total number of pixels and performing image filling on the target direction to obtain the overlapping area includes:
  • the first average value is obtained, and according to the calculation formula (left_count+right_count)/second quantity, the second average value is obtained;
  • Compare the first average value and the second average value take the direction with the smaller value as the target direction for content acquisition, and perform image filling on the display device in the environment image with the acquired content to obtain an overlay. bottom area.
  • the step of comparing the first average value and the second average value, taking the direction of the smaller value as the target direction to acquire content, and performing image filling on the display device in the environment image with the acquired content to obtain the overlapping area further includes:
  • the first target direction is the left direction and the second target direction is the right direction.
  • To obtain the first image and select an image with a length of W/2+B1 and a width of H in the right direction of the display device on the environment image for filling and horizontal flipping to obtain the second image;
  • W represents the number of pixel lengths occupied by the display device in the environment image
  • A1 represents the number of pixel lengths occupied
  • B1 represents the number of pixels occupied by the length
  • the calculation formula of the RGB value of the overlapping area is:
  • Result Color represents the RGB value of the stacked bottom area
  • Top Color represents the RGB value of A1
  • Bottom Color represents the RGB value of B1.
  • the step of adjusting the brightness of the overlay area to generate a target image that blends with the surrounding environment of the display device includes:
  • the first average brightness value is the average brightness value of the W/2-B1-1 column in the third image
  • the second average brightness is the The average brightness value of column W/2+A1+1 in the third image
  • the length of the overlay area is A1+B1 and the height is H, where H represents the number of widths of pixels occupied by the display device in the environment image.
  • the step of gradually adjusting the brightness of the overlay area from the first average brightness value to the second average brightness value to obtain the target image include:
  • the decreasing value of the gradient is calculated.
  • the calculation formula of the decreasing value is:
  • ⁇ Light represents the decrement value
  • LA represents the first average brightness value
  • LB represents the second average brightness value.
  • the adjustment formula for the RGB value of the overlay area is:
  • the step of obtaining an environment image includes:
  • the shooting interface of the display device or mobile terminal, where the shooting interface has a rectangular frame;
  • the image of the display device is placed within the rectangular frame to obtain the environment image.
  • the method before the step of performing image filling from at least two directions of the display device in the environment image to obtain the overlapping area, the method further includes:
  • the position information includes the X and Y coordinates of the display device in the environment image, the number W representing the length of the pixels occupied by the display device in the environment image, and the number W representing the position of the display device in the environment image.
  • the method further includes:
  • the target image is directly displayed on the display device.
  • the target image is sent to the display device for display on the display device.
  • the present invention also provides an image fusion device, which includes:
  • An acquisition module configured to acquire an environment image, where the environment image is an image representing the display device and the surrounding environment of the display device;
  • An overlay processing module configured to perform image filling from at least two directions of the display device in the environmental image to obtain an overlay area, where the brightness of the overlay area is different from the brightness of the non-overlay area;
  • a fusion module configured to perform brightness adjustment from the overlay area to generate a target image that is blended with the surrounding environment of the display device.
  • the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the steps of the image fusion method as described in any one of the first aspects are implemented.
  • the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the image fusion method as described in any one of the first aspects are implemented.
  • the invention provides an image fusion method, device, electronic device and storage medium, which fills in at least two directions of the display device in the acquired environmental image and adjusts the brightness to generate an image corresponding to the surroundings of the display device.
  • the target image of environmental fusion enables the display device and the surrounding environment to be integrated in different directions, improving the user's residential visual experience.
  • Figure 1 is a schematic flow chart of the image fusion method provided by the present invention.
  • Figure 2 is a schematic diagram of a rectangular frame provided by an embodiment of the present invention.
  • FIG3 is a rendering of an environment image provided by an embodiment of the present invention.
  • Figure 4 is a rendering of high-pass filtering of an environmental image provided by an embodiment of the present invention.
  • FIG5 is a schematic diagram of a detection rectangle provided by an embodiment of the present invention.
  • Figure 6 is a second schematic diagram of a probing rectangle provided by an embodiment of the present invention.
  • Figure 7 is one of the renderings of image filling provided by the embodiment of the present invention.
  • FIG8 is a second effect diagram of image filling provided by an embodiment of the present invention.
  • Figure 9 is the third effect diagram of image filling provided by the embodiment of the present invention.
  • Figure 10 is a rendering of the bottom-stacked area provided by the embodiment of the present invention.
  • Figure 11 is a rendering of a target image provided by an embodiment of the present invention.
  • Figure 12 is a schematic structural diagram of the image fusion device provided by the present invention.
  • Figure 13 is a schematic structural diagram of an image fusion device provided by an embodiment of the present invention.
  • Figure 14 is a schematic structural diagram of the electronic device provided by the present invention.
  • Environmental integration means that the display device obtains content from the surrounding environment, generates pictures that are close to the environment, and displays them on the display device, integrating the display device with the surrounding environment and improving the user's residential visual experience.
  • the present invention provides an image fusion method, device, electronic device and storage medium.
  • the display device is filled with images in at least two directions, and the brightness is adjusted to generate a target image that is integrated with the surrounding environment of the display device, so that the display device and the surrounding environment can be integrated in different directions to enhance the user's living vision. experience.
  • Figure 1 is a schematic flow chart of an image fusion method provided by the present invention. It is an image fusion method. The method includes:
  • Step 110 Obtain an environment image, which is an image representing the display device and the surrounding environment of the display device.
  • the environment image can be obtained through a shooting device provided on the display device, or the environment image can be obtained through a shooting device (such as a mobile phone) that is independent of the display device. Therefore, the present invention does not limit the method of acquiring the environment image.
  • Step 120 Perform image filling from at least two directions of the display device in the environment image to obtain an overlay area, where the brightness of the overlay area is lower than the brightness of the non-overlay area.
  • the present invention adapts and extends the display device to the surrounding environment in different directions, and selects the direction to obtain the displayed content according to the intensity of color changes around the display device.
  • Step 130 Adjust the brightness of the overlay area to generate a target image that blends with the surrounding environment of the display device.
  • the present invention performs deep fusion processing on the display content to generate a fused image. Integrate the TV with the surrounding environment to enhance the user's residential visual experience.
  • the step of obtaining an environment image includes:
  • Step 111 Call the shooting interface of the display device or mobile terminal (such as a mobile phone).
  • the shooting interface has a rectangular frame.
  • Step 112 Place the image of the display device within the rectangular frame to obtain the environment image.
  • the display device (such as a TV) needs to be in a rectangular frame, and the rectangular frame is in the middle of the camera interface.
  • the rectangular frame is a rectangle placed horizontally.
  • the display device can be completely within the rectangular frame. It does not need to be completely fitted, as long as it is inside the rectangular frame.
  • the image fusion method further includes:
  • Step 113 Determine the position of the display device in the environment image and save the position information.
  • the captured environment image is as shown in Figure 3
  • the position information of the display device in the environment image is (X, Y, W, H)
  • X and Y represent the coordinates of the display device in the environment image
  • W represents the number of lengths of pixels occupied by the display device in the environment image
  • H represents the number of widths of pixels occupied by the display device in the environment image.
  • the position information is obtained by traversing the display device (ie, the black area) within the rectangular frame in the environment image.
  • the environment image can be opened through the bitmap class, the image at the specified position (ie, within the rectangular frame) is read and saved in a two-dimensional array, and the pixels are traversed from left to right and top to bottom. If a pixel with an RGB value smaller than the preset value (for example, 35) is encountered, it is judged whether the pixels to the right and below the pixel meet the requirements. If so, the pixel on the right is cut to repeat the judgment. Until you find something that doesn't meet the requirements. Then cut to the pixel below the pixel for judgment. The area with the most consecutive points is used last. It should be noted that pixels with RGB values less than 35 represent dark areas, that is, black areas represented by the display device.
  • Step 114 Perform high-pass filtering on the environment image to obtain a processed environment image.
  • High-pass filtering is performed on the environment image to obtain a processed environment image (as shown in Figure 4).
  • High-pass filtering is a filtering method. The rule is that high-frequency signals can pass through normally, while low-frequency signals below the set threshold are blocked and weakened.
  • the step of performing image filling from at least two directions of the display device in the environment image to obtain the overlay area includes:
  • Step 121 Calculate the total number of pixels corresponding to each direction, where the total number of pixels represents the number of pixels whose RGB values are less than the preset value.
  • Step 122 Determine a target direction according to the total number of pixels, and perform image filling on the target direction to obtain an overlapping area.
  • the present invention can perform image filling on at least two directions of the display device in the environment image, such as the left direction and the right direction, the upper direction and the lower direction, or the left direction and the upper direction, or the left direction. , right direction and up direction, or the four directions of up, down, left and right. Therefore, the present invention does not limit the direction of image filling.
  • the following description takes the at least two directions including four directions of up, down, left and right as an example.
  • the total number of pixels corresponding to each direction is calculated.
  • the steps include:
  • Step 1211 Configure the number of exploration rectangles based on the ratio of the length and width of the display device to obtain a first number of exploration rectangles in the up and down directions and a second number of exploration rectangles in the left and right directions.
  • the exploration rectangles are used for Get the pixels whose RGB value is less than the preset value.
  • exploration rectangles rise from the edges of the display device in the environment image.
  • the widths are 8 to 10 pixels, and the lengths of the exploration rectangles in the left and right directions are both W/2.
  • ⁇ 3W/4 such as W/2
  • the height of the upper exploration rectangle is H/2 ⁇ 3H/4 (such as H/2)
  • the height of the lower exploration rectangle is also H/4 ⁇ 3H/4 (such as H /2)
  • the exploration rectangle below is not close to the display device, but has to be translated downward by a preset number (for example, 20) pixels. This is because there are devices such as panels, infrared heads, speakers, etc. at the bottom of the display device. , causing the bottom border to be thicker than the other three sides.
  • the ratio of the length to the width of the display device is 16:9, 16 exploration rectangles are obtained in the up-down direction and 9 exploration rectangles are obtained in the left-right direction (as shown in FIG. 6 ).
  • Step 1212 Accumulate and save the number of pixels detected in the four directions of up, down, left and right by the detection rectangle to obtain the total number of pixels in the corresponding directions.
  • corresponding variables are configured based on the four directions of up, down, left, and right, and the number of pixels whose RGB values are less than the preset value detected by the exploration rectangle corresponding to the up, down, left, and right directions are respectively accumulated and saved to the corresponding variables to obtain the corresponding The total number of pixels in the direction.
  • the variables corresponding to the upper, lower, left and right are top_count, bottom_count, left_count and right_count respectively.
  • the pixels whose RGB value is less than the preset value represent the pixels with darker color, that is, the above variables are used to store the total number of darker pixels.
  • the above-mentioned variables are also used to count the results of probing. When traversing within the probing rectangle and encountering darker pixels (that is, RGB is less than 35), the corresponding value will be increased by one.
  • the step of determining a target direction based on the total number of pixels and performing image filling on the target direction to obtain the overlay area includes:
  • Step 1221 Obtain the first average value according to the calculation formula (top_count+botton_count)/first quantity*preset weight, and obtain the second average value according to the calculation formula (left_count+right_count)/second quantity.
  • top_count+botton_count)/16*110% represents the first average value of changes in the up-down direction
  • (left_count+right_count)/9 represents the second average value of changes in the left-right direction.
  • the above preset weight (for example, 110%) is set because: first, display devices include wall-mounted display devices and desktop display devices, and desktop display devices have brackets. Generally speaking, the area of the bracket It will be less than 10% of the area of the display device; secondly, the border at the bottom of the display device will be thicker than other borders, and the black border at the bottom will make the overall visual effect more incoherent. Therefore, based on the above purpose, in order to achieve better fusion effect, it is necessary to increase the preset weight in the upper and lower directions.
  • Step 1222 Compare the first average value and the second average value, take the direction with the smaller value as the target direction for content acquisition, and fill the display device in the environment image with the acquired content. to get the bottom area.
  • the direction in which the content is acquired in the present invention is the up-down direction or the left-right direction, and the direction with a smaller value is because the smaller the value, it means that the color change of the environment image in this direction is smaller, that is, the content is relatively single. . Therefore, the less changing images are selected for filling, the better the fusion effect will be. If the first average value and the second average value are equal, the left and right directions are selected as the target directions.
  • first compare the total number of pixels in two opposite directions in the target direction, take the direction with the smaller value as the first target direction, and take the direction with the larger value as the second target direction.
  • the target direction determined above is the left and right direction
  • W represents the number of pixel lengths occupied by the display device in the environmental image
  • A1 represents the number of pixel lengths occupied
  • B1 represents the number of pixel lengths occupied
  • the selected numbers here should be in W/10 ⁇ A1+B1 ⁇ W/2, (A1, B1 are left and right (more than half of the part on the side), the closer it is to W/2, the better the superposition effect will be, but the larger the superimposed part will change the smoothness and beauty of the original image; and the closer it is to W/10, it will cause the brightness transition later. Too vigorous and you won't get a good blending effect.
  • the filled image can be horizontally flipped, but the present invention can also perform other processing, such as angle rotation, vertical flipping, etc., so the present invention does not specify what kind of processing is performed on the filled image. Make limitations.
  • the RGB values of the overlapping area are calculated to obtain a third image including the first image and the second image (as shown in Figure 10).
  • the overlapping area is the first image and the second image.
  • the calculation formula of the RGB value of the stacked bottom area is:
  • Result Color represents the RGB value of the stacked bottom area
  • Top Color represents the RGB value of A1
  • Bottom Color represents the RGB value of B1.
  • R, G, and B in the pixel (the data of the three colors of Red, Green, and Blue are operated separately).
  • the red formula becomes:
  • the red value of the last pixel (red of the Top image * red of the Button image)/255, modify the value to the response pixel, and save the final one as the third image.
  • the step of adjusting the brightness of the overlay area to generate a target image that blends with the surrounding environment of the display device includes:
  • Step 131 obtaining a first average brightness value and a second average brightness value, wherein the first average brightness value is the average brightness value of the W/2-B1-1th column in the third image, and the second average brightness value is the average brightness value of the W/2+A1+1th column in the third image.
  • Step 132 Gradually change the overlay area from the first average brightness value to the second average brightness value to adjust the brightness of the overlay area to obtain the target image;
  • a gradient is performed on the above-mentioned brightness, and a decreasing value is obtained by subtracting a large average brightness value from a small average brightness value, and then dividing by pixels.
  • the decrement value of the gradient is calculated.
  • the calculation formula of the decrement value ⁇ Light is:
  • the brightness of the first column on the left through the stacked bottom area (the length is A1+B1 and the height is H, H represents the number of pixel widths occupied by the display device in the environmental image) is LA, and gradually decreases until LB, the target image after brightness adjustment is shown in Figure 11.
  • the adjustment formula for the RGB value of the overlay area is:
  • Red (Red value in the pixel of the Top image in the stacked bottom area * Red value in the pixel of the Bottom image in the stacked bottom area + brightness decrease value of the column)/255;
  • the method includes:
  • Step 140 If the environment image is obtained through the display device, the target image is directly displayed on the display device; or
  • Step 150 If the environment image is obtained through the mobile terminal, the target image is sent to the display device for display on the display device.
  • the target image generated by the mobile terminal For example, push the target image generated by the mobile terminal to the display device and display it. If the above image processing algorithm is set on the display device, then the photographed environment image will be sent to the display device for processing. After processing, the generated target image will be The image is displayed.
  • the present invention adapts and extends the surrounding environment of the display device in different directions. According to the intensity of the color change around the display device, the direction to obtain the filling content is selected, and the filling content is deeply fused to achieve Generate fusion images to integrate the display device and the surrounding environment, improving the user's residential visual experience and pan-intelligent screen usage experience.
  • the image fusion device provided by the present invention is described below.
  • the image fusion device described below and the image fusion method described above can be referenced correspondingly.
  • An image fusion device 1200 includes an acquisition module 1210, a stack processing module 1220, and a fusion module 1230.
  • the acquisition module 1210 is used to acquire an environment image, where the environment image is an image representing the display device and the surrounding environment of the display device.
  • the overlay processing module 1220 is used to perform image filling from at least two directions of the display device in the environment image to obtain an overlay area, and the brightness of the overlay area is different from the non-overlay area. brightness.
  • the fusion module 1230 is used to adjust the brightness of the overlay area to generate a target image that is blended with the surrounding environment of the display device.
  • the stacking processing module 1220 is also used to:
  • the target direction is determined according to the total number of pixels, and image filling is performed on the target direction to obtain the overlapping area.
  • the stacking processing module 1220 is also used to:
  • the number of pixels detected in the four directions of up, down, left, and right by the detection rectangle is accumulated and saved to obtain the total number of pixels in the corresponding directions.
  • the stacking processing module 1220 is also used to:
  • the width of each exploration rectangle is 8 to 10 pixels
  • the lengths of the exploration rectangles in the left and right directions are W/2 ⁇ 3W/4
  • the heights of the exploration rectangles in the up and down directions are H/2 ⁇ 3H/4
  • W represents the number of lengths of pixels occupied by the display device in the environment image
  • H represents the number of widths of pixels occupied by the display device in the environment image.
  • the stacking processing module 1220 is also used to:
  • the variables corresponding to the upper, lower, left, and right are top_count, bottom_count, left_count, and right_count respectively.
  • the pixels whose RGB value is smaller than the preset value represent pixels with a darker color.
  • the stacking processing module 1220 is also used to:
  • the first average value is obtained, and according to the calculation formula (left_count+right_count)/second quantity, the second average value is obtained;
  • Compare the first average value and the second average value take the direction with the smaller value as the target direction for content acquisition, and perform image filling on the display device in the environment image with the acquired content to obtain an overlay. bottom area.
  • the stacking processing module 1220 is also used to:
  • the first target direction is the left direction and the second target direction is the right direction.
  • To obtain the first image and select an image with a length of W/2+B1 and a width of H in the right direction of the display device on the environment image for filling and horizontal flipping to obtain the second image;
  • W represents the number of pixel lengths occupied by the display device in the environment image
  • A1 represents the number of pixel lengths occupied
  • B1 represents the number of pixels occupied by the length
  • the calculation formula of the RGB value of the stacked bottom area is:
  • Result Color represents the RGB value of the stacked bottom area
  • Top Color represents the RGB value of A1
  • Bottom Color represents the RGB value of B1.
  • the fusion module 1230 is also used to:
  • the first average brightness value is the average brightness value of the W/2-B1-1 column in the third image
  • the second average brightness is the The average brightness value of column W/2+A1+1 in the third image
  • the length of the overlay area is A1+B1 and the height is H, where H represents the number of widths of pixels occupied by the display device in the environment image.
  • the fusion module 1230 is also used to:
  • ⁇ Light represents the decrement value
  • LA represents the first average brightness value
  • LB represents the second average brightness value.
  • the adjustment formula for the RGB value of the overlay area is:
  • the acquisition module 1210 is also used to:
  • the shooting interface of the display device or mobile terminal, where the shooting interface has a rectangular frame;
  • the image of the display device is placed within the rectangular frame to obtain the environment image.
  • the image fusion device 1200 further includes a preprocessing module, and the preprocessing module is used to:
  • the position information includes the X and Y coordinates of the display device in the environment image, the number W representing the length of the pixels occupied by the display device in the environment image, and the number W representing the position of the display device in the environment image.
  • the image fusion device 1200 further includes a display module, the display module is used for:
  • the target image is directly displayed on the display device.
  • the target image is sent to the display device for display on the display device.
  • FIG. 13 is a schematic structural diagram of an image fusion device provided by an embodiment of the present invention.
  • the image fusion device is applied to a mobile terminal, sends the generated target image to the display device, receives the target image through the receiving module of the display device, and displays the received target image through the display module.
  • Figure 14 illustrates a schematic diagram of the physical structure of an electronic device.
  • the electronic device may include: a processor (Processor) 1410, a communication interface (Communications Interface) 1420, a memory (Memory) 1430 and a communication bus 1440.
  • the processor 1410, the communication interface 1420, and the memory 1430 complete communication with each other through the communication bus 1440.
  • the processor 1410 can call logical instructions in the memory 1430 to execute the image fusion method, which includes:
  • the environment image is an image representing the display device and the surrounding environment of the display device;
  • the brightness of the overlay area is adjusted to generate a target image that blends with the surrounding environment of the display device.
  • the above-mentioned logical instructions in the memory 1430 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of the present invention essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .
  • the present invention also provides a computer program product.
  • the computer program product includes a computer program stored on a non-transitory computer-readable storage medium.
  • the computer program includes program instructions. When the program instructions are read by a computer, When executed, the computer can execute the image fusion method provided by each of the above methods.
  • the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is implemented when executed by a processor to execute the image fusion methods provided above.
  • An embodiment of the present invention provides an electronic device, a computer program product, and a processor-readable storage medium, on which the computer program stored enables the processor to implement all the method steps implemented by the above-mentioned method embodiment and can achieve the same technical effect.
  • the parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail herein.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated.
  • the components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.
  • each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disc, optical disk, etc., including a number of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Image Processing (AREA)

Abstract

本发明提供了一种图像融合方法、装置、电子设备及存储介质，所述方法包括获取一环境图像；从环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域；对叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。本发明能够实现在不同方向上让显示设备和周围环境融为一体，提升用户的居住视觉体验。

Description

图像融合方法、装置、电子设备及存储介质
相关申请
本申请要求2022年09月22日申请的，申请号为202211160692.7，名称为“图像融合方法、装置、电子设备及存储介质”的中国专利申请的优先权，在此将其全文引入作为参考。
技术领域
本发明涉及图像处理技术领域，尤其涉及一种图像融合方法、装置、电子设备及存储介质。
背景技术
显示设备，例如电视、显示器、广告牌等，通常在未工作之时，以黑屏的形式出现，而不能与周围环境相融合。环境融合是指显示设备从周围的环境中获取内容，生成贴近环境的图片，并显示在该显示设备上，让显示设备和周围环境融为一体，提升用户的居住视觉体验。
目前一些电视已经有自己的环境融合模式，但只能从电视的上方获取环境图像，而且融合效果不好。
发明内容
本发明提供一种图像融合方法、装置、电子设备及存储介质，用以解决现有技术中只能从电视的上方获取环境图像且融合效果不好的问题。
为实现上述目的，本发明提供如下技术方案：
第一方面，本发明提供一种图像融合方法，所述方法包括：
获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像；
从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度低于非叠底区域的亮度；
对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
在本发明一实施例中，所述从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域的步骤包括：
计算每个方向对应的像素总数，所述像素总数表示RGB值小于预设值的像素的个数；
根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域。
在本发明一实施例中，所述至少两个方向包括上下左右四个方向，所述计算每个方向对应的像素总数的步骤包括：
基于所述显示设备的长度和宽度的比例配置探查矩形的数量，以得到上下方向各为第一数量的探查矩形和左右方向各为第二数量的探查矩形，所述探查矩形用于获取RGB值小于预设值的像素；
将所述探查矩形探查到上下左右四个方向的像素的个数进行累计保存以得到对应方向的像素总数。
在本发明一实施例中，所述基于所述显示设备的长度和宽度的比例配置探查矩形的数量的步骤包括：
若所述显示设备的长度和宽度的比例是16：9，则得到上下方向各为16个探查矩形和左右方向各为9个探查矩形；
其中，每个探查矩形的宽度为8～10个像素，左右方向的探查矩形的长度均为W/2～3W/4，上下方向的探查矩形的高度均为H/2～3H/4，W表示显示设备在所述环境图像中所占像素的长度的个数，H表示显示设备在所述环境图像中所占像素的宽度的个数。
在本发明一实施例中，所述将所述探查矩形探查到上下左右四个方向的像素的个数进行累计保存以得到对应方向的像素总数的步骤包括：
基于上下左右四个方向配置对应的变量；
将上下左右方向对应的探查矩形探查到RGB值小于预设值的像素的个数分别进行累计并保存至对应的变量，以得到对应方向的像素总数；
其中，所述上下左右对应的变量分别为top_count、botton_count、left_count 以及right_count，所述RGB值小于预设值的像素表示颜色偏黑的像素。
在本发明一实施例中，所述根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域的步骤包括：
根据计算公式(top_count+botton_count)/第一数量*预设权重，得到第一平均值，并根据计算公式(left_count+right_count)/第二数量，得到第二平均值；
比较所述第一平均值和所述第二平均值，取较小值的方向作为目标方向以进行内容获取，并将获取到的内容对所述环境图像中的显示设备进行图像填充以得到叠底区域。
在本发明一实施例中，所述比较所述第一平均值和所述第二平均值，取较小值的方向作为目标方向以进行内容获取，并将获取到的内容对所述环境图像中的显示设备进行图像填充以得到叠底区域的步骤还包括：
比较所述目标方向中的两个相对方向的像素总数，将较小值的方向作为第一目标方向，将较大值的方向作为第二目标方向；
假设所述第一目标方向为左方向，所述第二目标方向为右方向，选择所述环境图像上显示设备左方向长度为W/2+A1和宽度为H的图像进行填充并进行水平翻转以得到第一图像，以及选择所述环境图像上显示设备右方向长度为W/2+B1和宽度为H的图像进行填充并进行水平翻转以得到第二图像；
计算所述叠底区域的RGB值，得到包括所述第一图像和所述第二图像的第三图像，所述叠底区域为所述第一图像和所述第二图像的交叉重叠部分；
其中，W表示显示设备在所述环境图像中所占像素的长度的个数，A1表示所占像素的长度的个数，B1表示所占像素的长度的个数，A1>B1且W/10<A1+B1<W/2。
在本发明一实施例中，所述叠底区域的RGB值的计算式为：
Result Color＝(Top Color)*(Bottom Color)/255；
其中，Result Color表示所述叠底区域的RGB值，Top Color表示A1的RGB值，Bottom Color表示B1的RGB值。
在本发明一实施例中，所述对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像的步骤包括：
获取第一平均亮度值和第二平均亮度值，所述第一平均亮度值为所述第三图像中第W/2-B1-1列的平均亮度值，所述第二平均亮度为所述第三图像中第W/2+A1+1列的平均亮度值；
对所述叠加区域从所述第一平均亮度值渐变至所述第二平均亮度值以对所述叠底区域进行亮度调整以得到所述目标图像；
其中，所述叠加区域的长度为A1+B1且高度为H，H表示显示设备在所述环境图像中所占像素的宽度的个数。
在本发明一实施例中，所述对所述叠加区域从所述第一平均亮度值渐变至所述第二平均亮度值以对所述叠底区域进行亮度调整以得到所述目标图像的步骤包括：
假设所述第一平均亮度值大于所述第二平均亮度值，则计算渐变的递减值，所述递减值的计算式为：
△Light＝(LA-LB)/(A1+B1)；
其中，△Light表示递减值，LA表示第一平均亮度值，LB表示第二平均亮度值。
在本发明一实施例中，所述叠底区域的RGB值的调整公式为：
Result Color＝((Top Color)*(Bottom Color)+△Light)/255。
在本发明一实施例中，所述获取一环境图像的步骤包括：
调用所述显示设备或移动终端的拍摄界面，所述拍摄界面具有一矩形框；
将所述显示设备的图像置于所述矩形框内以获取到所述环境图像。
在本发明一实施例中，在所述从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域步骤之前，所述方法还包括：
确定所述显示设备在所述环境图像中的位置并保存该位置信息；
对所述环境图像进行高通滤波处理以得到处理后的环境图像；
其中，所述位置信息包括所述显示设备在所述环境图像中的X、Y坐标、表示显示设备在该环境图像中所占像素的长度的个数W以及表示显示设备在该环境图像中所占像素的宽度的个数H。
在本发明一实施例中，在所述对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像步骤之后，所述方法还包括：
若通过所述显示设备获取所述环境图像，则将所述目标图像直接显示在所述显示设备上；或者
若通过移动终端获取所述环境图像，则将所述目标图像发送至所述显示设备以在所述显示设备上进行显示。
第二方面，本发明还提供一种图像融合装置，所述装置包括：
获取模块，用于获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像；
叠底处理模块，用于从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度不同于非叠底区域的亮度；
融合模块，用于从所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
第三方面，本发明还提供一种电子设备，包括存储器、处理器及存储在所述存储器上并在该处理器上运行的计算机程序，所述处理器执行所述程序时实现如第一方面任一项所述的图像融合方法的步骤。
第四方面，本发明还提供非暂态计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如第一方面任一项所述的图像融合方法的步骤。
本发明提供的一种图像融合方法、装置、电子设备及存储介质，通过对获取到的环境图像中的显示设备的至少两个方向进行图像填充，并进行亮度调整以生成与该显示设备的周围环境相融合的目标图像，实现在不同方向上让显示设备和周围环境融为一体，提升用户的居住视觉体验。
附图说明
下面结合附图，通过对本发明的具体实施方式详细描述，将使本发明的技术方案及其它有益效果显而易见。
图1是本发明提供的图像融合方法的流程示意图；
图2是本发明实施例提供的矩形框的示意图；
图3是本发明实施例提供的环境图像的效果图；
图4是本发明实施例提供的对环境图像进行高通滤波处理后的效果图；
图5是本发明实施例提供的探查矩形的示意图之一；
图6是本发明实施例提供的探查矩形的示意图之二；
图7是本发明实施例提供的图像填充的效果图之一；
图8是本发明实施例提供的图像填充的效果图之二；
图9是本发明实施例提供的图像填充的效果图之三；
图10是本发明实施例提供的叠底区域的效果图；
图11是本发明实施例提供的目标图像的效果图；
图12是本发明提供的图像融合装置的结构示意图；
图13是本发明实施例提供的图像融合装置的结构示意图；
图14是本发明提供的电子设备的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述。显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。
以下对本发明涉及的技术术语进行描述：
环境融合，是指显示设备从周围的环境中获取内容，生成贴近环境的图片，并显示在该显示设备上，让显示设备和周围环境融为一体，提升用户的居住视觉体验。
为了解决现有技术中只能从电视的上方获取环境图像且融合效果不好的问题，本发明提供的一种图像融合方法、装置、电子设备及存储介质，通过对获取到的环境图像中的显示设备的至少两个方向进行图像填充，并进行亮度调整以生成与该显示设备的周围环境相融合的目标图像，实现在不同方向上让显示设备和周围环境融为一体，提升用户的居住视觉体验。
下面结合图1-图14描述本发明的图像融合方法、装置、电子设备及存储介质。
请参考图1，图1是本发明提供的图像融合方法的流程示意图，一种图像融合方法，所述方法包括：
步骤110，获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像。
示例性地，可以通过显示设备上自带的拍摄设备获取所述环境图像，也可以通过独立于显示设备的拍摄设备(例如手机)获取所述环境图像。故，本发明对于获取所述环境图像的方式不做限定。
步骤120，从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度低于非叠底区域的亮度。
示例性地，与现有技术相比，本发明是显示设备对周围环境在不同方向上的适应和延伸，并根据显示设备周围颜色变化的剧烈程度，选择获得显示内容的方向。
步骤130，对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
示例性地，本发明对显示内容进行深度融合处理，生成融合图像。让电视和周围环境融为一体，提升用户的居住视觉体验。
以下对上述步骤110～步骤130进行具体描述。
示例性地，上述步骤110中，所述获取一环境图像的步骤包括：
步骤111，调用显示设备或移动终端(例如手机端)的拍摄界面，所述拍摄界面具有一矩形框。
步骤112，将显示设备的图像置于矩形框内以获取到所述环境图像。
例如，打开移动终端的应用程序，并打开拍摄界面，拍摄界面中具有一矩形框(如图2所示)。拍摄时，需要让显示设备(例如电视)处于矩形框中，矩形框在拍照界面的中间。矩形框为横放的长方形，拍照时可以让显示设备完整的处于矩形框内，可不需要完全贴合，在矩形框内部即可。
示例地，在执行上述120之前，所述图像融合方法还包括：
步骤113，确定所述显示设备在所述环境图像中的位置并保存该位置信息。
示例性地，假设拍摄到的环境图像如图3所示，假设显示设备在环境图像中的位置信息是(X，Y，W，H)，X、Y表示显示设备在该环境图像中坐标、W表示显示设备在该环境图像中所占像素的长度的个数，H表示显示设备在该环境图像中所占像素的宽度的个数。该位置信息是通过遍历所述环境图像中矩形框内的显示设备(即黑色区域)得到。
具体地，将所述环境图像可以通过bitmap类打开，读取指定位置(即矩形框内)的图像保存于二维数组，并从左到右和从上到下遍历像素。如果碰到RGB值小于预设值(例如35)的像素，则判断该像素的右和下的像素是否符合要求，如果符合，就切到右边的像素进行重复判断。直到发现不符合要求的点。再切到该像素的下方像素进行判断。最后使用连续点最多的区域。需要说明的是，RGB值小于35的像素表示颜色偏黑的区域，即显示设备所表示的黑色区域。
步骤114，对所述环境图像进行高通滤波处理以得到处理后的环境图像。
示例性地，对环境图像进行高通滤波(high-pass filter)处理，得到处理后的环境图像(如图4所示)。高通滤波是一种过滤方式，规则为高频信号能正常通过，而低于设定临界值的低频信号则被阻隔、减弱。
示例性地，上述步骤120中，所述从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域的步骤包括：
步骤121，计算每个方向对应的像素总数，所述像素总数表示RGB值小于预设值的像素的个数。
步骤122，根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域。
需要说明的是，本发明可对环境图像中的显示设备的至少两个方向进行图像填充，例如左方向和右方向，上方向和下方向，也可以是左方向和上方向，或者是左方向、右方向以及上方向，或者是上下左右四个方向。所以本发明对于图像填充的方向不做限定。
以下以所述至少两个方向包括上下左右四个方向为例进行说明。示例性地，上述步骤121中，所述计算每个方向对应的像素总数
的步骤包括：
步骤1211，基于所述显示设备的长度和宽度的比例配置探查矩形的数量，以得到上下方向各为第一数量的探查矩形和左右方向各为第二数量的探查矩形，所述探查矩形用于获取RGB值小于预设值的像素。
示例性地，如图5、图6所示，从环境图像中的显示设备的边缘向四周升出探查矩形，宽都为8～10个像素，左右方向的探查矩形的长度均为W/2～3W/4(例如W/2)，上方的探查矩形的高度为H/2～3H/4(例如H/2)，下方的探查矩形的高度也为H/4～3H/4(例如H/2)，但是下方的探查矩形并不是贴着显示设备的，而是要往下平移预设个(例如20个)像素，这是因为显示设备的底部会有如面板，红外头，音箱等设备，导致底部的边框要比其它三边厚。
具体地，若显示设备的长度和宽度的比例是16：9，则得到上下方向各为16个探查矩形和左右方向各为9个探查矩形(如图6所示)。
步骤1212，将所述探查矩形探查到上下左右四个方向的像素的个数进行累计保存以得到对应方向的像素总数。
示例性地，基于上下左右四个方向配置对应的变量，并将上下左右方向对应的探查矩形探查到RGB值小于预设值的像素的个数分别进行累计并保存至对应的变量，以得到对应方向的像素总数。
其中，所述上下左右对应的变量分别为top_count、botton_count、left_count以及right_count，所述RGB值小于预设值的像素表示颜色偏黑的像素，即上述变量用于保存偏黑像素总数。
并且，上述变量也用于统计探查的结果，在探查矩形内遍历时碰到的偏黑的像素(即RGB都小于35)，就将相应的值加一。
示例性地，上述步骤122中，所述根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域的步骤包括：
步骤1221，根据计算公式(top_count+botton_count)/第一数量*预设权重，得到第一平均值，并根据计算公式(left_count+right_count)/第二数量，得到第二平均值。
例如，(top_count+botton_count)/16*110％表示上下方向上的变化的第一平均值，(left_count+right_count)/9表示左右方向上的变化的第二平均值。
需要说明的是，上述预设权重(例如110％)的设置是因为：第一、显示 设备包括壁挂式的显示设备和台式的显示设备，而台式的显示设备具有支架，一般来说支架的面积会小于显示设备面积的10％；第二、显示设备底部的边框会比其它边框厚，底部的黑边，会让整体的视觉效果，有更多的不连贯感。所以基于上述目的，为了实现更好的融合效果，需要在上下方向增加预设权重。
步骤1222，比较所述第一平均值和所述第二平均值，取较小值的方向作为目标方向以进行内容获取，并将获取到的内容对所述环境图像中的显示设备进行图像填充以得到叠底区域。
需要说明的是，本发明进行内容获取的方向是上下方向或左右方向，而取值较小的方向，是因为值越小，表示该方向上的环境图像的颜色变化较小，即内容比较单一。因此，选择越少变化的图像进行填充，融合效果则更好。如果第一平均值和第二平均值相等，则选择左右方向作为目标方向。
示例性地，首先，比较所述目标方向中的两个相对方向的像素总数，将较小值的方向作为第一目标方向，将较大值的方向作为第二目标方向。
假设上述确定的所述目标方向为左右方向，则再次对比变量left_count和变量right_count的值，并假设变量left_count的值较小，则所述第一目标方向为左方向，所述第二目标方向为右方向。
然后，选择所述环境图像上显示设备左方向长度为W/2+A1(例如A1＝W/6)和宽度为H的图像进行填充并进行水平翻转以得到第一图像A(如图7、图9所示)，以及选择所述环境图像上显示设备右方向长度为W/2+B1(例如B1＝W/8)和宽度为H的图像进行填充并进行水平翻转以得到第二图像B(如图8、图9所示)。
其中，W表示显示设备在所述环境图像中所占像素的长度的个数，A1表示所占像素的长度的个数，B1表示所占像素的长度的个数，A1>B1且W/10<A1+B1<W/2。
需要说明的是，这里的A1＝W/6和B1＝W/8为优选的数值，实际上这里的选择的数字要在W/10<A1+B1<W/2，(A1，B1为左右侧超过一半的部分)，越接近W/2叠加的效会更好，但是叠加的部分变大，会改变原来图像的流畅性和美感；而越接近W/10，会导致后面的亮度过渡的太剧烈，不能得到很好的融合效果。
另外，由于上述的示例是左右方向，所以可以将填充的图像做水平翻转处理，但是本发明也可以做其它处理，例如角度旋转，垂直翻转等，所以本发明对于对填充图像进行何种处理不做限定。
最后，计算所述叠底区域的RGB值，得到包括所述第一图像和所述第二图像的第三图像(如图10所示)，所述叠底区域为所述第一图像和所述第二图像的交叉重叠部分。
示例地，所述叠底区域的RGB值的计算式为：
Result Color＝(Top Color)*(Bottom Color)/255；
其中，Result Color表示所述叠底区域的RGB值，Top Color表示A1的RGB值，Bottom Color表示B1的RGB值。
具体地，分别对像素中的R，G，B(Red，Green，Blue三种颜色的数据分别进行操作)。例如，红色公式变为：
最后像素的红色值＝(Top图像的红色*Button图像的红色)/255，将值修改到响应像素中，最后的保存为第三图像。
其它两种颜色Green和Blue与上述红色公式一样进行操作。
示例性地，上述步骤130中，对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像的步骤包括：
步骤131，获取第一平均亮度值和第二平均亮度值，所述第一平均亮度值为所述第三图像中第W/2-B1-1列的平均亮度值，所述第二平均亮度为所述第三图像中第W/2+A1+1列的平均亮度值。
由于填充图像后导致叠底区域的图像变暗(如图10所示)，所以海需要对叠底区域进行亮度调整。
例如，取第三图像中(W/2-W/8-1)列的第一平均亮度值LA，取第三图像中(W/2+W/6+1)的第二平均亮度值LB，这两个LA和LB就是黑色部分边缘的平均亮度值。
步骤132，对所述叠加区域从所述第一平均亮度值渐变至所述第二平均亮度值以对所述叠底区域进行亮度调整以得到所述目标图像；
示例性地，对上述亮度，进行渐变，通过将平均亮度值大的减去平均亮度值小的，再除以像素，得到递减的值。
具体地，假设第一平均亮度值LA大于第二平均亮度值LB，则计算渐变的递减值，所述递减值△Light的计算式为：
△Light＝(LA-LB)/(A1+B1)；
因此，通过叠底区域(长度为A1+B1且高度为H，H表示显示设备在所述环境图像中所占像素的宽度的个数)左边的第一列亮度为LA，逐渐递减，一直到LB，经亮度调整后的目标图像如图11所示。
示例性地，所述叠底区域的RGB值的调整公式为：
Result Color＝((Top Color)*(Bottom Color)+△Light)/255。
例如，用红色进行说明：
Red＝(叠底区域中Top图像的pixel中的Red值*叠底区域中Botton图像的pixel中的Red值+该列的亮度递减值)/255；
亮度公式Float Light＝(Red*0.2126f+Green*0.7152f+Blue*0.0722f)/255，上述数值可以选择保留小数点后四位。
在本发明的一实施例中，在执行上述步骤130之后，所述方法海包括：
步骤140，若通过所述显示设备获取所述环境图像，则将所述目标图像直接显示在所述显示设备上；或者
步骤150，若通过移动终端获取所述环境图像，则将所述目标图像发送至所述显示设备以在所述显示设备上进行显示。
例如，将移动终端生成的目标图像推送至显示设备并进行显示，如果上述图像处理的算法设置在显示设备上，则将拍照后的环境图像传送至显示设备端进行处理，处理后将生成的目标图像进行显示。
由此可知，本发明是对显示设备的周围环境在不同方向上的适应和延伸，根据显示设备的周围颜色变化的剧烈程度，选择获得填充内容的方向，并且对填充内容进行深度融合处理，以生成融合图像，让显示设备和周围环境融为一体，提升用户的居住视觉体验和泛智屏使用体验。
下面对本发明提供的图像融合装置进行描述，下文描述的图像融合装置与上文描述的图像融合方法可相互对应参照。
请参考图12，图12是本发明提供的图像融合装置的结构示意图。一种图像融合装置1200，包括获取模块1210、叠底处理模块1220以及融合模块1230。
示例性地，所述获取模块1210，用于获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像。
示例性地，所述叠底处理模块1220，用于从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度不同于非叠底区域的亮度。
示例性地，所述融合模块1230，用于对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
示例性地，所述叠底处理模块1220还用于：
计算每个方向对应的像素总数，所述像素总数表示RGB值小于预设值的像素的个数；
根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域。
示例性地，所述叠底处理模块1220还用于：
基于所述显示设备的长度和宽度的比例配置探查矩形的数量，以得到上下方向各为第一数量的探查矩形和左右方向各为第二数量的探查矩形，所述探查矩形用于获取RGB值小于预设值的像素；
将所述探查矩形探查到上下左右四个方向的像素的个数进行累计保存以得到对应方向的像素总数。
示例性地，所述叠底处理模块1220还用于：
若所述显示设备的长度和宽度的比例是16：9，则得到上下方向各为16个探查矩形和左右方向各为9个探查矩形；
其中，每个探查矩形的宽度为8～10个像素，左右方向的探查矩形的长度均为W/2～3W/4，上下方向的探查矩形的高度均为H/2～3H/4，W表示显示设备在所述环境图像中所占像素的长度的个数，H表示显示设备在所述环境图像中所占像素的宽度的个数。
示例性地，所述叠底处理模块1220还用于：
基于上下左右四个方向配置对应的变量；
将上下左右方向对应的探查矩形探查到RGB值小于预设值的像素的个数分别进行累计并保存至对应的变量，以得到对应方向的像素总数；
其中，所述上下左右对应的变量分别为top_count、botton_count、left_count以及right_count，所述RGB值小于预设值的像素表示颜色偏黑的像素。
示例性地，所述叠底处理模块1220还用于：
根据计算公式(top_count+botton_count)/第一数量*预设权重，得到第一平均值，并根据计算公式(left_count+right_count)/第二数量，得到第二平均值；
比较所述第一平均值和所述第二平均值，取较小值的方向作为目标方向以进行内容获取，并将获取到的内容对所述环境图像中的显示设备进行图像填充以得到叠底区域。
示例性地，所述叠底处理模块1220还用于：
比较所述目标方向中的两个相对方向的像素总数，将较小值的方向作为第一目标方向，将较大值的方向作为第二目标方向；
假设所述第一目标方向为左方向，所述第二目标方向为右方向，选择所述环境图像上显示设备左方向长度为W/2+A1和宽度为H的图像进行填充并进行水平翻转以得到第一图像，以及选择所述环境图像上显示设备右方向长度为W/2+B1和宽度为H的图像进行填充并进行水平翻转以得到第二图像；
计算所述叠底区域的RGB值，得到包括所述第一图像和所述第二图像的第三图像，所述叠底区域为所述第一图像和所述第二图像的交叉重叠部分；
其中，W表示显示设备在所述环境图像中所占像素的长度的个数，A1表示所占像素的长度的个数，B1表示所占像素的长度的个数，A1>B1且W/10<A1+B1<W/2。
示例性地，所述叠底区域的RGB值的计算式为：
Result Color＝(Top Color)*(Bottom Color)/255；
其中，Result Color表示所述叠底区域的RGB值，Top Color表示A1的RGB值，Bottom Color表示B1的RGB值。
示例性地，所述融合模块1230还用于：
获取第一平均亮度值和第二平均亮度值，所述第一平均亮度值为所述第三图像中第W/2-B1-1列的平均亮度值，所述第二平均亮度为所述第三图像中第W/2+A1+1列的平均亮度值；
对所述叠加区域从所述第一平均亮度值渐变至所述第二平均亮度值以对所述叠底区域进行亮度调整以得到所述目标图像；
其中，所述叠加区域的长度为A1+B1且高度为H，H表示显示设备在所述环境图像中所占像素的宽度的个数。
示例性地，所述融合模块1230还用于：
假设所述第一平均亮度值大于所述第二平均亮度值，则计算渐变的递减值，所述递减值的计算式为：
△Light＝(LA-LB)/(A1+B1)；
其中，△Light表示递减值，LA表示第一平均亮度值，LB表示第二平均亮度值。
示例性地，所述叠底区域的RGB值的调整公式为：
Result Color＝((Top Color)*(Bottom Color)+△Light)/255。
示例性地，所述获取模块1210还用于：
调用所述显示设备或移动终端的拍摄界面，所述拍摄界面具有一矩形框；
将所述显示设备的图像置于所述矩形框内以获取到所述环境图像。
示例性地，所述图像融合装置1200还包括预处理模块，所述预处理模块用于：
确定所述显示设备在所述环境图像中的位置并保存该位置信息；
对所述环境图像进行高通滤波处理以得到处理后的环境图像；
其中，所述位置信息包括所述显示设备在所述环境图像中的X、Y坐标、表示显示设备在该环境图像中所占像素的长度的个数W以及表示显示设备在该环境图像中所占像素的宽度的个数H。
示例性地，所述图像融合装置1200还包括显示模块，所述显示模块用于：
若通过所述显示设备获取所述环境图像，则将所述目标图像直接显示在所述显示设备上；或者
若通过移动终端获取所述环境图像，则将所述目标图像发送至所述显示设备以在所述显示设备上进行显示。
请参考图13，图13是本发明实施例提供的图像融合装置的结构示意图。所述图像融合装置应用于移动终端，将生成的目标图像发送至显示设备端，通 过显示设备端的接收模块接收所述目标图像，并通过显示模块对接收的目标图像进行显示。
在此需要说明的是，本发明实施例提供的上述图像融合装置，能够实现上述方法实施例所实现的所有方法步骤，且能够达到相同的技术效果，在此不再对本实施例中与方法实施例相同的部分及有益效果进行具体赘述。
图14示例了一种电子设备的实体结构示意图，如图14所示，该电子设备可以包括：处理器(Processor)1410、通信接口(Communications Interface)1420、存储器(Memory)1430和通信总线1440，其中，处理器1410，通信接口1420，存储器1430通过通信总线1440完成相互间的通信。处理器1410可以调用存储器1430中的逻辑指令，以执行所述图像融合方法，所述方法包括：
获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像；
从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度低于非叠底区域的亮度；
对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
此外，上述的存储器1430中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
另一方面，本发明还提供一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，计算机能够执行上述各方法所提供的 所述图像融合方法。
又一方面，本发明还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以执行上述各提供的所述图像融合方法。
本发明实施例提供的一种电子设备、一种计算机程序产品、一种处理器可读存储介质，其上存储的计算机程序使处理器能够实现上述方法实施例所实现的所有方法步骤，且能够达到相同的技术效果，在此不再对本实施例中与方法实施例相同的部分及有益效果进行具体赘述。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (17)

  1. 一种图像融合方法，所述方法包括：
    获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像；
    从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度低于非叠底区域的亮度；
    对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
  2. 根据权利要求1所述的图像融合方法，其中所述从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域的步骤包括：
    计算每个方向对应的像素总数，所述像素总数表示RGB值小于预设值的像素的个数；
    根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域。
  3. 根据权利要求2所述的图像融合方法，其中所述至少两个方向包括上下左右四个方向，所述计算每个方向对应的像素总数的步骤包括：
    基于所述显示设备的长度和宽度的比例配置探查矩形的数量，以得到上下方向各为第一数量的探查矩形和左右方向各为第二数量的探查矩形，所述探查矩形用于获取RGB值小于预设值的像素；
    将所述探查矩形探查到上下左右四个方向的像素的个数进行累计保存以得到对应方向的像素总数。
  4. 根据权利要求3所述的图像融合方法，其中所述基于所述显示设备的长度和宽度的比例配置探查矩形的数量的步骤包括：
    若所述显示设备的长度和宽度的比例是16：9，则得到上下方向各为16个探查矩形和左右方向各为9个探查矩形；
    其中，每个探查矩形的宽度为8～10个像素，左右方向的探查矩形的长度均为W/2～3W/4，上下方向的探查矩形的高度均为H/2～3H/4，W表示显示设备在所述环境图像中所占像素的长度的个数，H表示显示设备在所述环境图像 中所占像素的宽度的个数。
  5. 根据权利要求3所述的图像融合方法，其中所述将所述探查矩形探查到上下左右四个方向的像素的个数进行累计保存以得到对应方向的像素总数的步骤包括：
    基于上下左右四个方向配置对应的变量；
    将上下左右方向对应的探查矩形探查到RGB值小于预设值的像素的个数分别进行累计并保存至对应的变量，以得到对应方向的像素总数；
    其中，所述上下左右对应的变量分别为top_count、botton_count、left_count以及right_count，所述RGB值小于预设值的像素表示颜色偏黑的像素。
  6. 根据权利要求5所述的图像融合方法，其中所述根据所述像素总数确定目标方向，并对所述目标方向进行图像填充以得到叠底区域的步骤包括：
    根据计算公式(top_count+botton_count)/第一数量*预设权重，得到第一平均值，并根据计算公式(left_count+right_count)/第二数量，得到第二平均值；
    比较所述第一平均值和所述第二平均值，取较小值的方向作为目标方向以进行内容获取，并将获取到的内容对所述环境图像中的显示设备进行图像填充以得到叠底区域。
  7. 根据权利要求6所述的图像融合方法，其中所述比较所述第一平均值和所述第二平均值，取较小值的方向作为目标方向以进行内容获取，并将获取到的内容对所述环境图像中的显示设备进行图像填充以得到叠底区域的步骤还包括：
    比较所述目标方向中的两个相对方向的像素总数，将较小值的方向作为第一目标方向，将较大值的方向作为第二目标方向；
    假设所述第一目标方向为左方向，所述第二目标方向为右方向，选择所述环境图像上显示设备左方向长度为W/2+A1和宽度为H的图像进行填充并进行水平翻转以得到第一图像，以及选择所述环境图像上显示设备右方向长度为W/2+B1和宽度为H的图像进行填充并进行水平翻转以得到第二图像；
    计算所述叠底区域的RGB值，得到包括所述第一图像和所述第二图像的第三图像，所述叠底区域为所述第一图像和所述第二图像的交叉重叠部分；
    其中，W表示显示设备在所述环境图像中所占像素的长度的个数，A1表示所占像素的长度的个数，B1表示所占像素的长度的个数，A1>B1且W/10<A1+B1<W/2。
  8. 根据权利要求7所述的图像融合方法，其中所述叠底区域的RGB值的计算式为：
    Result Color＝(Top Color)*(Bottom Color)/255；
    其中，Result Color表示所述叠底区域的RGB值，Top Color表示A1的RGB值，Bottom Color表示B1的RGB值。
  9. 根据权利要求8所述的图像融合方法，其中所述对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像的步骤包括：
    获取第一平均亮度值和第二平均亮度值，所述第一平均亮度值为所述第三图像中第W/2-B1-1列的平均亮度值，所述第二平均亮度为所述第三图像中第W/2+A1+1列的平均亮度值；
    对所述叠加区域从所述第一平均亮度值渐变至所述第二平均亮度值以对所述叠底区域进行亮度调整以得到所述目标图像；
    其中，所述叠加区域的长度为A1+B1且高度为H，H表示显示设备在所述环境图像中所占像素的宽度的个数。
  10. 根据权利要求9所述的图像融合方法，其中所述对所述叠加区域从所述第一平均亮度值渐变至所述第二平均亮度值以对所述叠底区域进行亮度调整以得到所述目标图像的步骤包括：
    假设所述第一平均亮度值大于所述第二平均亮度值，则计算渐变的递减值，所述递减值的计算式为：
    △Light＝(LA-LB)/(A1+B1)；
    其中，△Light表示递减值，LA表示第一平均亮度值，LB表示第二平均亮度值。
  11. 根据权利要求10所述的图像融合方法，其中所述叠底区域的RGB值的调整公式为：
    Result Color＝((Top Color)*(Bottom Color)+△Light)/255。
  12. 根据权利要求1所述的图像融合方法，所述获取一环境图像的步骤包 括：
    调用所述显示设备或移动终端的拍摄界面，所述拍摄界面具有一矩形框；
    将所述显示设备的图像置于所述矩形框内以获取到所述环境图像。
  13. 根据权利要求1所述的图像融合方法，其中在所述对所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域步骤之前，所述方法还包括：
    确定所述显示设备在所述环境图像中的位置并保存该位置信息；
    对所述环境图像进行高通滤波处理以得到处理后的环境图像；
    其中，所述位置信息包括所述显示设备在所述环境图像中的X、Y坐标、表示显示设备在该环境图像中所占像素的长度的个数W以及表示显示设备在该环境图像中所占像素的宽度的个数H。
  14. 根据权利要求1所述的图像融合方法，其中在所述对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像步骤之后，所述方法还包括：
    若通过所述显示设备获取所述环境图像，则将所述目标图像直接显示在所述显示设备上；或者
    若通过移动终端获取所述环境图像，则将所述目标图像发送至所述显示设备以在所述显示设备上进行显示。
  15. 一种图像融合装置，所述装置包括：
    获取模块，用于获取一环境图像，所述环境图像为表示显示设备和该显示设备的周围环境的图像；
    叠底处理模块，用于从所述环境图像中的显示设备的至少两个方向进行图像填充以得到叠底区域，所述叠底区域的亮度不同于非叠底区域的亮度；
    融合模块，用于对所述叠底区域进行亮度调整以生成与该显示设备的周围环境相融合的目标图像。
  16. 一种电子设备，其包括存储器、处理器及存储在所述存储器上并在该处理器上运行的计算机程序，所述处理器执行所述程序时实现如权利要求1至14任一项所述的图像融合方法的步骤。
  17. 一种非暂态计算机可读存储介质，其上存储有计算机程序，所述计算 机程序被处理器执行时实现如权利要求1至14任一项所述的图像融合方法的步骤。
PCT/CN2022/129361 2022-09-22 2022-11-02 图像融合方法、装置、电子设备及存储介质 Ceased WO2024060360A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211160692.7A CN117152031B (zh) 2022-09-22 2022-09-22 图像融合方法、装置、电子设备及存储介质
CN202211160692.7 2022-09-22

Publications (1)

Publication Number Publication Date
WO2024060360A1 true WO2024060360A1 (zh) 2024-03-28

Family

ID=88906844

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/129361 Ceased WO2024060360A1 (zh) 2022-09-22 2022-11-02 图像融合方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN117152031B (zh)
WO (1) WO2024060360A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140168465A1 (en) * 2012-12-18 2014-06-19 Canon Kabushiki Kaisha Image processing apparatus and method for controlling the same
CN107071169A (zh) * 2017-03-31 2017-08-18 努比亚技术有限公司 屏幕壁纸的处理装置及方法
CN112712485A (zh) * 2019-10-24 2021-04-27 杭州海康威视数字技术股份有限公司 一种图像融合方法及装置
CN113645494A (zh) * 2021-08-10 2021-11-12 海信视像科技股份有限公司 一种屏幕融合方法、显示设备、终端设备及服务器

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107241555A (zh) * 2017-07-11 2017-10-10 深圳Tcl数字技术有限公司 合成图像的亮度调整方法、装置、电视及存储介质
JP7162124B2 (ja) * 2019-03-27 2022-10-27 富士フイルム株式会社 画像処理装置、投影システム、画像処理方法、及び画像処理プログラム
CN111862271B (zh) * 2020-07-24 2024-04-26 广州市百果园信息技术有限公司 显示区域填充方法、装置、设备及存储介质
CN111984164B (zh) * 2020-08-31 2023-05-02 Oppo广东移动通信有限公司 壁纸生成方法、装置、终端及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140168465A1 (en) * 2012-12-18 2014-06-19 Canon Kabushiki Kaisha Image processing apparatus and method for controlling the same
CN107071169A (zh) * 2017-03-31 2017-08-18 努比亚技术有限公司 屏幕壁纸的处理装置及方法
CN112712485A (zh) * 2019-10-24 2021-04-27 杭州海康威视数字技术股份有限公司 一种图像融合方法及装置
CN113645494A (zh) * 2021-08-10 2021-11-12 海信视像科技股份有限公司 一种屏幕融合方法、显示设备、终端设备及服务器

Also Published As

Publication number Publication date
CN117152031B (zh) 2026-03-20
CN117152031A (zh) 2023-12-01

Similar Documents

Publication Publication Date Title
US8538147B2 (en) Methods and appartuses for restoring color and enhancing electronic images
US8594451B2 (en) Edge mapping incorporating panchromatic pixels
CN104067311B (zh) 数字化妆
US7844127B2 (en) Edge mapping using panchromatic pixels
JP6786850B2 (ja) 画像処理装置、画像処理方法、画像処理システムおよびプログラム
US7822272B2 (en) Adaptive histogram equalization for images with strong local contrast
CN112950499B (zh) 图像处理方法、装置、电子设备及存储介质
US10565742B1 (en) Image processing method and apparatus
US8891867B2 (en) Image processing method
US10235741B2 (en) Image correction apparatus and image correction method
US11194536B2 (en) Image processing method and apparatus for displaying an image between two display screens
US20130064445A1 (en) Adaptive Dithering During Image Processing
JP6135952B2 (ja) 画像アンチエイリアシング方法および装置
JP2005122361A (ja) 画像処理装置及び方法、コンピュータプログラム、記録媒体
CN106815827A (zh) 基于Bayer格式的图像融合方法及图像融合装置
JP2006191607A (ja) デジタル撮影装置の色補間法
CN114495824B (zh) 一种针对oled显示器图像及文字边缘偏色的校正方法、系统、存储介质和处理器
TW202021330A (zh) 圖像縮放方法和裝置
US9928577B2 (en) Image correction apparatus and image correction method
WO2024060360A1 (zh) 图像融合方法、装置、电子设备及存储介质
JP2009010636A (ja) 適応ヒストグラム等化方法及び適応ヒストグラム等化装置
CN108735180B (zh) 一种影像处理方法及装置
CN110874816B (zh) 一种图像处理方法、装置、移动终端及存储介质
CN116263947A (zh) 图像融合方法、计算机程序产品、存储介质及电子设备
CN108257221B (zh) 色调调整方法、装置及存储介质、电子装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22959362

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22959362

Country of ref document: EP

Kind code of ref document: A1