WO2022021991A1 - 图像压缩方法及装置、图像显示方法及装置和介质 - Google Patents

图像压缩方法及装置、图像显示方法及装置和介质 Download PDF

Info

Publication number
WO2022021991A1
WO2022021991A1 PCT/CN2021/092369 CN2021092369W WO2022021991A1 WO 2022021991 A1 WO2022021991 A1 WO 2022021991A1 CN 2021092369 W CN2021092369 W CN 2021092369W WO 2022021991 A1 WO2022021991 A1 WO 2022021991A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
area
display
compressed
gaze
Prior art date
Application number
PCT/CN2021/092369
Other languages
English (en)
French (fr)
Inventor
姬治华
史天阔
张小牤
龚敬文
刘蕊
侯一凡
楚明磊
习艳会
孙炎
赵晨曦
彭项君
张硕
Original Assignee
京东方科技集团股份有限公司
北京京东方光电科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司, 北京京东方光电科技有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US17/764,359 priority Critical patent/US11917167B2/en
Publication of WO2022021991A1 publication Critical patent/WO2022021991A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation

Definitions

  • the present disclosure relates to the field of image and video technology, and in particular, to an image compression method, an image display method, an image compression device, an image display device, and a computer-readable medium.
  • VR Virtual Reality
  • AR Augmented Reality
  • the existing virtual reality system mainly simulates a virtual three-dimensional world through a high-performance computing system with a central processing unit, and provides users with visual, auditory and other sensory experiences, so as to make users feel like they are on the scene, and at the same time. Human-machine interaction is also possible.
  • the present disclosure aims to solve at least one of the technical problems existing in the prior art, and proposes an image compression method, an image display method, an image compression apparatus, an image display apparatus and a computer-readable medium.
  • an embodiment of the present disclosure provides an image compression method, including:
  • the non-gazing region is compressed, and a compressed image is generated according to the gazing region and the compressed non-gazing region.
  • the step of acquiring the gaze point of the human eye on the original image includes:
  • a user-side image including the user's eyes is acquired, and the state of the human eye is detected according to the user-side image, so as to determine the gaze point of the human eye.
  • the step of determining the non-gazing area of the image according to the gaze point of the human eye includes:
  • the remainder of the image is determined to be the non-gazing region.
  • the original image is rectangular
  • the step of determining that the gaze point of the human eye is the center of symmetry, and the area within the predetermined shape with the predetermined size is the gaze area includes:
  • the gaze area is determined according to the preset length and width with the gaze point of the human eye as the center of symmetry.
  • the step of compressing the non-gazing area includes:
  • the rearranged region is compressed.
  • the step of compressing the rearrangement region includes:
  • the image block is a rectangle, two of its four sides are parallel to the horizontal direction, and the other two sides are parallel to the vertical direction, the first direction is one of the horizontal direction and the vertical direction, the first The second direction is the other of the horizontal direction and the vertical direction; the first sampling multiple is greater than the second sampling multiple.
  • the original image, the gaze area, the image block, the rearranged area, and the compressed image are all rectangular.
  • the method before the step of sampling the chrominance channel of the rearrangement area in the first direction according to the preset first sampling multiple, the method further includes:
  • the color space of the rearrangement area is not the YUV space, convert the color space of the rearrangement area to the YUV space.
  • an image display method including:
  • the restored original image is displayed.
  • the own application processor transmits the compressed image to the display controller.
  • the step of decompressing the compressed non-gazing area includes:
  • the compressed non-fixation region is decompressed by nearest neighbor difference or bilinear difference.
  • the compressed non-gazing region is divided into a plurality of non-overlapping image blocks by dividing the non-gazing region, rearranging each of the image blocks, generating a rearranged region, and performing the rearrangement on the rearranged region. It is obtained by compressing the row area;
  • the method further includes:
  • the compressed rearranged region is reorganized to restore each of the image blocks.
  • the gaze area is a rectangle, two of its four sides are parallel to the horizontal direction, and the other two sides are parallel to the vertical direction, and the image block is also a rectangle; the method further includes:
  • the gaze area is rendered according to the display resolution, down-sampled in the horizontal direction according to the ratio of the image resolution and the display resolution, and displayed in a single-line co-opening manner in the vertical direction ; Render each of the image blocks in the same column as the gaze area according to the display resolution, downsample them in the horizontal direction according to the ratio of the image resolution and the display resolution, and vertically Display is performed in the same opening mode of multiple lines; for each of the image blocks in the same line with the gaze area, it is upsampled in the horizontal direction according to the ratio of the image resolution and the display resolution, and in the vertical direction It is displayed in the same opening mode of a single line in the direction; for the remaining image blocks, it is up-sampled in the horizontal direction according to the ratio of the image resolution and the display resolution, and in the vertical direction, it is displayed through multiple lines with the same opening. open to display.
  • an image compression apparatus including:
  • processors one or more processors
  • a storage unit for storing one or more programs
  • the one or more processors implement the image compression method as described in any one of the above embodiments.
  • an image display device including:
  • processors one or more processors
  • a storage unit for storing one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image display method according to any one of the above embodiments, so that the display unit displays The restored original image.
  • the image display device is a virtual reality device or an augmented reality device.
  • an embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored, wherein, when the program is executed by a processor, the image compression method according to any one of the foregoing embodiments is implemented. steps, or when the program is executed by the processor, the steps in the image display method described in any of the foregoing embodiments are implemented.
  • FIG. 1 is a flowchart of an image compression method according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of a specific implementation method of step S1 in an embodiment of the disclosure
  • step S1 is a flowchart of another specific implementation method of step S1 in an embodiment of the disclosure.
  • FIG. 4 is a flowchart of a specific implementation method of step S102 in an embodiment of the disclosure.
  • step S2 is a flowchart of a specific implementation method of step S2 in an embodiment of the disclosure.
  • 5a is a schematic diagram of an image processing process of another image compression method provided by an embodiment of the present disclosure.
  • FIG. 6 is a flowchart of a specific implementation method of step S202 in an embodiment of the disclosure.
  • FIG. 7 is a flowchart of an image display method according to an embodiment of the present disclosure.
  • FIG. 8 is a flowchart of another image display method provided by an embodiment of the present disclosure.
  • FIG. 9 is a flowchart of still another image display method provided by an embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of an image processing process of still another image display method provided by an embodiment of the present disclosure.
  • the image compression method, image display method, image compression device, image display device and computer-readable medium provided by the present disclosure can be used to obtain the gaze point of the human eye and divide the gaze area and the non-gaze area of the image.
  • the area and the non-gazing area are processed separately to achieve high-definition rendering of the gazing area, and low-definition rendering and compression of the non-gazing area.
  • the display technology corresponds to a large amount of stored data and a large transmission bandwidth.
  • FIG. 1 is a flowchart of an image compression method provided by an embodiment of the present disclosure. As shown in Figure 1, the method includes:
  • step S1 the gaze point of the human eye on the original image is acquired, and the gaze area and the non-gaze area of the original image are determined according to the gaze point of the human eye.
  • the original image is an image to be displayed that is rendered by the corresponding image compression apparatus according to the current state of the user-side device.
  • the content to be displayed is determined according to the state of a virtual reality or augmented reality head-mounted device (such as VR glasses or AR glasses), and then the original image is rendered by a graphics processing unit (GPU) of a host computer (such as a PC).
  • a graphics processing unit GPU
  • the original image needs to be sent to the display controller of the host or the head mounted device for actual display.
  • the gaze point of the human eye is determined by real-time detection or by setting a fixed point, and the gaze point of the human eye is the position in the original image that the user's human eye is currently directly looking at.
  • the fixation area refers to the area in the original image that is closer to the gaze point of the human eye, that is, the area that the user is currently focusing on; Areas that will be seen but not paid attention to.
  • FIG. 2 is a flowchart of a specific implementation method of step S1 in an embodiment of the present disclosure. Specifically, as shown in FIG. 2 , in step S1 , the step of acquiring the gaze point of the human eye on the original image includes: step S101 .
  • Step S101 acquiring a user-side image including the user's eyes, and detecting the state of the human eye according to the user-side image, so as to determine the gaze point of the human eye.
  • the user-side image including the user's eyes is acquired through the built-in image and video acquisition device, or the user-side image transmitted by the client is received.
  • the user-side image is an eye image or an eyeball image of the user; the state of the human eye is detected according to the user-side image, that is, the user's pupil image is extracted according to the user's eye image or eyeball image, and established based on the user's pupil position Coordinate system, and map the coordinates of the pupil center to the original image, or perform coordinate conversion according to the coordinate system established in the original image, thereby taking the mapping point or the point position corresponding to the conversion result as the human eye gaze point.
  • the user-side image may also include eye movement information, and the gaze point of the human eye is determined according to the eye movement information.
  • Step S2 compress the non-gazing region, and generate a compressed image according to the gazing region and the compressed non-gazing region.
  • step S2 the fixation area and the non-attention area of the original image are separately processed to achieve high-definition rendering of the fixation area, and low-definition rendering and compression of the non-attention area.
  • the method further includes: storing the compressed image for subsequent external sending or internal transmission of the compressed image.
  • the embodiment of the present disclosure provides an image compression method, which can be used to obtain the gaze point of the human eye and divide the gaze area and the non-gazing area of the image.
  • FIG. 3 is a flowchart of another specific implementation method of step S1 in an embodiment of the disclosure. Specifically, as shown in FIG. 3 , in step S1 , the step of determining the non-gazing area of the image according to the gaze point of the human eye includes: step S102 and step S103 .
  • Step S102 determining that the gaze point of the human eye is the center of symmetry, and the area located in the predetermined shape with the predetermined size is the gaze area.
  • the gaze area may be a circle, a rectangle or other graphics, and from the perspective of compression coding, preferably, the gaze area is a rectangle.
  • the method further includes: determining an area where the number of pixels satisfies a preset number condition, taking the gaze point of the human eye as the center of symmetry, as the gaze area.
  • FIG. 4 is a flowchart of a specific implementation method of step S102 in an embodiment of the present disclosure.
  • the original image is a rectangle
  • the gaze area is also a rectangle; as shown in FIG. 4 , in step S102, the step of determining the gaze area with the gaze point of the human eye as the center of symmetry and the area located in the predetermined shape with a predetermined size is the gaze area, including : Step S1021.
  • Step S1021 taking the gazing point of the human eye as the center of symmetry, and determining the gazing area according to the preset length and width.
  • the length and width of the gaze area can be adaptively set according to parameters such as brightness, contrast, gray distribution, color distribution, and number of pixels of the original image, and the length and width of the gaze area are smaller than any of the length and width of the original image.
  • the sides of the rectangle of the gaze area are respectively parallel to the corresponding sides of the original image (or the gaze area is a "small rectangle" in the rectangle of the original image).
  • Step S103 determining that the remaining part of the image is a non-gazing area.
  • An embodiment of the present disclosure provides an image compression method, which can be used to determine a gaze area and a non-gaze area by using the detected gaze point of the human eye, so as to realize the division of the original image.
  • FIG. 5 is a flowchart of a specific implementation method of step S2 in an embodiment of the disclosure. Specifically, as shown in FIG. 5 , step S2, the step of compressing the non-gazing area, includes steps S201 and S202.
  • Step S201 Divide the non-gazing area into a plurality of non-overlapping image blocks, and rearrange the image blocks to generate a rearrangement area.
  • the non-fixation area can be segmented by any segmentation method according to the parameters such as brightness, contrast, gray distribution, color distribution and pixel number of the non-fixation area;
  • the rearrangement area generated by rearranging the segmented image blocks can be a regular images or irregular shapes.
  • the original image, gaze area, image block, rearrangement area, and compressed image are all rectangular. Among them, from the perspective of compression coding, this setting has higher coding efficiency and lower coding complexity.
  • FIG. 5a is a schematic diagram of an image processing process of another image compression method provided by an embodiment of the present disclosure.
  • B is the gaze area
  • its width (the size in the horizontal direction in the figure, represented by the number of pixels) is 1080, which is half of the original image width of 2160.
  • the image block division of the non-gazing area is along B
  • the edge of the area and the extension line are drawn to obtain image blocks A1, A2, C1, C2, F1, F2, F3 and F4; among them, the widths of A1 and A2 must be the same as B; and since the width of B is the original half of the image, so the width sum of C1 and C2, the width sum of F1 and F2, and the width sum of F3 and F4 must also be equal to the width of B, so these image blocks can be rearranged into the shape of the rearranged image in the figure , and then compress the rearranged rearranged region, and rotate according to the optimized arrangement to obtain the D region, so that the original image, the gaze region, the image block, the rearranged region and the compressed image are all rectangles.
  • Step S202 compressing the rearrangement area.
  • FIG. 6 is a flowchart of a specific implementation method of step S202 in an embodiment of the present disclosure. Specifically, as shown in FIG. 6 , in step S202, the step of compressing the rearrangement area includes: step S2021 and step S2022.
  • Step S2021 down-sampling the chrominance channel of the rearrangement area in the first direction according to the preset first sampling multiple.
  • Step S2022 down-sampling the luminance channel and the chrominance channel of the rearrangement area in the second direction according to the preset second sampling multiple.
  • the image block segmented in step S201 is a rectangle, two of its four sides are parallel to the horizontal direction, the other two sides are parallel to the vertical direction, and the first direction is one of the horizontal direction and the vertical direction, The second direction is the other of the horizontal direction and the vertical direction; the first sampling multiple is greater than the second sampling multiple, and similarly, each sampling multiple can be based on the brightness, contrast, gray distribution, color distribution and number of pixels of the non-gazing area and other parameters for adaptive settings.
  • downsampling is the process of reducing the rearrangement area; since the luminance channel, that is, the Y channel, contains more image detail information, while the chrominance channel, that is, the U channel and V channel, contains less image detail information.
  • the luminance channel and the chrominance channel are sampled with different intervals and sampling multiples, which has achieved the purpose of taking into account the image compression ratio and peak signal-to-noise ratio of the non-gazing area.
  • step S2021 before the step of sampling the chrominance channel of the rearranged region in the first direction according to the preset first sampling multiple, the method further includes: if the color space of the rearrangement region is not YUV space, convert the color space of the rearrangement area to YUV space.
  • steps S2011 and S2022 to realize the compression of the rearrangement area is only an optional implementation manner in the present disclosure, which will not limit the technical solutions of the present disclosure, and in other different directions and Compression coding performed in the color space is also applicable to the technical solutions of the present disclosure.
  • the embodiment of the present disclosure provides an image compression method, which can be used for image rearrangement for irregular non-attention areas, so as to facilitate subsequent compression coding, improve coding efficiency, and reduce complexity.
  • FIG. 7 is a flowchart of an image display method according to an embodiment of the present disclosure. As shown in Figure 7, the method includes:
  • Step S3 Generate a compressed image by using any image compression method in the above-mentioned embodiment.
  • the generated compressed image includes a gaze region and a compressed non-gazing region.
  • the gaze region is losslessly compressed.
  • Step S4 Decompress the compressed non-gazing area to restore the restored original image.
  • the display controller can restore the original image according to the decompression method corresponding to the compression method, generate the restored original image, and actually display it.
  • the generated restored original image includes the original fixation area and the non-fixation area that loses some image details and color information (that is, the resolution of the restored original image is the same as the original image, but the actual information carried by the non-fixed area is less than the original image. ).
  • the method further includes: performing image enhancement processing on the position corresponding to the non-gazing area in the restored original image
  • the method further includes: transmitting the compressed image to the display controller by its own application processor.
  • the application processor such as a graphics processor, renders the picture that needs to be displayed at present to obtain the original image, and after generating the compressed image by any image compression method in the above-mentioned embodiment, spontaneously or based on the corresponding instruction, the The compressed image is transmitted to the display controller for decompression and actual display by the display controller.
  • the functional unit that performs graphics rendering is not integrated with the unit that performs display.
  • the graphics processor of the host is responsible for graphics rendering (generating original images), while the headset is responsible for displaying. display controller for the device. Therefore, the image needs to be sent (eg from the host to the headset).
  • the original image is compressed before sending in this embodiment, and the compressed image with a small amount of data is actually sent, the amount of data transmitted internally and entered into the buffer is reduced, and the system load is reduced.
  • step S4 the step of decompressing the compressed non-gazing region includes: decompressing the compressed non-gazing region by using the nearest neighbor difference or bilinear difference.
  • Step S5 displaying the restored original image.
  • the embodiment of the present disclosure provides an image display method, which can be used to separately process the fixation area and the non-fixation area of the original image, and perform low-definition rendering of the non-fixation area, without affecting the look and feel. , retain the necessary image quality, reduce the amount of unnecessary data, and then solve the problem of a large amount of stored data and a large transmission bandwidth corresponding to the virtual display technology.
  • FIG. 8 is a flowchart of another image display method according to an embodiment of the present disclosure.
  • the method is an optional embodiment based on the method shown in FIG. 7 .
  • the compressed non-fixation area is obtained by dividing the non-fixation area into a plurality of non-overlapping image blocks, rearranging each image block, generating a rearrangement area, and compressing the rearrangement area; specifically, The method not only includes steps S3 to S5, but also includes step S401 after the step of decompressing the compressed non-gazing area in step S4. Only step S401 will be described in detail below.
  • Step S401 Reorganize the compressed rearrangement area to restore each image block.
  • each image block is restored according to a reorganization method corresponding to the rearrangement method, so as to generate a restored original image.
  • the restored image blocks lose some image details and color information, but the size of the blocks remains unchanged.
  • FIG. 9 is a flowchart of still another image display method provided by an embodiment of the present disclosure.
  • the method is an optional embodiment based on the method shown in FIG. 8 .
  • the gaze area is a rectangle, two of its four sides are parallel to the horizontal direction, the other two sides are parallel to the vertical direction, and the image block is also a rectangle; specifically, the method not only includes steps S3 to S5, but in step S401 , after the step of reorganizing the compressed rearrangement region, step S402 is further included. Only step S402 will be described in detail below.
  • Step S402 If the display resolution corresponding to the display unit is greater than the image resolution of the restored original image, adjust the restored original image and display it.
  • the gaze area is rendered according to the display resolution, and it is down-sampled in the horizontal direction according to the ratio of the image resolution and the display resolution, and displayed in a single line with the same opening in the vertical direction; rendering and gaze according to the display resolution
  • Each image block whose area is in the same column is downsampled in the horizontal direction according to the ratio of image resolution and display resolution, and displayed in the vertical direction by opening multiple lines; for each image in the same line with the gaze area block, it is upsampled in the horizontal direction according to the ratio of the image resolution and the display resolution, and displayed in the vertical direction through a single-line opening method; for the other image blocks, it is The ratio of the resolution to the display resolution is up-sampled and displayed in the vertical direction through the simultaneous opening of multiple lines.
  • each image block in the same column (in the same row) as the gaze area includes: the column (row) sequence number set occupied by the internal pixels belongs to the image block of the column (row) sequence number set of the gaze area, and the column (row) sequence number set is the same as that of the image block.
  • Image blocks where the set of column (row) numbers of the gaze area partially intersects.
  • the display is performed in a single-row open mode, that is, the corresponding display data is transmitted to the display panel through a single-row open driving circuit, and the driving voltage only drives one row of pixel units. Therefore, the driving of the row of pixel units
  • the current is large, corresponding to high-resolution display; the display is carried out in the multi-line open mode, that is, the corresponding display data is transmitted to the display panel through the multi-line open drive circuit, and the driving voltage drives the multi-row pixel units.
  • the drive current of the pixel unit is relatively small, corresponding to low-resolution display, and the number of lines opened in multiple lines is determined according to the ratio of image resolution and display resolution.
  • the embodiment of the present disclosure provides an image display method, which can be used to display the reorganized image on a display device with mismatched resolution on the basis of retaining the original image quality and necessary image details, so as to improve the display quality of the image. adaptability.
  • FIG. 10 is a schematic diagram of an image processing process of still another image display method provided by an embodiment of the present disclosure.
  • the method uses the image compression method described in Figure 5a to generate a compressed image. Specifically, it is applied to an image display device.
  • the image display device is a virtual reality device, and the resolution corresponding to its display unit is 4320*4800.
  • the original image is a rectangle, its resolution is 2160*2400, and the corresponding color space is RGB space.
  • the user's eyeball image is acquired through the internal miniature image acquisition device, the center of the user's pupil is determined, and the position of the pupil center is mapped to the original image, so that the mapped point is used as the gaze point of the human eye;
  • the point is the center of symmetry, the fixation area B is determined according to the preset length and width, and the rest of the image is taken as the non-fixation area, where the fixation area B is a rectangle inside the original image, and its width (the size of the horizontal direction in the figure) ) is half of the original image, and its corresponding resolution is 1080*1600;
  • the non-gazing area is divided into multiple non-overlapping image blocks along the edge of the B area to obtain image blocks A1, A2, C1, C2, F1, F2 , F3 and F4, where the widths of A1 and A2 must be the
  • the width of B is half of the original image, the sum of the widths of C1 and C2, the sum of the widths of F1 and F2, and the sum of the widths of F3 and F4 must be equal to
  • the width of B rearrange each image block to generate a rearrangement area, and convert the color space of the rearrangement area to YUV space, each image block and the rearrangement area are rectangles, and the resolution corresponding to the rearrangement area is 1080 *3200, therefore, the corresponding resolution of the rearranged image is 1080*4800; the Y channel of the rearrangement area is 1/2 sampled in the vertical direction, and the U channel and V channel of the rearrangement area are in the vertical direction.
  • the internal graphics processor transmits the compressed image to the display controller. Therefore, the compressed image needs to be decompressed to restore the original image. Recombination is performed to restore each image block, and image blocks A1', A2', C1', C2', F1', F2', F3' and F4' are obtained, and the nearest neighbor difference value or
  • the bilinear difference decompresses the compressed image blocks to obtain the restored original image, and its corresponding resolution is 2160*2400; because the resolution corresponding to the current restored original image and the display resolution corresponding to the display unit exist.
  • the difference when displaying, for the gaze area B, the pixels are arranged and rendered according to the display resolution.
  • the corresponding resolution becomes twice the original, and it is weighted by two pixels in the horizontal direction to obtain a pixel. 1/2 sampling is performed in the way of pixels, and the area B' is obtained by displaying it in a single row in the vertical direction; for the image blocks A1' and A2' that are in the same column as the gaze area, the pixels are rendered according to the display resolution.
  • Blocks C1' and C2' are up-sampled with a sampling multiple of 2 in the horizontal direction by means of duplication or interpolation calculation, and are displayed in the vertical direction by a single-line opening method to obtain C1" and C2"; for the rest
  • the image blocks F1', F2', F3' and F4' are up-sampled with a sampling multiple of 2 in the horizontal direction, and displayed in the vertical direction through the same opening method of four lines to obtain F1", F2", F3 "and F4". So far, the display of the restored original image is completed, and the resolution of the final displayed image conforms to the resolution corresponding to the display unit, which is 4320*4800.
  • Embodiments of the present disclosure provide an image compression apparatus, including:
  • One or more processors a storage unit for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned embodiments of any image compression method.
  • Embodiments of the present disclosure provide an image display device, including:
  • One or more processors a storage unit for storing one or more programs; a display unit; when the one or more programs are executed by the one or more processors, the one or more processors are implemented as described above In any of the image display methods in the embodiments, the display unit displays the restored original image.
  • the image display device is a virtual reality device or an augmented reality device. Its display unit can correspond to an external display or a screen and related display elements inside the head-mounted device.
  • An embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored, wherein when the program is executed by a processor, the steps in any of the image compression methods in the above-mentioned embodiments are implemented, or the program is processed When the controller is executed, the steps in any of the image display methods in the above-mentioned embodiments are implemented.
  • Computer storage media includes both volatile and nonvolatile implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data flexible, removable and non-removable media.
  • Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or may Any other medium used to store desired information and which can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media, as is well known to those of ordinary skill in the art .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本公开提供了一种图像压缩方法，包括获取原始图像上的人眼注视点，并根据人眼注视点确定原始图像的注视区域和非注视区域；对非注视区域进行压缩，根据注视区域和压缩后的非注视区域生成压缩图像。本公开还提供了一种图像显示方法、图像压缩装置、图像显示装置和计算机可读介质。

Description

图像压缩方法及装置、图像显示方法及装置和介质 技术领域
本公开涉及图像视频技术领域，特别涉及一种图像压缩方法、图像显示方法、图像压缩装置、图像显示装置和计算机可读介质。
背景技术
随着电子科技水平的不断进步，虚拟现实(Virtual Reality，VR)或增强现实(Augmented Reality，AR)技术作为一种高新技术，已经越来越多地被应用在日常生活中。
现有的虚拟现实系统主要是通过带有中央处理器的高性能运算系统模拟一个虚拟的三维世界，并提供给使用者视觉、听觉等的感官体验，从而让使用者犹如身临其境，同时还可以进行人机互动。
发明内容
本公开旨在至少解决现有技术中存在的技术问题之一，提出了一种图像压缩方法、图像显示方法、图像压缩装置、图像显示装置和计算机可读介质。
为实现上述目的，第一方面，本公开实施例提供了一种图像压缩方法，包括：
获取原始图像上的人眼注视点，并根据所述人眼注视点确定所述原始图像的注视区域和非注视区域；
对所述非注视区域进行压缩，根据所述注视区域和压缩后的所述非注视区域生成压缩图像。
在一些实施例中，所述获取原始图像上的人眼注视点的步骤，包括：
获取包括用户眼睛的用户侧图像，根据所述用户侧图像检测人眼状态， 以确定所述人眼注视点。
在一些实施例中，所述根据所述人眼注视点确定所述图像的非注视区域的步骤，包括：
确定以所述人眼注视点为对称中心，位于具有预定尺寸的预定形状内的区域为注视区域；
确定所述图像的剩余部分为所述非注视区域。
在一些实施例中，所述原始图像为矩形；
所述确定以所述人眼注视点为对称中心，位于具有预定尺寸的预定形状内的区域为注视区域的步骤，包括：
以所述人眼注视点为对称中心，根据预先设置的长度和宽度确定所述注视区域。
在一些实施例中，所述对所述非注视区域进行压缩的步骤，包括：
将所述非注视区域分割为多个不重叠的图像块，并对各所述图像块进行重排，生成重排区域；
对所述重排区域进行压缩。
在一些实施例中，所述对所述重排区域进行压缩的步骤，包括：
根据预先设置的第一采样倍数对所述重排区域的色度通道在第一方向上进行下采样；
根据预先设置的第二采样倍数对所述重排区域的亮度通道和色度通道在第二方向上进行下采样；
其中，所述图像块为矩形，其四条边中有两条边平行于水平方向，另外两条边平行于垂直方向，所述第一方向为水平方向和垂直方向中的一者，所述第二方向为水平方向和垂直方向中的另一者；所述第一采样倍数大于所述第二采样倍数。
在一些实施例中，所述原始图像、所述注视区域、所述图像块、所述 重排区域以及所述压缩图像皆为矩形。
在一些实施例中，在所述根据预先设置的第一采样倍数对所述重排区域的色度通道在第一方向上进行采样的步骤之前，还包括：
若所述重排区域的色彩空间不为YUV空间，将所述重排区域的色彩空间转换为所述YUV空间。
第二方面，本公开实施例提供了一种图像显示方法，包括：
采用如上述实施例中任一所述的图像压缩方法生成压缩图像；
对压缩后的非注视区域进行解压缩，以还原得到复原原始图像；
显示所述复原原始图像。
在一些实施例中，在所述生成压缩图像和所述对所述压缩后的非注视区域进行解压缩的步骤之间，还包括：
自身的应用处理器向显示控制器传输所述压缩图像。
在一些实施例中，所述对所述压缩后的非注视区域进行解压缩的步骤，包括：
通过最邻近差值或双线性差值对所述压缩后的非注视区域进行解压缩。
在一些实施例中，所述压缩后的非注视区域是通过将非注视区域分割为多个不重叠的图像块，对各所述图像块进行重排，生成重排区域，并对所述重排区域进行压缩得到的；
在所述对所述压缩后的非注视区域进行解压缩的步骤之后，还包括：
对压缩后的所述重排区域进行重组，以还原各所述图像块。
在一些实施例中，所述注视区域为矩形，其四条边中有两条边平行于水平方向，另外两条边平行于垂直方向，所述图像块也为矩形；所述方法还包括：
若显示单元对应的显示分辨率大于所述复原原始图像的图像分辨率， 调整所述复原原始图像后进行显示；
其中，根据所述显示分辨率渲染所述注视区域，并对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行下采样，在垂直方向上通过单行同开方式进行显示；根据所述显示分辨率渲染与所述注视区域处于同列的各所述图像块，对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行下采样，在垂直方向上通过多行同开方式进行显示；对于与所述注视区域处于同行的各所述图像块，对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行上采样，在垂直方向上通过单行同开方式进行显示；对于其余各所述图像块，对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行上采样，在垂直方向上通过多行同开方式进行显示。
第三方面，本公开实施例提供了一种图像压缩装置，包括：
一个或多个处理器；
存储单元，用于存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如上述实施例中任一所述的图像压缩方法。
第四方面，本公开实施例提供了一种图像显示装置，包括：
一个或多个处理器；
存储单元，用于存储一个或多个程序；
显示单元；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如上述实施例中任一所述的图像显示方法，以使所述显示单元显示所述复原原始图像。
在一些实施例中，所述图像显示装置为虚拟现实设备或增强现实设备。
第五方面，本公开实施例提供了一种计算机可读介质，其上存储有计算机程序，其中，所述程序被处理器执行时实现如上述实施例中任一所述的图像压缩方法中的步骤，或所述程序被处理器执行时实现如上述实施例中任一所述的图像显示方法中的步骤。
附图说明
图1为本公开实施例提供的一种图像压缩方法的流程图；
图2为本公开实施例中步骤S1的一种具体实施方法流程图；
图3为本公开实施例中步骤S1的另一种具体实施方法流程图；
图4为本公开实施例中步骤S102的一种具体实施方法流程图；
图5为本公开实施例中步骤S2的一种具体实施方法流程图；
图5a为本公开实施例提供的另一种图像压缩方法的图像处理过程示意图；
图6为本公开实施例中步骤S202的一种具体实施方法流程图；
图7为本公开实施例提供的一种图像显示方法的流程图；
图8为本公开实施例提供的另一种图像显示方法的流程图；
图9为本公开实施例提供的又一种图像显示方法的流程图；
图10为本公开实施例提供的再一种图像显示方法的图像处理过程示意图。
具体实施方式
为使本领域的技术人员更好地理解本公开的技术方案，下面结合附图对本公开提供的图像压缩方法、图像显示方法、图像压缩装置、图像显示 装置和计算机可读介质进行详细描述。
在下文中将参考附图更充分地描述示例实施例，但是所述示例实施例可以以不同形式来体现且不应当被解释为限于本文阐述的实施例。反之，提供这些实施例的目的在于使本公开透彻和完整，并将使本领域技术人员充分理解本公开的范围。
本文所使用的术语仅用于描述特定实施例，且不意欲限制本公开。如本文所使用的，单数形式“一个”和“该”也意欲包括复数形式，除非上下文另外清楚指出。还将理解的是，当本说明书中使用术语“包括”和/或“由……制成”时，指定存在所述特征、整体、步骤、操作、元件和/或组件，但不排除存在或添加一个或多个其他特征、整体、步骤、操作、元件、组件和/或其群组。
将理解的是，虽然本文可以使用术语第一、第二等来描述各种元件，但这些元件不应当受限于这些术语。这些术语仅用于区分一个元件和另一元件。因此，在不背离本公开的指教的情况下，下文讨论的第一元件、第一组件或第一模块可称为第二元件、第二组件或第二模块。
除非另外限定，否则本文所用的所有术语(包括技术和科学术语)的含义与本领域普通技术人员通常理解的含义相同。还将理解，诸如那些在常用字典中限定的那些术语应当被解释为具有与其在相关技术以及本公开的背景下的含义一致的含义，且将不解释为具有理想化或过度形式上的含义，除非本文明确如此限定。
本公开所提供的图像压缩方法、图像显示方法、图像压缩装置、图像显示装置和计算机可读介质，可用于获取人眼注视点并划分图像的注视区域和非注视区域，通过对原始图像的注视区域和非注视区域分别进行单独处理，实现对注视区域的高清晰度渲染，以及对非注视区域的低清晰度渲染和压缩，在不影响观感的同时，减少非必要的数据量，进而解决虚拟显示技术对应的存储数据量较大，传输带宽较大的问题。
图1为本公开实施例提供的一种图像压缩方法的流程图。如图1所示， 该方法包括：
步骤S1、获取原始图像上的人眼注视点，并根据人眼注视点确定原始图像的注视区域和非注视区域。
其中，原始图像是对应的图像压缩装置根据用户侧设备的当前状态渲染得到的要显示的图像。例如，是根据虚拟现实或增强现实头戴式设备(如VR眼镜或AR眼镜)的状态确定要显示的内容，再由主机(如PC)的图形处理器(GPU)渲染得到原始图像。其中，该原始图像需要发送给主机或头戴式设备的显示控制器进行实际的显示。
其中，通过实时检测或设置固定点的方式确定人眼注视点，人眼注视点即用户的人眼当前直接注视的原始图像中的位置。而注视区域是指原始图像中距离人眼注视点较近的区域，也就是用户当前主要关注的区域；非注视区域则是指原始图像中距离人眼注视点较远的区域，也就是用户可能会看到但并不关注的区域。
图2为本公开实施例中步骤S1的一种具体实施方法流程图。具体地，如图2所示，在步骤S1中，获取原始图像上的人眼注视点的步骤，包括：步骤S101。
步骤S101、获取包括用户眼睛的用户侧图像，根据用户侧图像检测人眼状态，以确定人眼注视点。
其中，通过内置的图像视频采集设备获取包括用户眼部的用户侧图像，或接收客户端传输来的用户侧图像。
在一些实施例中，用户侧图像为用户的眼部图像或眼球图像；根据用户侧图像检测人眼状态，即根据用户的眼部图像或眼球图像提取用户的瞳孔图像，基于用户的瞳孔位置建立坐标系，并将瞳孔中心的坐标映射至原始图像上，或根据在原始图像中建立的坐标系进行坐标换算，由此，将映射点或换算结果对应的点位置作为人眼注视点。
或者，用户侧图像也可包括眼动信息，根据该眼动信息确定人眼注视 点。
步骤S2、对非注视区域进行压缩，根据注视区域和压缩后的非注视区域生成压缩图像。
在步骤S2中，对原始图像的注视区域和非注视区域分别进行单独处理，实现对注视区域的高清晰度渲染，以及对非注视区域的低清晰度渲染和压缩。
在一些实施例中，还包括：存储该压缩图像，以供后续对该压缩图像进行外部发送或内部传输。
本公开实施例提供了一种图像压缩方法，该方法可用于获取人眼注视点并划分图像的注视区域和非注视区域，通过对原始图像的注视区域和非注视区域分别进行单独处理，实现对用户关注的注视区域的高清晰度渲染，以及对用户不太关注的非注视区域的低清晰度渲染和压缩(因为用户不关注，故虽然低清晰较低，但对观感影响不大)，从而在不影响观感的同时，保留必要图像画质，减少非必要的数据量，进而解决虚拟显示技术对应的存储数据量较大，传输带宽较大的问题。
图3为本公开实施例中步骤S1的另一种具体实施方法流程图。具体地，如图3所示，在步骤S1中，根据人眼注视点确定图像的非注视区域的步骤，包括：步骤S102和步骤S103。
步骤S102、确定以人眼注视点为对称中心，位于具有预定尺寸的预定形状内的区域为注视区域。
其中，注视区域可为圆形、矩形或其他图形，从压缩编码的角度进行考虑，优选地，注视区域为矩形。
在一些实施例中，还包括：确定以人眼注视点为对称中心，像素数满足预先设置的数目条件的区域为注视区域。
图4为本公开实施例中步骤S102的一种具体实施方法流程图。具体地，原始图像为矩形，注视区域也为矩形；如图4所示，步骤S102，确定以人 眼注视点为对称中心，位于具有预定尺寸的预定形状内的区域为注视区域的步骤，包括：步骤S1021。
步骤S1021、以人眼注视点为对称中心，根据预先设置的长度和宽度确定注视区域。
其中，注视区域的长度和宽度可根据原始图像的亮度、对比度、灰度分布、色彩分布和像素数等参数进行适应性设置，注视区域的长度和宽度小于原始图像的长度和宽度中的任意一者，且注视区域的矩形的边分别与原始图像的相应的边平行(或者说注视区域是原始图像矩形中的“小矩形”)。
步骤S103、确定图像的剩余部分为非注视区域。
本公开实施例提供了一种图像压缩方法，该方法可用于通过检测出的人眼注视点确定注视区域和非注视区域，实现对原始图像的划分。
图5为本公开实施例中步骤S2的一种具体实施方法流程图。具体地，如图5所示，步骤S2，对非注视区域进行压缩的步骤，包括：步骤S201和步骤S202。
步骤S201、将非注视区域分割为多个不重叠的图像块，并对各图像块进行重排，生成重排区域。
其中，可根据非注视区域的亮度、对比度、灰度分布、色彩分布和像素数等参数使用任意分割方法对非注视区域进行分割；由分割后的图像块重排生成的重排区域可为规则图像或不规则图形。
在一些实施例中，原始图像、注视区域、图像块、重排区域以及压缩图像皆为矩形。其中，从压缩编码的角度进行考虑，该设置的编码效率较高，编码复杂度较低。
例如，图5a为本公开实施例提供的另一种图像压缩方法的图像处理过程示意图。如图5a所示，B为注视区域，其宽度(图中水平方向的尺寸，以像素个数代表)为1080，是原始图像宽度2160的一半，而非注视区域的图像块划分是沿着B区域的边的而延长线进行划的，从而得到图像块A1、 A2、C1、C2、F1、F2、F3和F4；其中，A1和A2的宽度必然与B相同；而由于B的宽度为原始图像的一半，故C1与C2的宽度和、F1与F2的宽度和以及F3与F4的宽度和也都必然等于B的宽度，因此可将这些图像块重排为图中重排后图像的形状，随后对重排后的重排区域进行压缩，根据优化排布进行旋转后得到D区域，实现原始图像、注视区域、图像块、重排区域以及压缩图像皆为矩形。
步骤S202、对重排区域进行压缩。
图6为本公开实施例中步骤S202的一种具体实施方法流程图。具体地，如图6所示，步骤S202，对重排区域进行压缩的步骤，包括：步骤S2021和步骤S2022。
步骤S2021、根据预先设置的第一采样倍数对重排区域的色度通道在第一方向上进行下采样。
步骤S2022、根据预先设置的第二采样倍数对重排区域的亮度通道和色度通道在第二方向上进行下采样。
其中，在步骤S201中分割出的图像块为矩形，其四条边中有两条边平行于水平方向，另外两条边平行于垂直方向，第一方向为水平方向和垂直方向中的一者，第二方向为水平方向和垂直方向中的另一者；第一采样倍数大于第二采样倍数，同样地，各采样倍数可根据非注视区域的亮度、对比度、灰度分布、色彩分布和像素数等参数进行适应性设置。
具体地，下采样即为将重排区域缩小的过程；由于亮度通道，即Y通道，包含较多图像细节信息，而色度通道，即U通道和V通道，包含较少图像细节信息，在YUV空间进行压缩时，对亮度通道和色度通道进行不同间隔和采样倍数的采样，已实现兼顾非注视区域的图像压缩比和峰值信噪比的目的。
在一些实施例中，在步骤S2021，根据预先设置的第一采样倍数对所重排区域的色度通道在第一方向上进行采样的步骤之前，还包括：若重排区 域的色彩空间不为YUV空间，将重排区域的色彩空间转换为YUV空间。
需要说明的是，采用步骤S2011和步骤S2022来实现对重排区域的压缩，仅为本公开中的一种可选实现方式，其不会对本公开的技术方案产生限制，在其他不同的方向和色彩空间进行的压缩编码同样适用于本公开的技术方案。
本公开实施例提供了一种图像压缩方法，该方法可用于对不规则的非注视区与进行图像重排，便于后续进行压缩编码，提升编码效率，降低复杂度。
图7为本公开实施例提供的一种图像显示方法的流程图。如图7所示，该方法包括：
步骤S3、采用如上述实施例中的任一图像压缩方法生成压缩图像。
其中，生成的压缩图像包括注视区域和压缩后的非注视区域。在一些实施例中，该注视区域为经过无损压缩的。
步骤S4、对压缩后的非注视区域进行解压缩，以还原得到复原原始图像。
其中，显示控制器可根据与压缩方式对应的解压缩方式，对原始图像进行还原，生成复原原始图像，并将其实际进行显示。生成的复原原始图像包括原先的注视区域和丢失了部分图像细节和色彩信息的非注视区域(即复原原始图像的分辨率与原始图像相同，但非关非注视区域携带的实际信息比原始图像少)。
在一些实施例中，还包括：对复原原始图像中非注视区域对应的位置进行图像增强处理
在一些实施例中，在步骤S3生成压缩图像和步骤S4对压缩后的非注视区域进行解压缩的步骤之间，还包括：自身的应用处理器向显示控制器传输压缩图像。
其中，应用处理器，例如图形处理器，对当前需要显示的画面进行渲 染得到原始图像，并采用如上述实施例中的任一图像压缩方法生成压缩图像后，自发地或基于相应指令，将该压缩图像传输至显示控制器，以供该显示控制器进行解压缩并进行实际显示。
一般来说，进行图形渲染的功能单元与进行显示的单元并不是一体的，如在虚拟现实设备中，主机的图形处理器负责进行图形渲染(产生原始图像)，而负责进行显示的是头戴式设备的显示控制器。因此，图像需要被发送(如从主机发送到头戴式设备)。而因为本实施例中在发送前对原始图像进行了压缩，实际发送的是数据量较小的压缩图像，所以内部传输和进入缓存的数据量减少，系统负荷降低。
在一些实施例中，步骤S4，对压缩后的非注视区域进行解压缩的步骤，包括：通过最邻近差值或双线性差值对压缩后的非注视区域进行解压缩。
步骤S5、显示复原原始图像。
本公开实施例提供了一种图像显示方法，该方法可用于获通过对原始图像的注视区域和非注视区域分别进行单独处理，以及对非注视区域的低清晰度渲染，在不影响观感的同时，保留必要图像画质，减少非必要的数据量，进而解决虚拟显示技术对应的存储数据量较大，传输带宽较大的问题。
图8为本公开实施例提供的另一种图像显示方法的流程图。如图8所示，该方法为基于图7所示方法的一种具体化可选实施方案。其中，压缩后的非注视区域是通过将非注视区域分割为多个不重叠的图像块，对各图像块进行重排，生成重排区域，并对重排区域进行压缩得到的；具体地，该方法不仅包括步骤S3～步骤S5，在步骤S4，对压缩后的非注视区域进行解压缩的步骤之后，还包括步骤S401。下面仅对步骤S401进行详细描述。
步骤S401、对压缩后的重排区域进行重组，以还原各图像块。
其中，根据与重排方式对应的重组方式，对各图像块进行还原，以生成复原原始图像。还原出的各图像块丢失了部分图像细节和色彩信息，但 块的大小不变。
图9为本公开实施例提供的又一种图像显示方法的流程图。如图9所示，该方法为基于图8所示方法的一种具体化可选实施方案。其中，注视区域为矩形，其四条边中有两条边平行于水平方向，另外两条边平行于垂直方向，图像块也为矩形；具体地，该方法不仅包括步骤S3～步骤S5，在步骤S401，对压缩后的重排区域进行重组的步骤之后，还包括步骤S402。下面仅对步骤S402进行详细描述。
步骤S402、若显示单元对应的显示分辨率大于复原原始图像的图像分辨率，调整复原原始图像后进行显示。
其中，根据显示分辨率渲染注视区域，并对其在水平方向上根据图像分辨率和显示分辨率之比进行下采样，在垂直方向上通过单行同开方式进行显示；根据显示分辨率渲染与注视区域处于同列的各图像块，对其在水平方向上根据图像分辨率和显示分辨率之比进行下采样，在垂直方向上通过多行同开方式进行显示；对于与注视区域处于同行的各图像块，对其在水平方向上根据图像分辨率和显示分辨率之比进行上采样，在垂直方向上通过单行同开方式进行显示；对于其余各所述图像块，对其在水平方向上根据图像分辨率和显示分辨率之比进行上采样，在垂直方向上通过多行同开方式进行显示。
具体地，与注视区域处于同列(同行)的各图像块包括：内部像素所占用的列(行)序号集合属于注视区域的列(行)序号集合的图像块，以及列(行)序号集合与注视区域的列(行)序号集合部分相交的图像块。
其中，对于相同的驱动电压，通过单行同开方式进行显示，即通过单行同开驱动电路将对应的显示数据传输至显示面板，其驱动电压只驱动一行像素单元，因此，该行像素单元的驱动电流较大，对应高分辨率显示；通过多行同开方式进行显示，即通过多行同开驱动电路将对应的显示数据传输至显示面板其驱动电压驱动多行像素单元，相对地，每行像素单元的驱动电流较小，对应低分辨率显示，其多行同开的行数根据图像分辨率和 显示分辨率之比确定。
本公开实施例提供了一种图像显示方法，该方法可用于在保留原有画质和必要图像细节的基础上，将重组后的图像显示在分辨率不匹配的显示设备上，提升图像显示的适应性。
下面对本公开提供的图像显示方法结合实际应用进行详细描述。
图10为本公开实施例提供的再一种图像显示方法的图像处理过程示意图。如图10所示，该方法采用如图5a中所描述的图像压缩方法生成压缩图像。具体地，应用于图像显示装置，该图像显示装置为虚拟现实设备，其显示单元对应的分辨率为4320*4800。
参见图5a，经图像处理器渲染后，原始图像为矩形，其分辨率为2160*2400，对应的色彩空间为RGB空间。首先，通过内部的微型图像采集设备获取用户的眼球图像，确定用户的瞳孔中心，并将瞳孔中心的点位置映射至原始图像上，由此，将映射点作为人眼注视点；以人眼注视点为对称中心，根据预先设置的长度和宽度确定注视区域B，并将图像的剩余部分作为非注视区域，其中，注视区域B为原始图像内部的一个矩形，其宽度(图中水平方向的尺寸)为原始图像的一半，其对应分辨率为1080*1600；将非注视区域沿着B区域的边分割为多个不重叠的图像块，得到图像块A1、A2、C1、C2、F1、F2、F3和F4，其中，A1和A2的宽度必然与B相同，由于B的宽度为原始图像的一半，故C1与C2的宽度和、F1与F2的宽度和以及F3与F4的宽度和必然等于B的宽度；对各图像块进行重排，生成重排区域，并将重排区域的色彩空间转换为YUV空间，各图像块和重排区域皆为矩形，重排区域对应的分辨率为1080*3200，由此，重排后图像对应的分辨率为1080*4800；对重排区域的Y通道在垂直方向上进行1/2采样，对重排区域的U通道和V通道在垂直方向上进行1/2采样，对重排区域的U通道和V通道在水平方向上进行1/5采样，在下采样完成后将其色彩空间转换回RGB空间，经优化排布后，得到区域D，其对应的分辨率为216*1600；最后生成的压缩图像对应的分辨率为1296*1600。
参见图10，压缩完成后，内部的图形处理器将该压缩图像传输至显示控制器，由此，需要对压缩图像进行解压缩，以还原得到复原原始图像，其中，对压缩后的重排区域进行重组，以还原各图像块，得到图像块A1’、A2’、C1’、C2’、F1’、F2’、F3’和F4’，根据进行下采样时的采样倍数通过最邻近差值或双线性差值对压缩后的各图像块进行解压缩，以得到复原原始图像，其对应的分辨率为2160*2400；由于当前复原原始图像对应的分辨率与显示单元对应的显示分辨率存在差异，在进行显示时，对于注视区域B，根据显示分辨率进行排布像素渲染，渲染后其对应的分辨率变为原先的两倍，则对其在水平方向上通过两个像素加权得到一个像素的方式进行1/2采样，在垂直方向上通过单行同开方式进行显示，得到区域B’；对于与注视区域处于同列的图像块A1’和A2’，根据显示分辨率进行排布像素渲染，对其在水平方向上通过间隔采样或多像素加权的方式进行1/2采样，在垂直方向上通过四行同开方式进行显示，得到A1”和A2”；对于与注视区域处于同行的图像块C1’和C2’，对其在水平方向上通过复制或插值计算的方式进行采样倍数为2的上采样，在垂直方向上通过单行同开方式进行显示，得到C1”和C2”；对于其余图像块F1’、F2’、F3’和F4’，对其在水平方向上进行采样倍数为2的上采样，在垂直方向上通过四行同开方式进行显示，得到F1”、F2”、F3”和F4”。至此，完成该复原原始图像的显示，最终显示图像的分辨率符合显示单元对应的分辨率，为4320*4800。
本公开实施例提供了一种图像压缩装置，包括：
一个或多个处理器；存储单元，用于存储一个或多个程序；当该一个或多个程序被该一个或多个处理器执行，使得该一个或多个处理器实现如上述实施例中的任一图像压缩方法。
本公开实施例提供了一种图像显示装置，包括：
一个或多个处理器；存储单元，用于存储一个或多个程序；显示单元；当该一个或多个程序被该一个或多个处理器执行，使得该一个或多个处理器实现如上述实施例中的任一图像显示方法，以使该显示单元显示复原原 始图像。
在一些实施例中，该图像显示装置为虚拟现实设备或增强现实设备。其显示单元可对应外接的显示器或头戴式设备内部的屏幕及相关显示元件。
本公开实施例提供了一种计算机可读介质，其上存储有计算机程序，其中，该程序被处理器执行时实现如上述实施例中的任一图像压缩方法中的步骤，或该程序被处理器执行时实现如上述实施例中的任一图像显示方法中的步骤。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
本文已经公开了示例实施例，并且虽然采用了具体术语，但它们仅用于并仅应当被解释为一般说明性含义，并且不用于限制的目的。在一些实 例中，对本领域技术人员显而易见的是，除非另外明确指出，否则可单独使用与特定实施例相结合描述的特征、特性和/或元素，或可与其他实施例相结合描述的特征、特性和/或元件组合使用。因此，本领域技术人员将理解，在不脱离由所附的权利要求阐明的本公开的范围的情况下，可进行各种形式和细节上的改变。

Claims (17)

  1. 一种图像压缩方法，其特征在于，包括：
    获取原始图像上的人眼注视点，并根据所述人眼注视点确定所述原始图像的注视区域和非注视区域；
    对所述非注视区域进行压缩，根据所述注视区域和压缩后的所述非注视区域生成压缩图像。
  2. 根据权利要求1所述的图像压缩方法，其特征在于，所述获取原始图像上的人眼注视点的步骤，包括：
    获取包括用户眼睛的用户侧图像，根据所述用户侧图像检测人眼状态，以确定所述人眼注视点。
  3. 根据权利要求1所述的图像压缩方法，其特征在于，所述根据所述人眼注视点确定所述图像的非注视区域的步骤，包括：
    确定以所述人眼注视点为对称中心，位于具有预定尺寸的预定形状内的区域为注视区域；
    确定所述图像的剩余部分为所述非注视区域。
  4. 根据权利要求3所述的图像压缩方法，其特征在于，所述原始图像为矩形；
    所述确定以所述人眼注视点为对称中心，位于具有预定尺寸的预定形状内的区域为注视区域的步骤，包括：
    以所述人眼注视点为对称中心，根据预先设置的长度和宽度确定所述注视区域。
  5. 根据权利要求1所述的图像压缩方法，其特征在于，所述对所述非注视区域进行压缩的步骤，包括：
    将所述非注视区域分割为多个不重叠的图像块，并对各所述图像块进行重排，生成重排区域；
    对所述重排区域进行压缩。
  6. 根据权利要求5所述的图像压缩方法，其特征在于，所述对所述重排区域进行压缩的步骤，包括：
    根据预先设置的第一采样倍数对所述重排区域的色度通道在第一方向上进行下采样；
    根据预先设置的第二采样倍数对所述重排区域的亮度通道和色度通道在第二方向上进行下采样；
    其中，所述图像块为矩形，其四条边中有两条边平行于水平方向，另外两条边平行于垂直方向，所述第一方向为水平方向和垂直方向中的一者，所述第二方向为水平方向和垂直方向中的另一者；所述第一采样倍数大于所述第二采样倍数。
  7. 根据权利要求5所述的图像压缩方法，其特征在于，所述原始图像、所述注视区域、所述图像块、所述重排区域以及所述压缩图像皆为矩形。
  8. 根据权利要求6所述的图像压缩方法，其特征在于，在所述根据预先设置的第一采样倍数对所述重排区域的色度通道在第一方向上进行采样的步骤之前，还包括：
    若所述重排区域的色彩空间不为YUV空间，将所述重排区域的色彩空 间转换为所述YUV空间。
  9. 一种图像显示方法，其特征在于，包括：
    采用如权利要求1至8中任一所述的图像压缩方法生成压缩图像；
    对压缩后的非注视区域进行解压缩，以还原得到复原原始图像；
    显示所述复原原始图像。
  10. 根据权利要求9所述的图像显示方法，其特征在于，在所述生成压缩图像和所述对所述压缩后的非注视区域进行解压缩的步骤之间，还包括：
    自身的应用处理器向显示控制器传输所述压缩图像。
  11. 根据权利要求9所述的图像显示方法，其特征在于，所述对所述压缩后的非注视区域进行解压缩的步骤，包括：
    通过最邻近差值或双线性差值对所述压缩后的非注视区域进行解压缩。
  12. 根据权利要求9所述的图像显示方法，其特征在于，所述压缩后的非注视区域是通过将非注视区域分割为多个不重叠的图像块，对各所述图像块进行重排，生成重排区域，并对所述重排区域进行压缩得到的；
    在所述对所述压缩后的非注视区域进行解压缩的步骤之后，还包括：
    对压缩后的所述重排区域进行重组，以还原各所述图像块。
  13. 根据权利要求12所述的图像显示方法，其特征在于，所述注视区域为矩形，其四条边中有两条边平行于水平方向，另外两条边平行于垂直方向，所述图像块也为矩形；所述方法还包括：
    若显示单元对应的显示分辨率大于所述复原原始图像的图像分辨率， 调整所述复原原始图像后进行显示；
    其中，根据所述显示分辨率渲染所述注视区域，并对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行下采样，在垂直方向上通过单行同开方式进行显示；根据所述显示分辨率渲染与所述注视区域处于同列的各所述图像块，对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行下采样，在垂直方向上通过多行同开方式进行显示；对于与所述注视区域处于同行的各所述图像块，对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行上采样，在垂直方向上通过单行同开方式进行显示；对于其余各所述图像块，对其在水平方向上根据所述图像分辨率和所述显示分辨率之比进行上采样，在垂直方向上通过多行同开方式进行显示。
  14. 一种图像压缩装置，包括：
    一个或多个处理器；
    存储单元，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-8中任一所述的图像压缩方法。
  15. 一种图像显示装置，包括：
    一个或多个处理器；
    存储单元，用于存储一个或多个程序；
    显示单元；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求9-13中任一所述的图像显示方法，以使所述显示单元显示所述复原原始图像。
  16. 根据权利要求15所述的图像显示装置，其特征在于，所述图像显示装置为虚拟现实设备或增强现实设备。
  17. 一种计算机可读介质，其上存储有计算机程序，其中，所述程序被处理器执行时实现如权利要求1-8中任一所述的图像压缩方法中的步骤，或，所述程序被处理器执行时实现如权利要求9-13中任一所述的图像显示方法中的步骤。
PCT/CN2021/092369 2020-07-31 2021-05-08 图像压缩方法及装置、图像显示方法及装置和介质 WO2022021991A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/764,359 US11917167B2 (en) 2020-07-31 2021-05-08 Image compression method and apparatus, image display method and apparatus, and medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010760340.X 2020-07-31
CN202010760340.XA CN114071150B (zh) 2020-07-31 2020-07-31 图像压缩方法及装置、图像显示方法及装置和介质

Publications (1)

Publication Number Publication Date
WO2022021991A1 true WO2022021991A1 (zh) 2022-02-03

Family

ID=80037522

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/092369 WO2022021991A1 (zh) 2020-07-31 2021-05-08 图像压缩方法及装置、图像显示方法及装置和介质

Country Status (3)

Country Link
US (1) US11917167B2 (zh)
CN (1) CN114071150B (zh)
WO (1) WO2022021991A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024093835A1 (zh) * 2022-11-01 2024-05-10 华为技术有限公司 一种图像数据的处理方法及相关设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117834830A (zh) * 2022-09-27 2024-04-05 万有引力（宁波）电子科技有限公司 图像处理器、处理方法、存储介质及扩展现实显示装置

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2926293A2 (en) * 2012-11-28 2015-10-07 The Trustees of Columbia University in the City of New York Methods, systems, and media for detecting gaze locking
CN107809641A (zh) * 2017-11-13 2018-03-16 北京京东方光电科技有限公司 图像数据传输方法、处理方法及图像处理设备、显示设备
CN108076384A (zh) * 2018-01-02 2018-05-25 京东方科技集团股份有限公司 一种基于虚拟现实的图像处理方法、装置、设备和介质
CN108270997A (zh) * 2016-12-30 2018-07-10 安讯士有限公司 注视控制的比特率
CN108391133A (zh) * 2018-03-01 2018-08-10 京东方科技集团股份有限公司 显示数据的处理方法、处理设备和显示设备
CN108665521A (zh) * 2018-05-16 2018-10-16 京东方科技集团股份有限公司 图像渲染方法、装置、系统、计算机可读存储介质及设备
CN108733202A (zh) * 2017-04-18 2018-11-02 北京传送科技有限公司 一种基于眼球追踪的数据压缩方法及其装置
CN110023881A (zh) * 2016-12-01 2019-07-16 上海云英谷科技有限公司 基于区域的显示数据处理和传输

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI468148B (zh) * 2012-03-23 2015-01-11 Crystalvue Medical Corp Auxiliary gaze and imaging focusing device
CN103475893B (zh) * 2013-09-13 2016-03-23 北京智谷睿拓技术服务有限公司 三维显示中对象的拾取装置及三维显示中对象的拾取方法
CN103500061B (zh) * 2013-09-26 2017-11-07 三星电子（中国）研发中心 控制显示器的方法及设备
US10334224B2 (en) * 2016-02-19 2019-06-25 Alcacruz Inc. Systems and method for GPU based virtual reality video streaming server
CN106325510B (zh) * 2016-08-19 2019-09-24 联想(北京)有限公司 信息处理方法及电子设备
JP2018147289A (ja) * 2017-03-07 2018-09-20 富士ゼロックス株式会社 表示装置、表示システム及びプログラム
CN106935224B (zh) * 2017-05-12 2019-06-07 京东方科技集团股份有限公司 显示装置及其驱动方法和驱动电路
CN107516335A (zh) * 2017-08-14 2017-12-26 歌尔股份有限公司 虚拟现实的图形渲染方法和装置
GB2568261B (en) * 2017-11-08 2022-01-26 Displaylink Uk Ltd System and method for presenting data at variable quality
CN109242943B (zh) * 2018-08-21 2023-03-21 腾讯科技（深圳）有限公司 一种图像渲染方法、装置及图像处理设备、存储介质
CN109658876B (zh) * 2019-02-28 2023-04-11 京东方科技集团股份有限公司 图像显示处理方法及装置、显示装置及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2926293A2 (en) * 2012-11-28 2015-10-07 The Trustees of Columbia University in the City of New York Methods, systems, and media for detecting gaze locking
CN110023881A (zh) * 2016-12-01 2019-07-16 上海云英谷科技有限公司 基于区域的显示数据处理和传输
CN108270997A (zh) * 2016-12-30 2018-07-10 安讯士有限公司 注视控制的比特率
CN108733202A (zh) * 2017-04-18 2018-11-02 北京传送科技有限公司 一种基于眼球追踪的数据压缩方法及其装置
CN107809641A (zh) * 2017-11-13 2018-03-16 北京京东方光电科技有限公司 图像数据传输方法、处理方法及图像处理设备、显示设备
CN108076384A (zh) * 2018-01-02 2018-05-25 京东方科技集团股份有限公司 一种基于虚拟现实的图像处理方法、装置、设备和介质
CN108391133A (zh) * 2018-03-01 2018-08-10 京东方科技集团股份有限公司 显示数据的处理方法、处理设备和显示设备
CN108665521A (zh) * 2018-05-16 2018-10-16 京东方科技集团股份有限公司 图像渲染方法、装置、系统、计算机可读存储介质及设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024093835A1 (zh) * 2022-11-01 2024-05-10 华为技术有限公司 一种图像数据的处理方法及相关设备

Also Published As

Publication number Publication date
CN114071150A (zh) 2022-02-18
CN114071150B (zh) 2023-06-16
US20220345723A1 (en) 2022-10-27
US11917167B2 (en) 2024-02-27

Similar Documents

Publication Publication Date Title
US6788309B1 (en) Method and apparatus for generating a video overlay
CN108665521B (zh) 图像渲染方法、装置、系统、计算机可读存储介质及设备
US11480804B2 (en) Distributed foveated rendering based on user gaze
US10540751B2 (en) Image data transmission method, image data processing method, image processing device and display device
CN105096797B (zh) 用于可变刷新率显示器的依赖刷新率的自适应抖动
DE202017105872U1 (de) Niedrigauflösendes RGB-Rendering für effiziente Übertragung
DE202017105882U1 (de) Fovealisierte Zweiwege-Graphikpipeline
EP0827615B1 (en) Dynamic image resizing
WO2022021991A1 (zh) 图像压缩方法及装置、图像显示方法及装置和介质
JP5439588B2 (ja) 表示パネルによる表示のための画像データ処理のための装置および方法
US20110304713A1 (en) Independently processing planes of display data
CN111292236B (zh) 一种减少中央凹注视渲染中的混叠伪像的方法和计算系统
CN111161660A (zh) 数据处理系统
CN108076384B (zh) 一种基于虚拟现实的图像处理方法、装置、设备和介质
CN115842907A (zh) 渲染方法、计算机产品及显示装置
WO2019101005A1 (zh) 像素补偿方法、装置和终端设备
WO1997014247A1 (en) Method and apparatus for video scaling and convolution for displaying computer graphics on a conventional television monitor
EP3993383A1 (en) Method and device for adjusting image quality, and readable storage medium
WO2022166712A1 (zh) 图像的显示方法、装置、可读介质和电子设备
WO2016033925A1 (zh) 显示方法及显示装置
WO2020015381A1 (en) Variable resolution rendering
EP3598393A1 (en) Rendering using a per-tile msaa level
CN112884661A (zh) 图像处理装置及其方法、显示装置和计算机可读存储介质
CN115185083B (zh) 一种ar头戴显示器自适应刷新率和分辨率渲染方法
WO2019159239A1 (ja) 画像処理装置、表示画像生成方法、およびフォントのデータ構造

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21849458

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21849458

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 09-08-23)

122 Ep: pct application non-entry in european phase

Ref document number: 21849458

Country of ref document: EP

Kind code of ref document: A1