WO2022111697A1 - 图片加载方法、电子设备、芯片及可读存储介质 - Google Patents

图片加载方法、电子设备、芯片及可读存储介质 Download PDF

Info

Publication number
WO2022111697A1
WO2022111697A1 PCT/CN2021/134087 CN2021134087W WO2022111697A1 WO 2022111697 A1 WO2022111697 A1 WO 2022111697A1 CN 2021134087 W CN2021134087 W CN 2021134087W WO 2022111697 A1 WO2022111697 A1 WO 2022111697A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
area
decoding
display
display area
Prior art date
Application number
PCT/CN2021/134087
Other languages
English (en)
French (fr)
Inventor
马明刚
刘志远
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022111697A1 publication Critical patent/WO2022111697A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/54—Browsing; Visualisation therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Definitions

  • the present application relates to the field of terminal technologies, and in particular, to a picture loading method, an electronic device, a chip, and a computer-readable storage medium.
  • the picture processing order of the existing display interface is usually performed in a fixed order. For example, for multiple pictures to be displayed, the processing order is from top to bottom, or from left to right, or from right to left.
  • the images in the user's area of interest may be processed and displayed at a slower rate, reducing the user experience.
  • a first aspect of the embodiments of the present application discloses a picture loading method, which includes: receiving a picture loading instruction; if there is a hotspot display area on a display interface, preferentially loading the picture content corresponding to the hotspot display area; wherein, the hotspot display area is The user's eye focus area.
  • the content of the pictures in the attention area of the human eye can be preferentially displayed, thereby improving the user experience.
  • the image loading method further includes: if it is detected that there is a viewing user before the display interface and the eye gaze position of the viewing user falls within the display interface, determining the display There is a hot spot display area on the interface; or if it is detected that there is no viewing user before the display interface, or there is a viewing user before the display interface and the eye gaze position of the viewing user does not fall within the display interface, determine the There is no hotspot display area on the display interface described above.
  • the eyeball attention areas of the viewing users in the preset whitelist are determined as the hot spot display area; Or if the eye gaze positions of a plurality of the viewing users fall within the display interface, the eye attention areas of the viewing users in the preset whitelist are determined as the hot spot display areas of the first priority level, and The eyeball attention area of the viewing user in the preset whitelist is determined as the hot spot display area of the second priority level.
  • the eyeball attention areas of the viewing users (such as the host) in the whitelist are preferentially decoded and displayed.
  • the display interface includes a plurality of pictures
  • the preferentially loading the picture content corresponding to the hot spot display area includes: packaging the decoding information and display position information of the plurality of pictures into multiple pictures. decoding tasks, wherein the decoding tasks are in one-to-one correspondence with the pictures, and the decoding information includes one or more of a picture storage path, a picture size, and a picture sampling rate;
  • the decoding task of the hot spot display area performs decoding processing, and performs rendering and display on the display interface according to the decoding result.
  • the pictures in the area of interest of the human eye can be preferentially decoded and displayed, thereby improving the user experience.
  • the method further includes: if the display position of the picture partially or completely falls within the hotspot display area, identifying the picture as a picture belonging to the hotspot display area.
  • the display interface includes a picture
  • the preferentially loading the picture content corresponding to the hotspot display area includes: rendering and displaying the picture on the display interface at a preset resolution ; preferentially select the picture area belonging to the hot spot display area for decoding processing, so as to render and display the picture area on the display interface with a first resolution; wherein, the first resolution is greater than the preset resolution Rate.
  • the preferentially selecting a picture area belonging to the hotspot display area for decoding processing includes: decomposing the picture into multiple areas according to a preset picture data block division rule; The regions are packaged into multiple region decoding tasks, and the region decoding tasks belonging to the hotspot display region are preferentially selected for decoding processing.
  • the decoding processing includes transcoding processing and color space conversion processing
  • the preferentially selecting a picture area belonging to the hotspot display area for decoding processing includes: converting the picture according to a preset picture
  • the data block division rules are decomposed into multiple areas; the multiple areas are packaged into multiple area decoding tasks; the transcoding processing is performed on the area decoding tasks belonging to the hot spot display area, and all the areas in the hot spot display area are processed.
  • the region decoding tasks of other dependent display regions perform the transcoding processing, and save the transcoding results; the region decoding tasks belonging to the hotspot display regions are preferentially selected to perform the color space conversion processing.
  • the decoding processing further includes down-sampling processing
  • the preferentially selecting an area decoding task belonging to the hotspot display area to perform the color space conversion processing includes: preferentially selecting areas belonging to the hotspot display area
  • the region decoding task of the region performs the color space conversion process and the downsampling process.
  • the method further includes: dividing a region of interest and a region of non-interest on the picture; when encoding the picture, using the region of interest as a coding dependency of the region of non-interest.
  • the coding dependency of the predicted region of interest as the region of non-interest can be realized, and it is convenient for the image decoding of the region of interest to be completed preferentially when the image decoding is performed on the terminal side subsequently.
  • the decoding processing includes transcoding processing and color space conversion processing
  • the preferentially selecting a picture area belonging to the hotspot display area for decoding processing includes: converting the picture according to a preset picture
  • the data block division rules are decomposed into multiple areas; the multiple areas are packaged into multiple area decoding tasks; if the picture area of the hot spot display area does not include a non-concerned area, the area belonging to the hot spot display area is preferentially selected
  • the decoding task performs the transcoding process and the color space conversion process.
  • the method further includes: if the picture area of the hotspot display area includes the non-concerned area, performing the transcoding process on the area decoding task belonging to the hotspot display area, and Perform the transcoding process on the area decoding tasks of other display areas that the hotspot display area depends on, and save the transcoding results; when each of the area decoding tasks completes the transcoding process, preferentially select the areas belonging to the hotspots
  • the area decoding task of the display area performs the color space conversion process.
  • the method further includes: if the display interface does not have a hotspot display area, loading the image content of the display interface according to a preset loading rule.
  • the image content of the display interface can be loaded by using a preset loading rule.
  • an embodiment of the present application provides a computer-readable storage medium, including computer instructions, when the computer instructions are executed on an electronic device, the electronic device causes the electronic device to execute the image loading method described in the first aspect.
  • an embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, the memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device Execute the image loading method described in the first aspect.
  • an embodiment of the present application provides a computer program product that, when the computer program product runs on a computer, causes the computer to execute the image loading method described in the first aspect.
  • an embodiment of the present application provides an apparatus, and the apparatus has a function of implementing the behavior of the electronic device in the method provided in the first aspect.
  • the functions can be implemented by hardware, or by executing corresponding software by hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the computer-readable storage medium described in the second aspect, the electronic device described in the third aspect, the computer program product described in the fourth aspect, and the apparatus described in the fifth aspect are all the same as the above-mentioned first aspect.
  • the method of the aspect corresponds to, therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, which will not be repeated here.
  • FIG. 1 is a schematic flowchart of a picture loading method provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of an application scenario in which the electronic device according to an embodiment of the present application displays multiple pictures;
  • FIG. 3 is a schematic diagram of an application scenario in which the electronic device according to an embodiment of the present application displays a single picture
  • FIG. 4 is a schematic diagram of functional modules of a picture loading apparatus provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a possible electronic device provided by an embodiment of the present application.
  • a picture loading method provided by an embodiment of the present application is applied to an electronic device 100 (as shown in FIG. 2 , FIG. 3 , or FIG. 5 ), and the electronic device 100 may be a TV, a computer, or an advertisement screen , mobile phones and other devices with a display screen.
  • the image loading method may include:
  • the picture loading instruction is used to trigger the display interface of the electronic device 100 to display one or more pictures.
  • the picture loading instruction may be to enter a certain application program (such as a picture application program, or other application program containing a picture) interface, a certain folder, click a certain picture of the picture thumbnail to display the original picture, and drag it.
  • the sidebar of the application interface, etc. is triggered. For example, the user clicks to enter the folder where the pictures are saved on the computer, the user enters the photo application of the mobile phone.
  • the display interface may be a display area of a display screen of the electronic device 100 .
  • the hot spot display area may be the eyeball attention area of the user
  • the electronic device 100 may be provided with a camera
  • the camera may be provided on the same side as the display screen of the electronic device 100
  • the electronic device 100 may determine the display based on the existing human eye tracking technology.
  • the electronic device 100 is equipped with an existing eye tracking device to obtain the eye movement trajectory data of the viewing user in front of the display interface to obtain a predicted viewing area (hot spot display area).
  • the camera of the electronic device 100 can be used to detect whether there is a viewing user in front of the display interface, when it is determined that there is a viewing user, continue to track and lock the eye movement angle of the current viewing user, and then confirm according to the eye movement angle
  • the current spatial coordinates of the exit pupil then map the current spatial coordinates of the pupil to the two-dimensional coordinate system of the display interface, and finally determine whether the eye gaze position of the viewing user falls within the display interface based on the placement of the pupil on the two-dimensional coordinate system. , and the eye area of interest when falling into the display interface.
  • the image content corresponding to the hotspot display area is preferentially loaded.
  • the hotspot display area may be defined as a high-priority area
  • the non-hotspot display area may be defined as a low-priority area.
  • the pictures in the high-priority area It is prioritized to perform operations such as image acquisition, image decoding, and image post-processing, so that in the case that the image processing capability of the electronic device 100 is not high, the image display processing of the high-priority area is preferentially completed, that is, the image display processing corresponding to the hotspot display area is preferentially loaded. image content.
  • the display interface is illustrated by taking a screen display area of a TV as an example.
  • the current number of pictures that can be displayed on the display interface is 15, which are picture 1 to picture 15.
  • the display area A1 is the high-priority area
  • the pictures covered by the display area A1 are Picture 6, Picture 7, Picture 11, and Picture 12.
  • the display interface is displaying pictures, it will give priority to the picture display processing of picture 6, picture 7, picture 11 and picture 12, so that picture 6, picture 7, picture 11 and picture 12 are displayed preferentially, and other pictures can be displayed in the picture. 6.
  • Picture 7, picture 11 and picture 12 are processed after the picture display process is completed.
  • the decoding information of each picture and the display position information on the display interface may be packaged into multiple decoding tasks, and then the decoding tasks belonging to the hotspot display area are preferentially selected for execution.
  • the decoding process can further realize that the decoding result of the hot spot display area is preferentially rendered and displayed on the display interface.
  • the multiple decoding tasks are in one-to-one correspondence with multiple pictures, and the decoding information of the pictures may include one or more of a picture storage path, a picture size, a picture rotation angle, and a picture sampling rate.
  • 15 decoding tasks can be obtained correspondingly, and the decoding tasks corresponding to picture 6, picture 7, picture 11 and picture 12 are preferentially selected for decoding processing, so that picture 6, picture 7, picture 11 And the picture 12 can be rendered and displayed on the display interface in advance compared with other pictures.
  • the processing order of each picture can be set according to actual requirements (for example, from left to right, from top to bottom, etc.), which is not limited here.
  • the processing order of each picture can also be set according to actual requirements (for example, from left to right, from top to bottom, etc.), which is not limited here.
  • the picture when the display position of the picture partially or completely falls within the hotspot display area, the picture may be identified as a picture belonging to the hotspot display area, that is, it needs to be preferentially displayed. For example, it can be set so that when half or more of the display position of the picture falls into the hot spot display area, it is determined that the picture belongs to the picture in the hot spot display area, or it can be set as one-third or three of the display position of the picture. When more than one part falls into the hot spot display area, it is determined that the image belongs to the image in the hot spot display area, which is not limited here.
  • the existing image loading rules can be used to load the image content of other non-hotspot display areas. For example, the image content of the non-hotspot display area is loaded sequentially from left to right and from top to bottom.
  • FIG. 3 it is a full-screen or original image display scene of a single image.
  • the single image can be selected as a higher-resolution image.
  • the display interface is also illustrated by taking the screen display area of a TV as an example.
  • a picture with a preset resolution lower than the original picture resolution
  • the picture area A2 belonging to the hot spot display area is preferentially selected for decoding processing, so as to realize the first resolution of the picture area A2 (For example, the original image resolution, or other specified resolutions lower than the original image resolution) are rendered and displayed on the display interface, and the first resolution is greater than the preset resolution.
  • the picture I1 can be displayed on the display interface at a resolution of 400P first, and then the hotspots are preferentially selected.
  • the picture area A2 of the display area is decoded, so that the picture area A2 is displayed at a resolution of 1440P first compared to other picture areas.
  • the display of the picture in the display state of the preset resolution is defined as the non-HD display state
  • the display of the picture in the display state of the first resolution is defined as the high-definition display state, that is, the picture is displayed during the display process.
  • all image areas are displayed in a non-HD display state first, and then the image areas belonging to the hotspot display area are preferentially selected to be displayed in a high-definition display state.
  • the picture when a single high-resolution picture is displayed, the picture may be decomposed into multiple regions according to preset picture data block division rules, and then the multiple regions may be packaged into multiple region decoding tasks , each region decoding task corresponds to a region, and then the region decoding task belonging to the hotspot display region is preferentially selected for decoding processing.
  • the preset picture data block division rule can be set according to actual requirements, for example, it is divided according to the 8*8 size data block BD1, that is, the 8*8 pixel area is one area decoding task.
  • the data block BD1 that partially or completely falls into the picture area A2 is preferentially subjected to decoding processing.
  • the picture decoding process can be roughly divided into a transcoding process, a color space conversion process, and a downsampling process.
  • some application scenarios of picture decoding and display may not include downsampling processing.
  • downsampling processing is required.
  • the DC coefficient differential encoding technology is used in the picture encoding process, and the picture will consider the difference between adjacent blocks when encoding. Similarity, reducing the amount of picture coding data, that is, there may be coding dependencies between adjacent data blocks BD1.
  • the transcoding process is dependent, while the color space conversion process and the downsampling process are independent of each other. If in the decoding process, the data block BD1 of the hotspot display area and the display area on which the hotspot display area depends is firstly subjected to transcoding processing, and then the data block BD1 belonging to the hotspot display area is preferentially selected for color space conversion processing and downsampling processing. The user can still see the picture content of the hot spot display area more quickly.
  • the encoding process of a picture is generally to encode data blocks included in the picture in sequence from left to right and top to bottom, that is, the right data block is encoded with reference to the left data block, and the lower data block is encoded with reference to the left data block.
  • the upper data block is encoded, and based on this encoding rule, data blocks of other display areas (non-hotspot display areas) on which the encoding of the hotspot display area depends can be determined.
  • the picture when displaying a single high-resolution picture, the picture is decomposed into multiple area decoding tasks according to the preset picture data block division rules, and then the hotspot display area and the area of the display area on which the hotspot display area depends are decoded. The task is transcoded, and the transcoding result is saved.
  • the area decoding task belonging to the hotspot display area is preferentially selected for color space conversion processing and downsampling processing, so as to preferentially load the corresponding hotspot display area. image content.
  • the transcoding process includes an inverse difference process.
  • the image is decomposed into multiple area decoding tasks according to the preset image data block division rules, and then the hotspot display area and the hotspot display area are processed.
  • the region decoding task of the dependent display area is transcoded, and the results of the inverse difference process of the region decoding task are saved.
  • the results of the inverse difference process of these region decoding tasks are obtained, the region that belongs to the hotspot display area is preferentially selected for decoding.
  • the task performs color space conversion processing and down-sampling processing, which can implement preferential loading of the image content corresponding to the hotspot display area.
  • the non-concerned areas are encoded with reference to the attention areas, and then the coding can be performed during the encoding process.
  • the region of interest and the region of non-interest are divided on the picture, and the number of the region of interest and the region of non-interest can be set according to actual needs.
  • the data block BD1 of the region of interest can be used as the encoding dependency of the data block BD1 of the region of non-interest, and the encoding compression rate can be improved.
  • picture decoding is required, the picture is decomposed into multiple area decoding tasks.
  • the picture area of the hot spot display area does not include the non-concerned area, it indicates that there is no coding dependency between the picture area of the hot spot display area and the non-concerned area, and it can be selected first.
  • the area decoding task belonging to the hot spot display area is subjected to transcoding processing, color space conversion processing and downsampling processing, and there is no need to perform transcoding processing on other non-hot spot display areas.
  • the picture area of the hotspot display area includes a non-concerned area
  • the area decoding task of the hotspot display area and the display area on which the hotspot display area depends can be decoded.
  • the area decoding task belonging to the hotspot display area is preferentially selected for color space conversion processing and downsampling processing.
  • the eyeball attention area of the viewing user in the preset whitelist may be determined as the hot spot display area, and the eyeball attention area of other viewing users not in the preset whitelist is ignored.
  • the loading rule for the multiple hotspot display areas may be: randomly load the image content corresponding to each hotspot display area, or The image content corresponding to each hotspot display area is loaded in the right-to-right, top-to-bottom loading order, or other custom loading rules.
  • the preset whitelist can be pre-built and saved to the electronic device 100. For example, the preset whitelist saves the owner information of the electronic device 100, and it can be determined whether the current viewing user is in the preset whitelist through face recognition technology. user.
  • the eyeball attention area of the viewing users in the preset whitelist may also be determined as the hot spot display area of the first priority level, and the The eyeball attention areas of the viewing users who are not in the preset whitelist are determined as the hot spot display areas of the second priority level, and the non-hot spot display areas are of the third priority level.
  • the electronic device 100 first performs image display processing on the hotspot display areas corresponding to the viewing users in the whitelist, and then performs image display processing on the hotspot display areas corresponding to the viewing users not in the whitelist, and finally performs image display processing on the hotspot display areas corresponding to the viewing users not in the whitelist.
  • Image display processing is performed in the hotspot display area. It can be understood that, for the display area of each priority level, the loading order of the image content, such as from left to right and from top to bottom, can also be used to perform the loading of the picture content.
  • the display interface does not have a hotspot display area, load the image content of the display interface according to a preset loading rule.
  • the image content of the display interface may be loaded according to a preset loading rule.
  • the preset loading rule may be an existing image loading rule, such as loading image content from left to right and top to bottom.
  • the above picture loading method determines the user's attention area based on the attention range of the human eye, and adjusts the decoding order of multiple pictures on the display interface or the decoding order of the area blocks in a single picture, so that the pictures or picture blocks within the user's attention area are given priority.
  • Decoding display improves user experience.
  • the picture loading apparatus 10 may be provided in the electronic device 100 .
  • the picture loading apparatus 10 may include an eye tracking module 101 , a decoding task generation module 102 , a decoding task management module 103 , a decoding module 104 and a display module 105 .
  • the eye-tracking module 101 is configured to track the user's eye-focused area on the display interface based on the user's eye-eye data captured by the camera, and the eye-focused area is the hot spot display area.
  • the eyeball attention area tracked by the eyeball tracking module 101 can be transmitted to the decoding task management module 103 .
  • the decoding task generating module 102 is configured to package the display position information and decryption information of multiple pictures into multiple decoding tasks in response to the picture loading instruction, and each decoding task corresponds to one picture.
  • the multiple decoding tasks packaged by the decoding task generating module 102 may be transmitted to the decoding task management module 103 .
  • the decoding task management module 103 is configured to select a decoding task from a plurality of decoding tasks and transmit it to the decoding module 104 for decoding processing.
  • the decoding task management module 103 preferentially selects the decoding task belonging to the hot spot display area and transmits it to the decoding module 104 for decoding processing.
  • the display module 105 is configured to control the display interface of the electronic device 100 to render and display according to the decoding result output by the decoding module 104 .
  • the display module 105 first renders and displays the picture on the display interface at a preset resolution.
  • the eye-tracking module 101 is configured to track the user's eye-focused area on the display interface based on the user's eye-eye data captured by the camera, and the eye-focused area is the hot spot display area.
  • the eyeball attention area tracked by the eyeball tracking module 101 can be transmitted to the decoding task management module 103 .
  • the decoding task generating module 102 is configured to, in response to the picture loading instruction, divide the decoding of a single picture into multiple region decoding tasks, and store them in the region decoding task queue, and each region decoding task may correspond to an image block.
  • the region decoding task queue generated by the decoding task generating module 102 can be transmitted to the decoding task management module 103.
  • the decoding task management module 103 is configured to select a region decoding task from the region decoding task queue and transmit it to the decoding module 104 for decoding processing.
  • the decoding task management module 103 selects a region decoding task, it preferentially selects the region decoding task belonging to the hotspot display region and transmits it to the decoding module 104 for decoding processing.
  • the image bitmap data of the image block decoded by the decoding module 104 may be stored in the image region pool.
  • the display module 105 is configured to extract the decoded image bitmap data of the image block from the image area pool and render and display it on the designated area.
  • each module in the above picture loading apparatus 10 is only used for illustration. In other embodiments, the picture loading apparatus 10 may be divided into different modules as required to complete the above picture loading apparatus 10. Function.
  • each module in the picture loading apparatus 10 provided in the embodiment of the present application may be in the form of a computer program, and the above-mentioned one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments can be used Describe the execution process of the image loading apparatus 10 in the electronic device 100 .
  • the electronic device 100 may include a processor 1001 , a memory 1002 , a communication bus 1003 , and a camera 1004 .
  • the memory 1002 is used to store one or more computer programs 1005 (eg, the picture loading apparatus 10 of FIG. 4).
  • One or more computer programs 1005 are configured to be executed by the processor 1001 .
  • the one or more computer programs 1005 may include instructions, and the above-mentioned instructions may be used to implement the above-mentioned picture loading method in the electronic device 100 .
  • the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown, or some components may be combined, or some components may be split, or a different arrangement of components.
  • the processor 1001 may include one or more processing units, for example, the processor 1001 may include an application processor (application processor, AP), a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP) ), controller, video codec, DSP, CPU, baseband processor, and/or neural-network processing unit (NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • graphics processor graphics processor
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • DSP digital signal processor
  • CPU central processing unit
  • baseband processor baseband processor
  • NPU neural-network processing unit
  • the processor 1001 may also be provided with a memory for storing instructions and data.
  • the memory in processor 1001 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 1001 . If the processor 1001 needs to use the instruction or data again, it can be called directly from this memory. Repeated access is avoided, and the waiting time of the processor 1001 is reduced, thereby improving the efficiency of the system.
  • the processor 1001 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, SIM interface, and/or USB interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transceiver
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • memory 1002 may include high-speed random access memory, and may also include non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (Secure) Digital, SD) card, flash card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (Secure) Digital, SD) card, flash card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • This embodiment also provides a computer storage medium, where computer instructions are stored in the computer storage medium, and when the computer instructions are executed on the electronic device, the electronic device executes the above-mentioned relevant method steps to implement the image loading method in the above-mentioned embodiment.
  • This embodiment also provides a computer program product, which when the computer program product runs on the computer, causes the computer to execute the above-mentioned relevant steps, so as to realize the image loading method in the above-mentioned embodiment.
  • the embodiments of the present application also provide an apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a connected processor and a memory; wherein, the memory is used for storing computer execution instructions, and when the apparatus is running, The processor can execute the computer-executed instructions stored in the memory, so that the chip executes the image loading method in the above method embodiments.
  • the first electronic device, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved, reference may be made to the provided above. The beneficial effects in the corresponding method will not be repeated here.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined. Or it may be integrated into another device, or some features may be omitted, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components shown as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed to multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a readable storage medium.
  • a readable storage medium including several instructions to make a device (may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请实施例提供了一种图片加载方法，涉及终端领域。通过在显示界面加载图片时，若确定电子设备的显示界面存在热点显示区域，优先加载热点显示区域对应的图片内容，其中该热点显示区域为用户眼球关注区域。本申请实施例还提供了一种电子设备、芯片及计算机可读存储介质。本申请基于人眼的关注区域来调整图片的加载顺序，使得人眼关注区域内的图片内容被优先显示出来，提升用户使用体验。

Description

图片加载方法、电子设备、芯片及可读存储介质
本申请要求于2020年11月30日提交中国专利局，申请号为202011379419.4、申请名称为“图片加载方法、电子设备、芯片及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及一种图片加载方法、电子设备、芯片及计算机可读存储介质。
背景技术
用户在浏览图片的时候，比如在大屏设备上浏览图片，通常会有一个视觉关注区域。但是现有的显示界面的图片处理顺序通常是按照固定顺序进行显示处理，比如对于多张待显示图片，处理顺序为从上至下，或从左至右，或从右至左。在一些图片处理能力较弱的设备上，用户关注区域内的图片可能会被较慢地进行处理显示，降低了用户使用体验。
发明内容
有鉴于此，有必要提供一种图片加载方法，其可克服上述问题，可以使得人眼关注区域内的图片内容被优先显示出来。
本申请实施例第一方面公开了一种图片加载方法，包括：接收图片加载指令；若显示界面存在热点显示区域，优先加载所述热点显示区域对应的图片内容；其中，所述热点显示区域为用户眼球关注区域。
通过采用该技术方案，可使得人眼关注区域内的图片内容被优先显示出来，提升用户使用体验。
在一种可能的实现方式中，所述图片加载方法还包括：若侦测到所述显示界面前存在观看用户且所述观看用户的眼球注视位置落入所述显示界面内，确定所述显示界面存在热点显示区域；或若侦测到所述显示界面前不存在观看用户、或者所述显示界面前存在观看用户且所述观看用户的眼球注视位置未落入所述显示界面内，确定所述显示界面不存在热点显示区域。
通过采用该技术方案，可实现确定显示界面是否存在热点显示区域，便于后续对人眼关注区域内的图片内容进行优先解码显示。
在一种可能的实现方式中，若多个所述观看用户的眼球注视位置落入所述显示界面内，将在预设白名单内的观看用户的眼球关注区域确定为所述热点显示区域；或若多个所述观看用户的眼球注视位置落入所述显示界面内，将在预设白名单内的观看用户的眼球关注区域确定为第一优先等级的热点显示区域，及将未在所述预设白名单内的观看用户的眼球关注区域确定为第二优先等级的热点显示区域。
通过采用该技术方案，可实现在显示界面存在多个热点显示区域时，优先将白名单内的 观看用户(比如机主)的眼球关注区域进行优先解码显示。
在一种可能的实现方式中，所述显示界面包括多张图片，所述优先加载所述热点显示区域对应的图片内容，包括：将所述多张图片的解码信息及显示位置信息打包成多个解码任务，其中，所述多个解码任务与所述多张图片一一对应，所述解码信息包括图片存储路径、图片大小及图片采样率中的一种或多种；优先选取属于所述热点显示区域的解码任务进行解码处理，并根据解码结果在所述显示界面进行渲染显示。
通过采用该技术方案，可实现在多张图片显示的场景中，将人眼关注区域内的图片进行优先解码显示，提升用户使用体验。
在一种可能的实现方式中，所述方法还包括：若图片的显示位置部分落入或者全部落入所述热点显示区域，将所述图片认定为属于所述热点显示区域的图片。
在一种可能的实现方式中，所述显示界面包括一张图片，所述优先加载所述热点显示区域对应的图片内容包括：将所述图片以预设分辨率在所述显示界面进行渲染显示；优先选取属于所述热点显示区域的图片区域进行解码处理，以将所述图片区域以第一分辨率在所述显示界面进行渲染显示；其中，所述第一分辨率大于所述预设分辨率。
通过采用该技术方案，可实现在单张图片全屏显示的场景中，将人眼关注区域内的图片内容进行优先解码显示，提升用户使用体验。
在一种可能的实现方式中，所述优先选取属于所述热点显示区域的图片区域进行解码处理，包括：将所述图片按照预设图片数据块划分规则分解为多个区域；将所述多个区域打包成多个区域解码任务，并优先选取属于所述热点显示区域的区域解码任务进行解码处理。
通过采用该技术方案，可实现在单张图片全屏显示的场景中，将人眼关注区域内的图片内容进行优先解码显示，提升用户使用体验。
在一种可能的实现方式中，所述解码处理包括转码处理及颜色空间转换处理，所述优先选取属于所述热点显示区域的图片区域进行解码处理，包括：将所述图片按照预设图片数据块划分规则分解为多个区域；将所述多个区域打包成多个区域解码任务；对属于所述热点显示区域的区域解码任务进行所述转码处理，及对所述热点显示区域所依赖的其他显示区域的区域解码任务进行所述转码处理，并保存转码结果；优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理。
通过采用该技术方案，可实现在存在编码依赖的单张图片的全屏显示场景中，将人眼关注区域内的图片内容进行优先解码显示，提升用户使用体验。
在一种可能的实现方式中，所述解码处理还包括下采样处理，所述优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理，包括：优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理与所述下采样处理。
在一种可能的实现方式中，所述方法还包括：在所述图片上划分关注区域与非关注区域；当对所述图片进行编码时，将所述关注区域作为非关注区域的编码依赖。
通过采用该技术方案，可实现将预测的关注区域作为非关注区域的编码依赖，便于后续再终端侧进行图片解码时，优先完成关注区域的图像解码。
在一种可能的实现方式中，所述解码处理包括转码处理及颜色空间转换处理，所述优先选取属于所述热点显示区域的图片区域进行解码处理，包括：将所述图片按照预设图片数据块划分规则分解为多个区域；将所述多个区域打包成多个区域解码任务；若所述热点显示区域的图片区域不包括非关注区域时，优先选取属于所述热点显示区域的区域解码任务进行所 述转码处理与所述颜色空间转换处理。
通过采用该技术方案，可实现在存在编码依赖的单张图片的全屏显示场景中，仅需对热点显示区域的区域解码任务进行转码处理即可先将人眼关注区域内的图片内容优先显示出来。
在一种可能的实现方式中，所述方法还包括：若所述热点显示区域的图片区域包括所述非关注区域，对属于所述热点显示区域的区域解码任务进行所述转码处理，及对所述热点显示区域所依赖的其他显示区域的区域解码任务进行所述转码处理，并保存转码结果；在每一所述区域解码任务均完成转码处理时，优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理。
在一种可能的实现方式中，所述方法还包括：若所述显示界面不存在热点显示区域，按照预设加载规则加载所述显示界面的图片内容。
通过采用该技术方案，可实现在显示界面不存在热点显示区域时，采用预设加载规则加载显示界面的图片内容。
第二方面，本申请实施例提供一种计算机可读存储介质，包括计算机指令，当计算机指令在电子设备上运行时，使得电子设备执行如第一方面所述的图片加载方法。
第三方面，本申请实施例提供一种电子设备，所述电子设备包括处理器和存储器，所述存储器用于存储指令，所述处理器用于调用所述存储器中的指令，使得所述电子设备执行如第一方面所述的图片加载方法。
第四方面，本申请实施例提供一种计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机执行如第一方面所述的图片加载方法。
第五方面，本申请实施例提供一种装置，该装置具有实现上述第一方面所提供的方法中的电子设备行为的功能。功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。硬件或软件包括一个或多个与上述功能相对应的模块。
可以理解地，上述提供的第二方面所述的计算机可读存储介质，第三方面所述的电子设备，第四方面所述的计算机程序产品，第五方面所述的装置均与上述第一方面的方法对应，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
附图说明
图1为本申请一实施例提供的图片加载方法的流程示意图；
图2为本申请一实施例提供的电子设备进行多张图片显示的应用场景示意图；
图3为本申请一实施例提供的电子设备进行单张图片显示的应用场景示意图；
图4为本申请一实施例提供的图片加载装置的功能模块示意图；
图5为本申请一实施例提供的一种可能的电子设备的结构示意图。
具体实施方式
需要说明的是，本申请中“至少一个”是指一个或者多个，“多个”是指两个或多于两个。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。本申请的说明书和权利要求书及附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不是用于描述特定的顺序或先后次序。
为了便于理解，示例性的给出了部分与本申请实施例相关概念的说明以供参考。
参照图1所示，本申请实施例提供的一种图片加载方法，应用于电子设备100(如图2、图3、或图5所示)中，电子设备100可以是电视、电脑、广告屏、手机等具备显示屏的设备。本实施例中，图片加载方法可以包括：
11、接收图片加载指令。
在一些实施例中，所述图片加载指令用于触发电子设备100的显示界面显示一张或多张图片。所述图片加载指令可以是进入某个应用程序(比如图片应用程序、或者其他包含有图片的应用程序)界面、某个文件夹、点击图片缩略图的某一张图片进行原图显示、拖动应用程序界面的侧边栏等触发产生。比如，用户点击进入电脑上保存图片的文件夹、用户进入手机的照片应用程序。
在一些实施例中，当接收到图片加载指令时，可以确定显示界面是否存在热点显示区域，以判断是否需要进行优先显示处理。所述显示界面可以是电子设备100的显示屏的显示区域。所述热点显示区域可以为用户眼球关注区域，电子设备100可以设置有摄像头，该摄像头可以与电子设备100的显示屏设置在同一面，电子设备100可以基于现有的人眼追踪技术来确定显示界面是否存在热点显示区域，比如电子设备100装设现有的眼球追踪装置来获取显示界面前的观看用户的眼球运动轨迹数据，得到预测观看区域(热点显示区域)。当侦测到显示界面前存在观看用户且观看用户的眼球注视位置落入显示界面内时，可以确定显示界面存在热点显示区域；当侦测到显示界面前存在观看用户，但观看用户的眼球注视位置未落入显示界面内时，可以确定显示界面不存在热点显示区域；当侦测到显示界面前不存在观看用户，可以确定显示界面不存在热点显示区域。
举例而言，可以通过电子设备100的摄像头来侦测显示界面前是否存在观看用户，当确定存在观看用户时，再继续追踪并锁定当前观看用户的眼球的运动角度，然后根据眼球的运动角度确认出瞳孔当前的空间坐标，再将瞳孔当前的空间坐标映射到显示界面的二维坐标系上，最后基于瞳孔在二维坐标系上的落点确定观看用户的眼球注视位置是否落入显示界面内，以及落入显示界面内时的眼球关注区域。
12、若显示界面存在热点显示区域，优先加载所述热点显示区域对应的图片内容。
在一些实施例中，若显示界面存在热点显示区域，可以将热点显示区域定义为高优先级区域，非热点显示区域定义为低优先级区域，在进行图片显示时，高优先级区域内的图片被优先进行图片获取、图片解码、图片后处理等操作，从而使得在电子设备100的图片处理能力不高的情形下，优先完成高优先级区域的图片显示处理，即优先加载热点显示区域对应的图片内容。
举例而言，如图2所示，为多张图片的缩略显示场景，显示界面以电视的屏幕显示区域为例进行举例说明。显示界面当前可显示的图片数量为15张，分别为图片1～图片15。假设通过人眼追踪技术确定观看用户的眼球关注区域为显示区域A1，显示区域A1即为高优先级区域，显示区域A1涵盖的图片为图片6、图片7、图片11及图片12。显示界面在进行图片显示时，会优先对图片6、图片7、图片11及图片12进行图片显示处理，使得图片6、图片7、图片11及图片12被优先显示出来，而其他图片可以在图片6、图片7、图片11及图片12完成图片显示处理后再进行相应的处理。
在一些实施例中，在进行多张图片显示处理时，可以将每一图片的解码信息及在显示界面上的显示位置信息打包成多个解码任务，再优先选取属于热点显示区域的解码任务进行解 码处理，进而可以实现优先将热点显示区域的解码结果在显示界面上进行渲染显示。该多个解码任务与多张图片一一对应，图片的解码信息可以包括图片存储路径、图片大小、图片旋转角度、图片采样率中的一种或多种。如图2所示，对于15张图片，可以对应得到15个解码任务，优先选取与图片6、图片7、图片11及图片12对应的解码任务进行解码处理，使得图片6、图片7、图片11及图片12相比于其他图片可以提前在显示界面上进行渲染显示。
在一些实施例中，当高优先级区域包括多张图片时，每张图片的处理顺序可以根据实际需求进行设定(比如，从左至右、从上至下等)，在此不作限定，当低优先级区域包括多张图片时，每张图片的处理顺序同样可以根据实际需求进行设定(比如，从左至右、从上至下等)，在此亦不作限定。
在一些实施例中，当图片的显示位置部分落入或者全部落入热点显示区域时，可以将该图片认定为属于热点显示区域的图片，即需要被优先进行显示处理。比如，可以设定为当图片的显示位置的一半或者一半以上落入热点显示区域时，认定该图片属于热点显示区域的图片，也可以设定为当图片的显示位置的三分之一或者三分之一以上落入热点显示区域时，认定该图片属于热点显示区域的图片，在此不作限定。
在一些实施例中，当热点显示区域没有图片时，即热点显示区域对应的图片内容为空，无需进行优先加载，此时可以采用现有的图片加载规则加载其他非热点显示区域的图片内容，比如，从左至右、从上至下依次加载非热点显示区域的图片内容。
如图3所示，为单张图片的全屏或原图显示场景，该单张图片可选为分辨率较高的图片，显示界面同样以电视的屏幕显示区域为例进行举例说明。在进行图片显示时，可以先显示预设分辨率的图片(低于原图分辨率)，然后优先选取属于热点显示区域的图片区域A2进行解码处理，实现将该图片区域A2以第一分辨率(比如，为原图分辨率，或者为其他低于原图分辨率的指定分辨率)在显示界面进行渲染显示，第一分辨率大于预设分辨率。比如，对于分辨率为1440P的图片I1，若电子设备100的图片解码能力较弱，解码耗时较长，可以先将图片I1以400P的分辨率在显示界面上进行显示，然后优先选取属于热点显示区域的图片区域A2进行解码处理，使得将该图片区域A2相比于其他图片区域先以1440P的分辨率显示出来。
例如，在图3中，将图片以预设分辨率的显示状态进行显示定义为非高清显示状态，将该图片以第一分辨率的显示状态进行显示定义为高清显示状态，即图片在显示过程中，所有的图片区域先以非高清显示状态进行显示，然后再优先选取属于热点显示区域的图片区域以高清显示状态进行显示。
在一些实施例中，在进行单张高分辨率的图片显示处理时，可以将所述图片按照预设图片数据块划分规则分解为多个区域，再将多个区域打包成多个区域解码任务，每一区域解码任务对应一区域，然后优先选取属于所述热点显示区域的区域解码任务进行解码处理。该预设图片数据块划分规则可以根据实际需求进行设定，比如，按照8*8大小的数据块BD1进行划分，即8*8像素区域为一个区域解码任务。在图3中，部分落入或者全部落入图片区域A2的数据块BD1被优先进行解码处理。
在一些实施例中，图片解码过程可以大体划分为转码处理、颜色空间转换处理、下采样处理。其中，有些图片解码显示的应用场景也可能不包括下采样处理，比如对于4K分辨率的图片I1，若最终在电子设备的显示界面进行2K分辨率的显示，则需要进行下采样处理。 在进行单张高分辨率的图片显示处理时，对于某些格式的图片，比如JPEG格式图片，在进行图片编码过程中使用直流系数差分编码技术，图片在编码时会考虑相邻块之间的相似度，降低图片编码数据量，即相邻数据块BD1之间可能具有编码依赖。在进行图片解码时需按照一定的顺序才能解码成功。此种情形下，直接进行分块解码并没有效果。对于数据块BD1而言，转码处理存在依赖，而颜色空间转换处理与下采样处理是相互独立的。若在解码过程中，先对热点显示区域及热点显示区域所依赖的显示区域的数据块BD1进行转码处理，再优先选取属于热点显示区域的数据块BD1进行颜色空间转换处理与下采样处理，使得用户依然可以更快的看到热点显示区域的图片内容。
在一些实施例中，图片的编码过程一般是遵循从左到右、从上到下的顺序依次对图片所包含的数据块进行编码，即右边数据块参考左边数据块进行编码，下方数据块参考上方数据块进行编码，基于该种编码规则可以确定热点显示区域的编码所依赖的其他显示区域(非热点显示区域)的数据块。
例如，在进行单张高分辨率的图片显示处理时，将图片按照预设图片数据块划分规则分解为多个区域解码任务，再对热点显示区域及热点显示区域所依赖的显示区域的区域解码任务进行转码处理，并保存转码结果，当完成上述转码处理时，再优先选取属于热点显示区域的区域解码任务进行颜色空间转换处理与下采样处理，实现优先加载出热点显示区域对应的图片内容。
例如，转码处理包括反差分过程，在进行单张高分辨率的图片显示处理时，将图片按照预设图片数据块划分规则分解为多个区域解码任务，再对热点显示区域及热点显示区域所依赖的显示区域的区域解码任务进行转码处理，并保存区域解码任务的反差分过程的结果，当得到这些区域解码任务的反差分过程的结果时，再优先选取属于热点显示区域的区域解码任务进行颜色空间转换处理与下采样处理，可以实现优先加载出热点显示区域对应的图片内容。
在一些实施例中，对于单张高分辨率的图片显示场景，若在图片编码时提前预测图片的哪些区域可能会成为人眼关注区域，让非关注区域参考关注区域进行编码，进而可以在进行图片解码时，可以不需要对非热点显示区域的数据块BD1进行转码处理，而仅对热点显示区域的数据块BD1进行转码处理即可。具体地，在图片编码前，在图片上划分关注区域与非关注区域，关注区域与非关注区域的数量可以根据实际需求进行设定。当对该图片进行编码时，可以将关注区域的数据块BD1作为非关注区域的数据块BD1的编码依赖，提高编码压缩率。在需要进行图片解码时，将图片分解为多个区域解码任务，当热点显示区域的图片区域不包括非关注区域时，表明热点显示区域的图片区域与非关注区域不存在编码依赖，可以优先选取属于热点显示区域的区域解码任务进行转码处理、颜色空间转换处理与下采样处理，无需对其他非热点显示区域进行转码处理。当热点显示区域的图片区域包括非关注区域时，表明热点显示区域的图片区域与非关注区域存在编码依赖，此时，可以对热点显示区域及热点显示区域所依赖的显示区域的区域解码任务进行转码处理，在完成转码处理后，再优先选取属于热点显示区域的区域解码任务进行颜色空间转换处理与下采样处理。
在一些实施例中，显示界面前可能存在多个观看用户，进而基于人眼追踪技术可能在显示界面上确定多个眼球关注区域。此时，可以将在预设白名单内的观看用户的眼球关注区域确定为热点显示区域，而忽略其他不在预设白名单内的观看用户的眼球关注区域。若预设白名单内的观看用户包括多个(即包括多个热点显示区域)，该多个热点显示区域的加载规则可以是：随机加载每一热点显示区域对应的图片内容，或者按照从左至右、从上至下的加载顺 序对每一热点显示区域对应的图片内容进行加载，也可以是其他自定义的加载规则。该预设白名单可以预先构建并保存至电子设备100，比如该预设白名单保存了电子设备100的机主信息，可以通过人脸识别技术来确定当前观看用户是否为预设白名单内的用户。
在一些实施例中，当多个观看用户的眼球注视位置落入显示界面内时，也可以将在预设白名单内的观看用户的眼球关注区域确定为第一优先等级的热点显示区域，将未在该预设白名单内的观看用户的眼球关注区域确定为第二优先等级的热点显示区域，非热点显示区域为第三优先等级。即对于多个热点显示区域，电子设备100先对白名单内的观看用户对应的热点显示区域进行图片显示处理，再对不在白名单内的观看用户对应的热点显示区域进行图片显示处理，最后对非热点显示区域进行图片显示处理。可以理解，对于每一优先等级的显示区域，同样可以采用诸如从左至右、从上至下的加载顺序执行图片内容加载。
13、若所述显示界面不存在热点显示区域，按照预设加载规则加载所述显示界面的图片内容。
在一些实施例中，若显示界面不存在热点显示区域时，可以按照预设加载规则加载所述显示界面的图片内容。该预设加载规则可以是现有的图片加载规则，比如从左至右、从上至下进行图片内容加载。
上述图片加载方法，基于人眼的关注范围确定用户的关注区域，调整多张图片在显示界面的解码顺序或者单张图片中的区域块的解码顺序，使得用户关注范围内的图片或者图片块优先进行解码显示，提升了用户使用体验。
参照图4所示，本申请一实施例提供的一种图片加载装置10的功能模块示意图。图片加载装置10可以设置在电子设备100中。图片加载装置10可以包括眼球追踪模块101、解码任务生成模块102、解码任务管理模块103、解码模块104及显示模块105。
对于多张图片的缩略显示场景：
眼球追踪模块101用于基于摄像头捕捉到用户眼球数据来追踪该用户在显示界面的眼球关注区域，该眼球关注区域即为热点显示区域。眼球追踪模块101追踪到的眼球关注区域可以传给解码任务管理模块103。
解码任务生成模块102用于响应于图片加载指令，将多张图片的显示位置信息及解密信息打包成多个解码任务，每一解码任务对应一张图片。解码任务生成模块102打包得到的多个解码任务可以传给解码任务管理模块103。
解码任务管理模块103用于从多个解码任务中选取解码任务传送给解码模块104进行解码处理。解码任务管理模块103在进行解码任务选取时，优先选取属于热点显示区域的解码任务传送给解码模块104进行解码处理。
显示模块105用于根据解码模块104输出的解码结果控制电子设备100的显示界面进行渲染显示。
对于单张图片的全屏或原图显示场景：
显示模块105先将图片以预设分辨率在显示界面进行渲染显示。
眼球追踪模块101用于基于摄像头捕捉到用户眼球数据来追踪该用户在显示界面的眼球关注区域，该眼球关注区域即为热点显示区域。眼球追踪模块101追踪到的眼球关注区域可以传给解码任务管理模块103。
解码任务生成模块102用于响应于图片加载指令，将单张图片的解码划分为多个区域解码任务,并存入区域解码任务队列，每一区域解码任务可以对应一图像块。解码任务生成模 块102生成的区域解码任务队列可以传给解码任务管理模块103。
解码任务管理模块103用于从区域解码任务队列中选取区域解码任务传送给解码模块104进行解码处理。解码任务管理模块103进行区域解码任务选取时，优先选取属于热点显示区域的区域解码任务传送给解码模块104进行解码处理。解码模块104解码得到的图像块的图像位图数据可以存入图像区域池。
显示模块105用于从图像区域池中提取已解码好的图像块的图像位图数据并渲染显示在指定区域上。
可以理解的是，上述图片加载装置10中各个模块的划分仅用于举例说明，在其他的实施例中，可将图片加载装置10按照需要划分为不同的模块，以完成上述图片加载装置10的功能。
本申请实施例中提供的图片加载装置10中的各个模块的实现，可为计算机程序的形式，上述一个或多个模块可以是能够完成特定功能的一系列计算机程序指令段，所述指令段可用于描述图片加载装置10在电子设备100中的执行过程。
参考图5，为本申请实施例提供的电子设备100的硬件结构示意图。如图5所示，电子设备100可以包括处理器1001、存储器1002、通信总线1003、摄像头1004。存储器1002用于存储一个或多个计算机程序1005(比如，图4的图片加载装置10)。一个或多个计算机程序1005被配置为被该处理器1001执行。该一个或多个计算机程序1005可以包括指令，上述指令可以用于实现在电子设备100中执行上述图片加载方法。
可以理解的是，本实施例示意的结构并不构成对电子设备100的具体限定。在另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。
处理器1001可以包括一个或多个处理单元，例如：处理器1001可以包括应用处理器(application processor，AP)，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，DSP，CPU，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
处理器1001还可以设置有存储器，用于存储指令和数据。在一些实施例中，处理器1001中的存储器为高速缓冲存储器。该存储器可以保存处理器1001刚用过或循环使用的指令或数据。如果处理器1001需要再次使用该指令或数据，可从该存储器中直接调用。避免了重复存取，减少了处理器1001的等待时间，因而提高了系统的效率。
在一些实施例中，处理器1001可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，SIM接口，和/或USB接口等。
在一些实施例中，存储器1002可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
本实施例还提供一种计算机存储介质，该计算机存储介质中存储有计算机指令，当该计算机指令在电子设备上运行时，使得电子设备执行上述相关方法步骤实现上述实施例中的图片加载方法。
本实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的图片加载方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的图片加载方法。
其中，本实施例提供的第一电子设备、计算机存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，该模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
该作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
该集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。

Claims (16)

  1. 一种图片加载方法，其特征在于，包括：
    接收图片加载指令；
    若显示界面存在热点显示区域，优先加载所述热点显示区域对应的图片内容；
    其中，所述热点显示区域为用户眼球关注区域。
  2. 如权利要求1所述的图片加载方法，其特征在于，所述方法还包括：
    若侦测到所述显示界面前存在观看用户且所述观看用户的眼球注视位置落入所述显示界面内，确定所述显示界面存在热点显示区域；或
    若侦测到所述显示界面前不存在观看用户、或者所述显示界面前存在观看用户且所述观看用户的眼球注视位置未落入所述显示界面内，确定所述显示界面不存在热点显示区域。
  3. 如权利要求2所述的图片加载方法，其特征在于，所述方法还包括：
    若多个所述观看用户的眼球注视位置落入所述显示界面内，将在预设白名单内的观看用户的眼球关注区域确定为所述热点显示区域；或
    若多个所述观看用户的眼球注视位置落入所述显示界面内，将在预设白名单内的观看用户的眼球关注区域确定为第一优先等级的热点显示区域，及将未在所述预设白名单内的观看用户的眼球关注区域确定为第二优先等级的热点显示区域。
  4. 如权利要求1至3任意一项所述的图片加载方法，其特征在于，所述显示界面包括多张图片，所述优先加载所述热点显示区域对应的图片内容，包括：
    将所述多张图片的解码信息及显示位置信息打包成多个解码任务，其中，所述多个解码任务与所述多张图片一一对应，所述解码信息包括图片存储路径、图片大小及图片采样率中的一种或多种；
    优先选取属于所述热点显示区域的解码任务进行解码处理，并根据解码结果在所述显示界面进行渲染显示。
  5. 如权利要求4所述的图片加载方法，其特征在于，所述方法还包括：
    若图片的显示位置部分落入或者全部落入所述热点显示区域，将所述图片认定为属于所述热点显示区域的图片。
  6. 如权利要求1至3任意一项所述的图片加载方法，其特征在于，所述显示界面包括一张图片，所述优先加载所述热点显示区域对应的图片内容包括：
    将所述图片以预设分辨率在所述显示界面进行渲染显示；
    优先选取属于所述热点显示区域的图片区域进行解码处理，以将所述图片区域以第一分辨率在所述显示界面进行渲染显示；
    其中，所述第一分辨率大于所述预设分辨率。
  7. 如权利要求6所述的图片加载方法，其特征在于，所述优先选取属于所述热点显示区域的图片区域进行解码处理，包括：
    将所述图片按照预设图片数据块划分规则分解为多个区域；
    将所述多个区域打包成多个区域解码任务，并优先选取属于所述热点显示区域的区域解码任务进行解码处理。
  8. 如权利要求6所述的图片加载方法，其特征在于，所述解码处理包括转码处理及颜色 空间转换处理，所述优先选取属于所述热点显示区域的图片区域进行解码处理，包括：
    将所述图片按照预设图片数据块划分规则分解为多个区域；
    将所述多个区域打包成多个区域解码任务；
    对属于所述热点显示区域的区域解码任务进行所述转码处理，及对所述热点显示区域所依赖的其他显示区域的区域解码任务进行所述转码处理，并保存转码结果；优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理。
  9. 如权利要求8所述的图片加载方法，其特征在于，所述解码处理还包括下采样处理，所述优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理，包括：
    优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理与所述下采样处理。
  10. 如权利要求6所述的图片加载方法，其特征在于，所述方法还包括：
    在所述图片上划分关注区域与非关注区域；
    当对所述图片进行编码时，将所述关注区域作为非关注区域的编码依赖。
  11. 如权利要求10所述的图片加载方法，其特征在于，所述解码处理包括转码处理及颜色空间转换处理，所述优先选取属于所述热点显示区域的图片区域进行解码处理，包括：
    将所述图片按照预设图片数据块划分规则分解为多个区域；
    将所述多个区域打包成多个区域解码任务；
    若所述热点显示区域的图片区域不包括非关注区域，优先选取属于所述热点显示区域的区域解码任务进行所述转码处理与所述颜色空间转换处理。
  12. 如权利要求11所述的图片加载方法，其特征在于，所述方法还包括：
    若所述热点显示区域的图片区域包括所述非关注区域，对属于所述热点显示区域的区域解码任务进行所述转码处理，及对所述热点显示区域所依赖的其他显示区域的区域解码任务进行所述转码处理，并保存转码结果；
    优先选取属于所述热点显示区域的区域解码任务进行所述颜色空间转换处理。
  13. 如权利要求1所述的图片加载方法，其特征在于，所述方法还包括：
    若所述显示界面不存在热点显示区域，按照预设加载规则加载所述显示界面的图片内容。
  14. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储计算机指令，当所述计算机指令在电子设备上运行时，使得所述电子设备执行如权利要求1至权利要求13中任一项所述的图片加载方法。
  15. 一种电子设备，其特征在于，所述电子设备包括处理器和存储器，所述存储器用于存储指令，所述处理器用于调用所述存储器中的指令，使得所述电子设备执行权利要求1至权利要求13中任一项所述的图片加载方法。
  16. 一种芯片，与电子设备中的存储器耦合，其特征在于，所述芯片用于控制所述电子设备执行权利要求1至权利要求13中任一项所述的图片加载方法。
PCT/CN2021/134087 2020-11-30 2021-11-29 图片加载方法、电子设备、芯片及可读存储介质 WO2022111697A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011379419.4A CN114579781A (zh) 2020-11-30 2020-11-30 图片加载方法、电子设备、芯片及可读存储介质
CN202011379419.4 2020-11-30

Publications (1)

Publication Number Publication Date
WO2022111697A1 true WO2022111697A1 (zh) 2022-06-02

Family

ID=81754042

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/134087 WO2022111697A1 (zh) 2020-11-30 2021-11-29 图片加载方法、电子设备、芯片及可读存储介质

Country Status (2)

Country Link
CN (1) CN114579781A (zh)
WO (1) WO2022111697A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105224647A (zh) * 2015-09-29 2016-01-06 努比亚技术有限公司 图片显示方法及装置
CN106250503A (zh) * 2016-07-29 2016-12-21 维沃移动通信有限公司 一种图片处理的方法及移动终端
US10157448B2 (en) * 2016-02-12 2018-12-18 Qualcomm Incorporated Foveated video rendering
CN111861657A (zh) * 2020-07-15 2020-10-30 广东纯态数字化信息科技有限公司 显示屏及其画面显示方法和计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105224647A (zh) * 2015-09-29 2016-01-06 努比亚技术有限公司 图片显示方法及装置
US10157448B2 (en) * 2016-02-12 2018-12-18 Qualcomm Incorporated Foveated video rendering
CN106250503A (zh) * 2016-07-29 2016-12-21 维沃移动通信有限公司 一种图片处理的方法及移动终端
CN111861657A (zh) * 2020-07-15 2020-10-30 广东纯态数字化信息科技有限公司 显示屏及其画面显示方法和计算机可读存储介质

Also Published As

Publication number Publication date
CN114579781A (zh) 2022-06-03

Similar Documents

Publication Publication Date Title
US10692274B2 (en) Image processing apparatus and method
WO2021052458A1 (zh) 机器翻译的方法和电子设备
US20220360736A1 (en) Method for frame interpolation and related products
CN116320394A (zh) 用于视频编码和解码的自适应传递函数
CN113126937B (zh) 一种显示终端调整方法及显示终端
EP3534616A1 (en) Method, device and system for playing 360 degree panoramic video
US20230259321A1 (en) Screen projection method and electronic device
KR20230133970A (ko) 사진촬영 방법, 장치 및 전자기기
US20230362328A1 (en) Video frame insertion method and apparatus, and electronic device
CN115484403B (zh) 录像方法和相关装置
US20120281022A1 (en) Electronic apparatus and image display method
US20220172440A1 (en) Extended field of view generation for split-rendering for virtual reality streaming
EP4109879A1 (en) Image color retention method and device
CN112199061B (zh) 一种显示方法和设备
WO2022111697A1 (zh) 图片加载方法、电子设备、芯片及可读存储介质
CN113064728A (zh) 高负载应用图像显示方法、终端及可读存储介质
CN110377773B (zh) 图片处理方法、装置、移动终端以及存储介质
US20240171701A1 (en) Image obtaining method and electronic device
CN111953990A (zh) 编码方法及装置
CN110362188B (zh) 图片处理方法、装置、移动终端以及存储介质
CN108235144B (zh) 播放内容获取方法、装置及计算设备
CN112419134A (zh) 一种图像处理方法及装置
US11962743B2 (en) 3D display system and 3D display method
CN116744106B (zh) 相机应用的控制方法和终端设备
CN113749614B (zh) 皮肤检测方法和设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21897197

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21897197

Country of ref document: EP

Kind code of ref document: A1