TWI838028B - Electronic device and method for loading graphic resource and image processing model - Google Patents

Abstract

An electronic device and a method for loading a graphic resource and an image processing model are provided. The method includes: executing a first task to initialize a graphic resource by using a first thread of a processor; executing a second task to load an image processing model by using a second thread of the processor; determining whether the first task and the second task are complete; obtaining a graphic setting in response to the first task and the second task being completed; determining whether the image processing model matches the graphic setting, and executing the second task again to update the image processing model by using the second thread in response to the image processing model not matching the graphic setting; and generating an output frame according to the graphic resource and the image processing model, and transmitting the output frame via a transceiver.

Description

Electronic device and method for reading graphics resources and image processing models

The present invention relates to an image processing technology, and more particularly to an electronic device and method for reading graphic resources and image processing models.

With the development of machine learning technology, more and more displays are beginning to use machine learning technology to process images to provide users with a diverse viewing experience. For example, a display that supports 3D image playback technology can use machine learning technology to convert 2D images into 3D images in real time. However, the application of machine learning-based image processing technology may consume a large amount of computing resources of the display, thereby reducing the quality of the output image or causing delays in the output image. Therefore, how to save the computing resources required by image processing technology is one of the important topics in this field.

The present invention provides an electronic device and method for reading graphic resources and image processing models, which can minimize the computing resources used for reading graphic resources and image processing models.

The electronic device for reading graphics resources and image processing models of the present invention comprises a processor and a transceiver. The processor is coupled to the transceiver, wherein the processor is configured to: use a first thread to execute a first task to initialize graphics resources; use a second thread to execute a second task to read the image processing model; determine whether the first task and the second task have been completed; in response to the first task and the second task being completed, obtain graphics settings; determine whether the image processing model matches the graphics settings, and in response to the image processing model not matching the graphics settings, use a second thread to execute the second task again to update the image processing model; and generate output frames according to the graphics resources and the image processing model, and transmit the output frames through the transceiver.

In one embodiment of the present invention, the processor is further configured to: determine whether the graphics resource matches the graphics setting; and in response to the graphics resource not matching the graphics setting, execute the first task again using the first thread to update the graphics resource.

In an embodiment of the present invention, the electronic device further comprises a storage medium coupled to the processor, wherein the first task comprises: clearing the previous graphic resources corresponding to the previous frame in the storage medium; and initializing the graphic resources corresponding to the current frame to store the graphic resources in the storage medium.

In one embodiment of the present invention, the electronic device further comprises a cache memory coupled to the processor, wherein the second task comprises: checking whether the cache memory stores an image processing model corresponding to the graphics setting; and in response to the cache memory storing the image processing model corresponding to the graphics setting, reading the image processing model from the cache memory.

In one embodiment of the present invention, the electronic device further comprises a storage medium. The storage medium is coupled to the processor and stores a plurality of image processing models, wherein the second task further comprises: in response to the cache memory not storing the image processing model corresponding to the graphics setting, reading an image processing model from the storage medium and storing the image processing model in the cache memory.

In one embodiment of the present invention, the above-mentioned graphic resources include a three-dimensional grid and texture corresponding to the current frame of the output image.

In one embodiment of the present invention, the processor is further configured to: perform depth prediction on the current frame using an image processing model to generate depth information; update the three-dimensional grid according to the depth information; and map the texture to the updated three-dimensional grid to generate an output frame.

In one embodiment of the present invention, the processor is further configured to: receive a user instruction through a transceiver; and in response to receiving the user instruction, use the first execution thread to execute the first task, and use the second execution thread to execute the second task.

In one embodiment of the present invention, the processor is further configured to start generating an output image including an output frame according to a user instruction.

In one embodiment of the present invention, the processor is further configured to stop generating an output image including an output frame according to a user instruction.

In one embodiment of the present invention, the electronic device further comprises a storage medium coupled to the processor and storing an application for generating an output image, wherein the processor is further configured to: in response to the application being activated, use a first execution thread to execute a first task, and use a second execution thread to execute a second task.

The present invention provides a method for reading graphics resources and image processing models, comprising: using a first execution thread of a processor to execute a first task to initialize graphics resources; using a second execution thread of the processor to execute a second task to read the image processing model; determining whether the first task and the second task have been completed; in response to the first task and the second task being completed, obtaining graphics settings; determining whether the image processing model matches the graphics settings, and in response to the image processing model not matching the graphics settings, using a second execution thread to execute the second task again to update the image processing model; and generating output frames according to the graphics resources and the image processing model, and transmitting the output frames through a transceiver.

Based on the above, the electronic device of the present invention can use multiple threads to read graphics resources and image processing models, and only reset the graphics resources and image processing models when necessary. Accordingly, the present invention can save the consumption of computing resources and avoid the reduction of output image quality or delay of output image.

In order to make the content of the present invention more clearly understood, the following embodiments are specifically cited as examples by which the present invention can be truly implemented. In addition, wherever possible, elements/components/steps with the same reference numerals in the drawings and embodiments represent the same or similar components.

FIG1 is a schematic diagram of an electronic device 100 for reading graphics resources and image processing models according to an embodiment of the present invention. The electronic device 100 may include a processor 110, a storage medium 120, a transceiver 130, and a cache 140.

The processor 110 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations of the above components. The processor 110 may be coupled to the storage medium 120, the transceiver 130, and the cache 140, and access and execute multiple modules and various applications stored in the storage medium 120. The processor 110 may have a multi-thread capability. In this embodiment, the processor 110 may run a thread 111 and a thread 112.

The storage medium 120 or cache memory 140 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or similar components or a combination of the above components, and is used to store multiple modules or various applications that can be executed by the processor 110. In this embodiment, the storage medium 120 can store multiple modules including multiple image processing models 121, a storage area 122 for storing graphic resources, and an application (e.g., image playback software) 123 for generating output images, and its functions will be described later.

The transceiver 130 transmits and receives signals wirelessly or wiredly. The transceiver 130 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like.

FIG. 2 is a flow chart of a method for reading a graphics resource and an image processing model according to an embodiment of the present invention, wherein the method may be implemented by the electronic device 100 shown in FIG. 1 .

In step S201, the processor 110 may activate the application 123. In one embodiment, the processor 110 may receive a user command for activating the application 123 through the transceiver 130. The processor 110 may activate the application 123 in response to receiving the user command. In this embodiment, the processor 110 may receive an input image through the transceiver 130, wherein the input image is, for example, a two-dimensional image. The application 123 may be used to convert the input image into a three-dimensional output image and play the output image.

In step S202, the processor 110 may obtain a graphic setting. The graphic setting may indicate the graphic resources and the image processing model 121 required for use by the application 123. The graphic setting may be a default. For example, the processor 110 may read the default graphic setting from the storage medium 120. The graphic setting may also be determined by the user. For example, the processor 110 may obtain the graphic setting from the user instruction as described in step S201.

In step S203, the processor 110 may use the thread 111 to execute the task 300 for initializing the graphics resources. In step S204, the processor 110 may use the thread 112 to execute the task 400 for reading the image processing model 121. The thread 111 and the thread 112 may run in parallel. That is, the processor 110 may execute step S203 and step S204 at the same time.

Task 300 may include steps S301 and S302. In step S301, thread 111 may clear previous graphics resources corresponding to previous frames of output images played by application 123 in storage area 122. Previous graphics resources may include but are not limited to 3D meshes or textures corresponding to previous frames of output images.

In step S302, the execution thread 111 may initialize the graphics resource corresponding to the current frame of the output image to store the graphics resource in the storage area 122. Specifically, the execution thread 111 may generate a three-dimensional grid or texture for generating the output image according to the current frame of the input image according to the graphics setting and the input image. The execution thread 111 may store the generated three-dimensional grid or texture as the graphics resource of the current frame in the storage area 122. The graphics resource may be used to perform image processing on the current frame of the input image to generate the output frame.

Task 400 is used to speed up the reading rate of the image processing model 121. Task 400 may include steps S401 to S404. In step S401, the execution thread 112 may check whether the image processing model 121 corresponding to the graphics setting is stored in the cache 140. Specifically, the execution thread 112 may determine that the model required to perform image processing on the input image is the image processing model 121 based on the graphics setting and the input image. Then, the execution thread 112 may determine whether the image processing model 121 corresponding to the graphics setting has been read into the cache 140. If the cache memory 140 has stored the image processing model 121 corresponding to the graphics setting, the execution thread 112 may execute step S402. If the cache memory 140 has not stored the image processing model 121 corresponding to the graphics setting, the execution thread 112 may execute step S403.

In step S402, the thread 112 may read the image processing model 121 from the cache 140. The image processing model 121 may be used to perform image processing on the current frame of the input image to generate an output frame.

In step S403 , the thread 112 may select an image processing model 121 corresponding to the graphic setting from a plurality of image processing models 121 stored in the storage medium 120 , and read the selected image processing model 121 .

In step S404, the thread 112 may store the selected image processing model 121 in the cache 140 and read the selected image processing model 121 from the cache 140. If the cache 140 has not been created, the thread 112 may create the cache 140 in the storage medium 120 and store the selected image processing model 121 in the cache 140.

In step S205, the processor 110 may wait to receive a user instruction. In step S206, the processor 110 may receive a user instruction via the transceiver 130, wherein the user instruction is used to instruct the application 123 to play the output image.

After receiving the user instruction, in step S207, the processor 110 can determine whether the task 300 and the task 400 have been completed. If the task 300 and the task 400 have been completed, the process proceeds to step S209. If the task 300 and the task 400 have not been completed, the process proceeds to step S208.

In step S208 , the processor 110 may wait for the thread 111 to complete the task 300 , and wait for the thread 112 to complete the task 400 .

In step S209, the processor 110 may obtain the graphics settings again. The graphics settings described in step S209 may be the same as or different from the graphics settings described in step S202. For example, the processor 110 may adjust the graphics resources and image processing model 121 required for use by the application 123 based on changes in the input image or received instructions, thereby making the graphics settings described in step S209 different from the graphics settings described in step S202.

In step S210, the processor 110 may determine whether it is necessary to reset the image processing model 121 in the cache memory 140. Specifically, the processor 110 may determine whether the image processing model 121 in the cache memory 140 matches the graphics setting of step S209. If the image processing model 121 in the cache memory 140 matches the graphics setting of step S209, the processor 110 may determine that it is not necessary to reset the image processing model in the cache memory 140, and then execute step S212. If the image processing model 121 in the cache memory 140 does not match the graphics setting of step S209, the processor 110 may determine that it is necessary to reset the image processing model 121 in the cache memory 140. Accordingly, the processor 110 may use the thread 112 to execute step S211. In step S211, the thread 112 may execute the task 300 again to update the image processing model 121 in the cache 140.

In step S212, the processor 110 may determine whether it is necessary to reset the graphics resources in the storage area 122. Specifically, the processor 110 may determine whether the graphics resources in the storage area 122 match the graphics settings of step S209. If the graphics resources in the storage area 122 match the graphics settings of step S209, the processor 110 may determine that it is not necessary to reset the graphics resources in the storage area 122, and then execute step S214. If the graphics resources in the storage area 122 do not match the graphics settings of step S209, the processor 110 may determine that it is necessary to reset the graphics resources in the storage area 122. Accordingly, the processor 110 may use the thread 111 to execute step S213. In step S213, the thread 111 may execute the task 300 again to update the graphics resources in the storage area 122.

In step S214, the processor 110 may determine whether the task 300 corresponding to step S211 and the task 400 corresponding to step S213 have been completed. If the task 300 and the task 400 have been completed, the process proceeds to step S215. If the task 300 and the task 400 have not been completed, the process proceeds to step S208. In step S208, the processor 110 may wait for the thread 111 to complete the task 300, and wait for the thread 112 to complete the task 400.

In step S215, the processor 110 may start to generate an output image including an output frame, and may transmit the output frame through the transceiver 130. Specifically, the processor 110 may generate an output frame based on the graphics resources in the storage area 122 and the image processing model 121 in the cache memory 140. In one embodiment, the processor 110 may use the image processing model 121 in the cache memory 140 to perform depth estimation on the current frame of the input image to generate depth information. The processor 110 may update the three-dimensional grid of the current frame according to the depth information, and map the texture to the updated three-dimensional grid to generate an output frame. In other words, the processor 110 may use the image processing model 121 in the cache 140 and the graphics resources in the storage area 122 to convert a two-dimensional input image into a three-dimensional output image.

In step S216, the processor 110 may continuously run the application 123 to generate output images, and transmit the output images through the transceiver 130. For example, the processor 110 may transmit the output images to a display supporting three-dimensional images through the transceiver 130 to play the output images through the display.

In step S217, the processor 110 may receive a user instruction via the transceiver 130, wherein the user instruction is used to instruct the electronic device 100 to stop generating the output image. In step S218, the processor 110 may stop generating the output image according to the user instruction as described in step S217.

FIG. 3 is a flow chart of a method for reading graphics resources and image processing models according to another embodiment of the present invention, wherein the method can be implemented by the electronic device 100 shown in FIG. 1. In step S501, a first task is executed using a first thread of a processor to initialize graphics resources. In step S502, a second task is executed using a second thread of a processor to read an image processing model. In step S503, it is determined whether the first task and the second task have been completed. In step S504, in response to the first task and the second task having been completed, graphics settings are obtained. In step S505, it is determined whether the image processing model matches the graphics setting, and in response to the image processing model not matching the graphics setting, the second task is executed again using the second thread to update the image processing model. In step S506, an output frame is generated according to the graphics resource and the image processing model, and the output frame is transmitted through the transceiver.

In summary, the electronic device of the present invention can utilize multiple threads to read graphics resources and image processing models. The electronic device can determine whether the graphics resources or image processing model need to be reset when a specific event occurs (for example, the application is activated, the user instructs the application to start generating output images, or the user instructs the application to stop generating output images). If the judgment result is no, the electronic device can continue to apply the graphics resources or image processing model in use, thereby achieving the effect of saving computing resources. If the judgment result is yes, the electronic device can utilize multiple threads to reset the graphics resources and image processing model, thereby reducing the time required for the reset. Accordingly, the present invention can save the consumption of computing resources and avoid the reduction of the quality of the output image or avoid the delay of the output image.

100: electronic device 110: processor 111, 112: thread 120: storage medium 121: image processing model 122: storage area 123: application 130: transceiver 140: cache memory 300, 400: task S201, S202, S203, S204, S205, S206, S207, S208, S209, S210, S211, S212, S213, S214, S215, S216, S217, S218, S301, S302, S401, S402, S403, S404, S501, S502, S503, S504, S505, S506: Steps

FIG. 1 is a schematic diagram of an electronic device for reading a graphics resource and an image processing model according to an embodiment of the present invention. FIG. 2 is a flow chart of a method for reading a graphics resource and an image processing model according to an embodiment of the present invention. FIG. 3 is a flow chart of a method for reading a graphics resource and an image processing model according to another embodiment of the present invention.

S501, S502, S503, S504, S505, S506: Steps

Claims (12)

An electronic device for reading graphics resources and image processing models, comprising: a transceiver; and a processor coupled to the transceiver, wherein the processor is configured to: use a first thread to execute a first task to initialize graphics resources; use a second thread to execute a second task to read the image processing model; determine whether the first task and the second task have been completed; in response to the first task and the second task being completed, obtain graphics settings; determine whether the image processing model matches the graphics settings, and in response to the image processing model not matching the graphics settings, use the second thread to execute the second task again to update the image processing model; and generate an output frame according to the graphics resources and the image processing model, and transmit the output frame through the transceiver. An electronic device as described in claim 1, wherein the processor is further configured to: determine whether the graphics resource matches the graphics setting; and in response to the graphics resource not matching the graphics setting, execute the first task again using the first execution thread to update the graphics resource. The electronic device as described in claim 1 further comprises: A storage medium coupled to the processor, wherein the first task comprises: clearing the previous graphics resources corresponding to the previous frame in the storage medium; and initializing the graphics resources corresponding to the current frame to store the graphics resources in the storage medium. The electronic device as described in claim 1 further comprises: a cache memory coupled to the processor, wherein the second task comprises: checking whether the cache memory stores the image processing model corresponding to the graphics setting; and in response to the cache memory storing the image processing model corresponding to the graphics setting, reading the image processing model from the cache memory. The electronic device as described in claim 4 further includes: a storage medium coupled to the processor and storing a plurality of image processing models, wherein the second task further includes: in response to the cache memory not storing the image processing model corresponding to the graphics setting, reading the image processing model from the plurality of image processing models from the storage medium, and storing the image processing model in the cache memory. An electronic device as described in claim 1, wherein the graphics resource includes a three-dimensional grid and texture corresponding to the current frame of the output image. An electronic device as described in claim 6, wherein the processor is further configured to: Perform depth prediction on the current frame using the image processing model to generate depth information; update the three-dimensional grid according to the depth information; and map the texture to the updated three-dimensional grid to generate the output frame. An electronic device as described in claim 1, wherein the processor is further configured to: receive a user instruction through the transceiver; and in response to receiving the user instruction, use the first execution thread to execute the first task, and use the second execution thread to execute the second task. An electronic device as described in claim 8, wherein the processor is further configured to: start generating an output image including the output frame according to the user instruction. An electronic device as described in claim 8, wherein the processor is further configured to: stop generating an output image including the output frame according to the user instruction. The electronic device as claimed in claim 1 further comprises: a storage medium coupled to the processor and storing an application for generating an output image, wherein the processor is further configured to: in response to the application being activated, use the first execution thread to execute the first task, and use the second execution thread to execute the second task. A method for reading graphics resources and image processing models, comprising: Using a first thread of a processor to execute a first task to initialize graphics resources; using a second thread of the processor to execute a second task to read an image processing model; determining whether the first task and the second task have been completed; in response to the first task and the second task being completed, obtaining graphics settings; determining whether the image processing model matches the graphics settings, and in response to the image processing model not matching the graphics settings, using the second thread to execute the second task again to update the image processing model; and generating an output frame according to the graphics resources and the image processing model, and transmitting the output frame through a transceiver.

Images