KR20150096639A - Media workload scheduler - Google Patents

Abstract

Disclosed are a method and system for scheduling a media workload. The method comprises a step of modeling characteristics of the media workload. A GPU utilization rate and a memory bandwidth of the media workload can be determined. Moreover, the media workload can be scheduled by changing the characteristics of the media workload to adjust the GPU utilization rate and the memory bandwidth.

Description

Media Workload Scheduler {MEDIA WORKLOAD SCHEDULER}

The present invention generally relates to controlling workload. More particularly, the present invention relates to adjusting media workloads in a graphics processing unit (GPU).

The media workload is relatively large and can take full advantage of GPU time and memory bandwidth. The media workload includes one or more data streams obtained by a media application from media components of an electronic device, such as a camera or a video player. As used herein, media refers to all data processed by a GPU. For example, tablets and smart phones may have two or more cameras, at least one front camera and at least one rear camera, to support video capture. The cameras can be used simultaneously. Each camera has its own data stream, and includes processing to encode and decode the data stream for each data stream. For example, a computing device may be used for video conferencing with multiple participants, where each participant has its own data stream that is encoded and decoded.

1 is a block diagram of a computing device that may be used to schedule a workload in accordance with the present invention.
2 is a schematic diagram of a system that may be used to schedule media workloads in accordance with the present invention.
3 is a process flow diagram illustrating a method for scheduling media workloads in accordance with the present invention.
4A is a process flow diagram illustrating a method for scheduling media workloads in accordance with the present invention.
4B is a process flow diagram illustrating a method for adjusting features in accordance with the present invention.
5 is a block diagram illustrating a type of non-executing computer-readable medium storing code for scheduling media workloads in accordance with the present invention.
6 is a block diagram of an exemplary embodiment for implementing shared physical memory.
FIG. 7 is a schematic diagram of a small form factor device 700 in which the system of FIG. 6 may be implemented.

Like reference numerals are used to refer to like components and features throughout the specification and drawings. The 100th generation number originally refers to the features shown in FIG. 1, the 200th generation number originally refers to the features shown in FIG. 2, and so on.

GPUs can be used for various media functions such as media playback and 3D applications. Many uses of GPUs include encoding and decoding multiple data streams. As used herein, the media refers to any graphical content composed of various data streams, such as on-line video broadcasts or video conferencing images. As described above, the media workload of the GPU may be relatively large. When the media workload exceeds a predetermined threshold, a bottleneck may occur. The threshold may include the maximum processing capacity of the GPU. The bottleneck can cause the GPU to interrupt data processing, resulting in degraded media functionality. For example, the playback of the video may be corrupted and may include a lag of video playback, or multiple starts and stops during playback. Thus, the embodiments described herein relate to the scheduling of media workloads. Media workloads can be automatically adjusted based on GPU utilization and memory bandwidth.

Video standards, such as standards by the Moving Picture Experts Group (MPEG) and H.264, compress the video stream by processing the video stream frame by frame. A frame is a still image of video similar to a picture. Each frame can be subdivided into macroblocks, which can be further divided into slices. A macroblock is typically a block of pixels. For example, in the H.264 standard, a macroblock is 16x16 pixels in size. A slice is a unit of data in a frame that can be independently encoded or decoded. Thus, even if one slice of the frame is lost, the other slices of the frame are still decodable. In an embodiment, the slices can be used in parallel with encoding and decoding as if they were independent of other slices in the image. The slice header may include address information for at least a portion of the slices in the frame. This address information enables identification of the starting position of the slices. The slice header may include any filter information associated with the slice, as well as other information including the coding scheme of the slice, such as MPEG or H.264.

In the following description and claims, the terms "coupled" and "connected" It should be understood that these terms are not intended to be synonymous with each other. Rather, in certain embodiments, "connected" can be used to indicate that two or more components are in direct physical or electrical contact with each other. "Coupled" may mean that two or more components are in direct physical or electrical contact. However, "coupled" may also mean that two or more components are not in direct contact with each other, but still cooperate or interact with each other.

Some embodiments may be implemented in hardware, firmware, and software alone or in combination. Some embodiments may be implemented as instructions stored on a machine-readable medium that can be read and executed by a computing platform to perform the operations described herein. A machine-readable medium may include any mechanism for storing or transferring information in a machine-readable form, e.g., in a computer-readable form. For example, the machine-readable medium may be a read only memory (ROM), a random access memory (RAM), a magnetic disk storage medium, an optical storage medium, a flash memory device, or an electrical, optical, acoustical, For example, carrier waves, infrared signals, digital signals, or interfaces for transmitting and / or receiving signals.

Embodiments are embodiments or examples. Reference in the specification to "an embodiment", "an embodiment", "some embodiments", "several embodiments", or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiments It is to be understood that the phrase " an embodiment ", " an embodiment ", or "some embodiments" The components or aspects of the embodiments may be combined with the components or aspects of the other embodiments.

It is not necessary that all of the components, features, structures, and features described and illustrated herein are included in a particular embodiment or embodiment. In the specification, for example, a component, feature, structure, or characteristic may be "may", "may", "may", or " quot; could ", it should not be construed that the particular component, feature, structure, or characteristic is included. If a "one" or "one" element is referred to in the specification or claims, this does not mean that there is only one component. Where reference is made to "additional" components in the specification or claims, this does not exclude that there are more than one additional component.

It should be noted that although some embodiments are described with reference to particular implementations, other implementations are possible according to some embodiments. In addition, the arrangement and / or order of the circuit components or other features illustrated in the figures and / or described herein need not be arranged in the particular way illustrated and described. Many other arrangements are possible according to some embodiments.

In each of the systems shown in the figures, elements in some instances may have the same or different reference numerals to indicate that the elements presented may be different and / or similar. However, the components may be flexibly capable of different implementations and may operate with some or all of the systems shown and described herein. The various components shown in the drawings may be the same or different. One is referred to as a first component and the second component is arbitrary.

1 is a block diagram of a computing device 100 that may be used to schedule workloads in accordance with an embodiment of the invention. The computing device 100 may include, for example, a laptop computer, a desktop computer, a tablet computer, a mobile device, a server, or a cellular phone, among others. The computing device 100 includes a central processing unit (CPU) 102 configured to execute stored instructions and a memory device 104 that stores instructions executable by the CPU 102. [ The CPU 102 may be a single-core processor, a multicore processor, a computing cluster, or any number of other configurations. Moreover, the computing device 100 may include more than one CPU 102. [ The instructions executed by the CPU 102 may be used to schedule media workloads.

The computing device 100 may also include a graphics processing unit (GPU) As shown, the CPU 102 may be coupled to the GPU 104 via the bus 106. However, in some embodiments, CPU 102 and GPU 104 are located on the same die. The GPU 104 may be configured to perform even some graphical operation within the computing device 100. For example, the GPU 104 may be configured to be displayed to a user of the computing device 100 by rendering or manipulating a graphical image, graphics frame, video, or the like.

The GPU 104 may also include a sampling engine 108, a 3D engine 110, and a media engine 112. The engine is a component of the GPU 104 that can be used for concurrent processing of data passed to the GPU 104. Although three GPU engines are shown, a GPU can include any number of engines. For example, the GPU 104 may include a rendering engine and a multi-format CODEC (MFX) engine. As shown in computing device 100, 3D engine 110 may process 3D image data in parallel with media engine 112, which processes other types of media. The sampling engine 108 may collect data from the GPU counter 114. Additionally, in an embodiment, the rendering engine may process the 3D image data while the MFX engine decodes the data transmitted to the GPU 104. [ The GPU counter 114 may collect performance data associated with the GPU 104. In particular, the GPU counter 114 may measure the load and throughput of the GPU 104. The sampling engine 108 may send GPU performance data from the GPU counter 114 to the kernel mode driver (KMD 116). The KMD 116 may be configured to receive data from the GPU counter 114 based on data from the GPU counter 114 GPU utilization and memory bandwidth can be calculated.

The computing device 100 includes one or more cameras 118. As described above, the cellular phone may have two front cameras and one rear camera. Multiple cameras can be used to capture video. Additionally, the computing device 100 may include a memory device 120. The memory device 120 may include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory system. For example, the memory device 120 may include a dynamic random access memory (DRAM).

The memory device 120 includes a media feature modeling database 122. The media feature modeling database 122 includes a set of features that can be modified to adjust the GPU utilization and memory bandwidth of the media workload according to an embodiment. The media feature modeling database 122 may be modeled offline or in real time.

The media feature modeling database 122 may include various features. Each feature may change some aspect of the media workload, such as decoding, video processing, encoding, or memory access. In an embodiment, the media workload may be decoded and encoded using a CODEC. Thus, the decoding feature or encoding feature may be changed by changing the corresponding CODEC.

In one example, the decoding aspect of the media workload may be determined by disabling in-loop de-blocking with a slice header or during bi-predicted frames (B-frames) Can be changed by skipping the prediction frame (P-frame). In-loop de-blocking is a technique that, when decoding media playback to improve visual quality and predictive performance of video by smoothing sharp edges that can be created between slices when a block coding technique is used, It is the applied video filter. As described above, a slice is a unit of data in each frame to be encoded or decoded. Each slice may contain information about the slice, such as a header having the encoding or decoding format of the slice, among others. In-loop de-blocking may be disabled by removing the de-blocking filter information contained in the header of each slice. By disabling this feature, a portion of the media workload used for decoding can be reduced.

The portion of the media workload used to decode may also be reduced by skipping B-frames or P-frames while decoding the video of the media workload. As described above, each frame is a still picture that is displayed continuously with other frames to render the entire video. Various frame types may be used to compress the entire video including the predicted picture frame (P-frame) and the bidirectionally predicted frame (B-frame). The P-frame encodes the intra-image variation that occurs in comparison to the previous frame. Therefore, if the video is a ball that recovers across the still background, the motion of the ball is encoded without encoding the still background. So the size of the encoded file is reduced compared to encoding the entire frame including the still background. The B-frame compares the current frame with the previous frame, and encodes the variation between the two frames to encode with the video information. So, when the video is interlaced over the still background, the changed position of the ball is encoded without encoding the still background. When video is encoded using P-frames or B-frames, several P-frames or B-frames may be skipped when decoding video to reduce media workload. When decoding video, several non-sequential P-frames or B-frames may reduce the GPU usage of the media workload and are generally not detectable by the human eye.

Decoding may occur during various uses, including, but not limited to, high resolution playback, WiDi playback, or during video conferencing. Also, in the example, in-loop de-blocking is disabled prior to skipping B- or P-frames during decoding. When in-loop de-blocking is disabled, degradation in video quality is typically very small compared to degradation in video quality that occurs when skipping B- or P-frames during decoding. Thus, it may be desirable to disable in-loop de-blocking primarily because it has a small impact on video quality when compared to the degradation of video quality when skipping B-frames or P-frames during decoding.

One or more video processing features, such as image stabilization, frame rate conversion, gamma correction enhancement, skin tone enhancement, frame rate reduction, and / The frame size can also be changed. For example, by reducing the frame rate or frame size of the data, the GPU utilization can be reduced, thereby making it possible for the GPU to process the data stream without detracting from the media playback as perceived by the user. Although not limited to this, video processing occurs during various uses, including high resolution playback, WiDi playback, or during video conferencing. In the example, when the video processing feature is changed, the frame rate may be disabled before disabling the other video processing feature. This is typically a tedious process within the GPU pipeline. After frame rate conversion is disabled, image stabilization may be disabled to further reduce media workload. Image stabilization can also be a large part of GPU time. Gamma correction enhancement and skin tone enhancement may be disabled after image stabilization is disabled. Finally, in the example, other features such as frame rate reduction and frame size may be disabled. By disabling the video processing features in the order described above, the features can be disabled in a manner that minimizes GPU utilization while minimizing image quality.

Encoding features such as bit rate, motion estimation, and macroblock type may be changed to adjust the portion of the media workload used to encode the video signal. In particular, features such as the bit rate of data processed by encoding can be reduced while motion estimation techniques such as flash media encoding (FME) and hierarchical motion estimation (HME) It can be turned on or turned off in response to high GPU utilization due to encoding. The macroblock type can also be changed. For example, a macroblock may have a different size, such as 16x16 pixels or 8x8 pixels.

In the example, during encoding, the video rate may be lowered before adjusting other features of the media workload. The bit rate can consume a large amount of GPU time, and lowering the bit rate can reduce the GPU time used for encoding. In the example, after the bit rate is lowered, the motion estimation complexity can be lowered, thereby reducing the GPU time used for motion estimation. Finally, in the example, after the motion estimation complexity is lowered, the macroblock shape may be changed to change the media workload. Similar to the video processing features, by disabling the encoding features in the order described above, the features can be disabled in a manner that has minimal impact on image quality while reducing GPU utilization.

Memory bandwidth can also be reduced by changing the characteristics of the media workload. For example, some workloads can use most of the memory, while other workloads use less memory. The workload may be limited by the bandwidth of the memory available when the data is retrieved from the memory. Thus, by reducing the number of execution unit threads in the GPU, the media workload can be reduced, thereby preventing any jitter, lag, or freeze of GPU performance as a result of limited memory bandwidth. Other features that affect memory bandwidth may also be changed. In an example, features affecting the memory bandwidth may be changed in the order in which features using most of the memory bandwidth are disabled prior to the remaining features.

The memory device 120 may also include a workload scheduler 124 that adjusts the workload according to the GPU and memory inputs. In an embodiment, the workload scheduler 124 may communicate with the sampling engine 108 and the media feature modeling database 122 to determine which features of the media workload to change in real time. In this way, GPU utilization and memory bandwidth are achieved that can maximize the use of the GPU without compromising the performance of the GPU recognized by the user. The workload is delivered from the workload scheduler 124 and processed as described in FIG. 2 and then delivered to the GPU 104.

The computing device 100 may include a storage 126. The storage 126 is a physical memory, such as a hard drive, optical drive, thumb drive, drive array, or any combination thereof. The storage 126 may also include a remote storage drive. The storage may also include one or more media applications 128.

The CPU 102 may also be coupled to an input / output (I / O) device 132 configured to connect the computing device 100 to one or more I / O devices 132 via the bus 106. The I / O device 132 may include, for example, a keyboard and a pointing device, and the pointing device may include a touchpad or touch screen, among others. The I / O device 132 may be a built-in component of the computing device 100 or may be a device connected externally to the computing device 100.

The CPU 102 may also be linked to the display interface 134 configured to connect the computing device 100 to the display device 136 via the bus 106. [ The display device 136 may include a display screen that is a built-in component of the computing device 100. Display device 136 may also include a computer monitor, television, or projector connected to computing device 100, among others. The network interface controller (NIC) 138 may be configured to connect the computing device 100 to the network 140 via the bus 106. The network 140 may be a wired network, a wireless network, or a cellular network. The network 140 may be any wide area network (WAN), or some local area network (LAN) or the Internet. For example, the network 140 may be a 3GPP LTE network or a WiFi network.

The block diagram of FIG. 1 is not intended to depict computing device 100 as including all of the components shown in FIG. In addition, the computing device 100 may include any number of additional components not shown in FIG. 1, depending on the specific implementation details.

2 is a schematic diagram of a system 200 that may be used to schedule media workloads in accordance with an embodiment of the present invention. Items with like reference numerals are as described with respect to FIG. The system 200 may include a media application 128. The media application 128 may include, among others, a movie-playing application, video conferencing software, or camera software. The workload created in the media application 128 may be communicated to the scheduling component 200. The scheduling component 200 includes a media feature modeling database 122, a sampling engine 108, a workload scheduler 124, and a comparator 204. As described with respect to FIG. 1, the workload scheduler 124 may include a sampling engine 108 and a media feature modeling database 122 (not shown) to determine which features of the media workload to change in real time to regulate GPU utilization and memory bandwidth. ). ≪ / RTI > If the GPU utilization or memory bandwidth is below the threshold, the comparator 204 will send a message to the workload scheduler 124 that no change in media workload should occur from the application 128. If the GPU utilization or memory bandwidth exceeds the threshold, the comparator 204 will send a message to the workload scheduler 124 that the media workload should be changed. For example, if the GPU utilization exceeds the threshold 95%, the media workload can be changed. Likewise, when the percentage of memory bandwidth is exceeded, bandwidth can be used as a threshold. Thus, when the threshold of 5 gigabytes per second is exceeded, the media workload may change.

The workload may be passed from the scheduling component 200 to the CODEC 206. A CODEC is a software or hardware component of a computing device that is capable of converting media workloads into a format that can be processed by the GPU. The converted workload is passed to the media driver 208. In an embodiment, the media driver 208 places the converted workload in a buffer. The AKMD 116 may be used to calculate the memory bandwidth 212 as well as the GPU utilization 210 of the converted workload. In addition, hardware such as GPU 104 can handle the converted workload from the buffer.

The AKMD 116 also delivers the memory bandwidth 212 as feedback 214 to the scheduling component 200 as well as the computed GPU utilization 210. Feedback 214 is used by comparator 204 to determine whether GPU utilization 210 or memory bandwidth 212 is above or below a certain threshold. As described above, the workload scheduler may change the media workload according to the result of the comparator 204. [

3 is a process flow diagram illustrating a method 300 for scheduling a media workload in accordance with the present invention. In various embodiments, the method 300 may be executed on a computing device, such as the computing device 100. In another embodiment, the method 300 can be performed using a system such as the system 200 described above with respect to FIG.

The method begins by modeling the characteristics of the media workload at block 302. In an embodiment, the feature is an aspect of the workload handled by the GPU or by any of the GPU engines that can be adjusted to prevent or solve the data processing bottleneck of the GPU. Data processing bottlenecks may occur when GPU utilization is above a threshold or when memory bandwidth is above a threshold. Moreover, in an embodiment, the features may be stored in a media feature modeling database.

At block 304, the GPU utilization and memory bandwidth of the media workload can be determined. As described above, GPU utilization and memory bandwidth may be determined using a kernel mode driver. At block 306, the media workload can be scheduled by changing the characteristics of the media workload to adjust GPU utilization and memory bandwidth. In an embodiment, adjusting GPU utilization and memory bandwidth may include providing dynamic control of media workloads, adjusting encoding and decoding of media, adjusting video playback, and adjusting parameters of video conferencing, . ≪ / RTI > Also, in an embodiment, features may be removed from the media workload.

4A is a process flow diagram illustrating a method 400 for scheduling a media workload in accordance with the present invention. At block 402, the media scheduler will schedule the workload according to the default settings of each type of usage model. The usage model represents the type of work performed by the workload. The default settings of the usage model include settings that enable the use of the usage model. For example, video playback is a form of use model. The default settings for video playback operations include, but are not limited to, enabling video processing features, performing image stabilization, and providing frame rate conversion. At block 404, information from the GPU counter is obtained. In an embodiment, after a new workload is sent to the GPU, the scheduling component may collect feedback from the GPU counter so that GPU utilization and memory bandwidth can be calculated. Further, in an embodiment, GPU utilization may be calculated for each GPU engine. At block 406, the method determines if the GPU or GPU engine is overloaded. If a GPU or GPU engine is overloaded, the process flow proceeds to block 408. [ If no GPU or GPU engine is overloaded, process flow continues to block 410.

At block 408, the characteristics of the media workload are changed to reduce GPU utilization. For example, if the 3D engine of the GPU is overloaded such that the utilization of the 3D engine exceeds a predetermined threshold, features that reduce the utilization of the 3D engine are changed. The process flow then returns to block 402 to schedule the workload. Figure 4b further describes a method for adjusting features according to an embodiment.

At block 412, it is determined whether the memory is limited bandwidth. If the memory bandwidth is limited, then the process flow continues to block 414. If the memory bandwidth is not limited, the process flow returns to block 402 to schedule the workload.

At block 414, the features may be modified to reduce memory bandwidth. In an embodiment, if the media engine is overloaded such that the maximum memory bandwidth is exceeded, features that reduce memory bandwidth may be changed. The process flow returns to block 402 to schedule the workload.

4B is a process flow diagram illustrating a method 420 for changing features of a media workload according to an embodiment. At block 422, it is determined if the rendering engine is overloaded. In the example, the rendering engine may be the engine of the GPU rendering the 3D image. If the rendering engine of the GPU is overloaded so that the utilization of the rendering engine exceeds the predetermined threshold, the process flow continues to block 424. [

At block 424, features that reduce the utilization of the rendering engine are modified. At block 426, it is determined whether the MFX engine is overloaded. In the example, the MFX engine is the GPU engine that decodes various data streams. If the MFX engine is overloaded, the process flow proceeds to block 428.

At block 428, the CODEC features that reduce the utilization of the MFX engine are changed. As discussed above, a CODEC is a software or hardware component of a computing device that is capable of transforming a media workload into a format that can be processed by a GPU. In an embodiment, the CODEC may be a CODEC 206 (see FIG. 2). Also, in an embodiment, the CODEC may be a component of the GPU engine, such as the MFX engine or the GPU counter 114 (see FIG. 1). Various aspects of the CODEC can be changed to reduce the utilization of the MFX engine. The process flow returns to block 402 to schedule the workload.

The process flow diagrams of FIGS. 3, 4A and 4B are not intended to indicate that the blocks of method 300 and method 400 are executed in any particular order, or that all of the blocks are included in all cases. Also, any number of additional blocks may be included in methods 300 and 400, depending on the specific implementation details.

5 is a block diagram illustrating one or more types of non-transitory computer readable media 500 that store code for scheduling media workloads in accordance with an embodiment. Thus, one or more types of non-volatile computer readable media may store code for scheduling media workloads according to an embodiment. One or more types of non-volatile computer readable media 500 may be accessed by the processor 502 via the computer bus 504. [ In addition, one or more types of non-volatile computer readable media 500 may include code that directs processor 502 to perform the methods described herein.

The various software components described herein may be stored in one or more types of non-volatile computer readable media 500 as shown in FIG. For example, the modeling module 506 may be configured to model the characteristics of the media workload. Utilization module 508 may be configured to determine GPU utilization and memory bandwidth of the media workload. In addition, the scheduling module 510 can be configured to schedule media workloads by changing the characteristics of the media workload to adjust GPU utilization and memory bandwidth.

The block diagram of FIG. 5 is not meant to imply that one or more types of non-volatile computer-readable media 500 include all of the components shown in FIG. In addition, one or more types of non-volatile computer readable media 500 may include any number of additional components not shown in FIG. 5, depending on the specific implementation details.

6 is a block diagram of an exemplary system 600 for implementing shared physical memory. Items with like reference numerals are as described with reference to Figs. In some embodiments, the system 600 is a media system. The system 600 may also be a personal computer (PC), a laptop computer, an ultra-laptop computer, a tablet, a touchpad, a portable computer, a handheld computer, a palmtop computer, a personal digital assistant (PDA), a cellular telephone, A PDA, a television, a smart device (e.g., smart phone, smart tablet or smart television), a mobile internet device (MID), a messaging device, or a data communication device.

In various embodiments, the system 600 includes a platform 602 coupled to a display 604. Platform 602 may receive content from a content device such as content service device (s) 606 or content delivery device (s) 608, or other similar content sources. The navigation controller 610, which includes one or more navigation features, may be used, for example, to interact with the platform 602 and / or the display 604. These components are described in further detail below.

The platform 602 may be any of a variety of devices including a chipset 612, a central processing unit (CPU) 102, a memory device 120, a storage device 126, a graphics subsystem 614, an application 128, Any combination may be included. The chipset 612 may provide intercommunication between the CPU 102, the memory device 120, the storage device 126, the graphics subsystem 614, the application 128, and the radio 616. For example, the chipset 612 may include a storage adapter (not shown) capable of providing intercommunication with the storage device 126.

The CPU 102 may be a computer system, such as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processor, an x86 instruction set compatible processors, May be implemented as any other microprocessor or central processing unit (CPU). In some embodiments, CPU 102 includes dual core processor (s) or dual core mobile processor (s) and the like.

The memory device 120 may be implemented as a volatile memory device, such as, but not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), or a static RAM (SRAM). The storage device 126 may be a non-volatile storage device, such as a magnetic disk drive, an optical disk drive, a tape drive, an internal storage device, an attached storage device, a flash memory, a battery backup SDRAM ≪ / RTI > and / or network accessible storage devices. In some embodiments, the storage device 126 includes techniques to increase the storage performance improvement protection of expensive digital media, for example, when multiple hard drives are involved.

The graphics subsystem 614 may include processing functionality for images such as still or video for display. Graphics subsystem 614 may include, for example, a graphics processing unit (GPU) such as a GPU 104 (see FIG. 1) or a visual processing unit (VPU). The analog or digital interface may be used to couple the graphics subsystem 614 and the display 604 via communication. For example, the interface may be a high-definition multimedia interface, a display port, wireless HDMI, and / or HD compliant techniques. Graphics subsystem 614 may be integrated into CPU 102 or chipset 612. Alternatively, the graphics subsystem 614 may be a stand-alone card coupled to the chipset 612 via communication.

The graphics and / or video processing techniques described herein may be implemented in various hardware architectures. For example, the graphics and / or video capabilities may be integrated within the chipset 612. [ Alternatively, a discrete graphics and / or video processor may be used. In yet another embodiment, the graphics and / or video functions may be implemented by a general purpose processor including a multicore processor. In yet another embodiment, the functions may be implemented in a consumer electronics device.

The radio 616 may include one or more radios capable of transmitting and receiving signals using various suitable wireless communication technologies. Such techniques may involve communication across one or more wireless networks. Exemplary wireless networks include wireless local area networks (WLANs), wireless personal area networks (WPANs), metropolitan area networks (WMANs), cellular networks, or satellite networks. When communicating throughout such a network, the radio 616 may operate in accordance with one or more application criteria in any version.

Display 604 may include any television type of monitor or display. For example, the display 604 may include a computer display screen, a touch screen display, a video monitor, or a television. Display 604 may be digital and / or analog. In some embodiments, display 604 may be a holographic display. Display 604 may also be a transparent surface that can accept a visual projection. Such a projection can convey various types of information, images, or objects. For example, such a projection may be a visual overlay for a mobile augmented reality (MAR) application. Under the control of one or more applications 128, the platform 602 may display the user interface 618 on the display 604.

The content service device (s) 606 may serve as a host in a national, global, or proprietary service and thus be accessible to the platform 602 via the Internet, for example. Content service device (s) 606 may be coupled to platform 602 and / or display 604. The platform 602 and / or the content service device (s) 606 may be coupled to the network 140 to communicate (e.g., transmit and / or receive) media information to and from the network 140 . Content delivery device (s) 608 may be coupled to platform 602 and / or display 604.

The content service device (s) 606 may include a cable television box, a personal computer, a network, a telephone, or an internet-enabled device capable of conveying digital information. In addition, the content service device (s) 606 may be any other similar entity capable of communicating content either unidirectionally or bi-directionally between the content provider and the platform 602 or display 604 over the network 140, Device. The content may be communicated unidirectionally and / or bi-directionally from any and all of the components and content providers in system 600 via network 140. [ An example of the content may include any media information, including, for example, video, music, medical and game information, and the like.

The content service device (s) 606 may receive content such as cable television programming, including media information, digital information or other content. An example of a content provider may include any cable or satellite television or radio or Internet content provider.

In some embodiments, the platform 602 receives control signals from the navigation controller 610 that include one or more navigation features. The navigation features of the navigation controller 610 may be used, for example, to interact with the user interface 618. The navigation controller 610 may be a pointing device, which may be a computer hardware component (particularly a human interface device) that allows the user to input spatial (e.g., persistent, multidimensional) data into the computer. Many systems, such as graphical user interfaces (GUIs), televisions, and monitors, allow a user to control and provide data to a computer or television using physical gestures. Physical gestures include, but are not limited to facial expressions, facial movements, multiple limb movements, body movements, body language, or some combination thereof. Such physical gestures can be recognized and translated into commands or commands.

The movement of the navigation features of the navigation controller 610 may be echoed on the display 604 by a pointer, cursor, focus ring, or other visual indicator displayed on the display 604. [ For example, under the control of the application 128, the navigation features placed in the navigation controller 610 may be mapped to the virtual navigation features displayed in the user interface 618. In some embodiments, the navigation controller 610 may not be a separate component, but rather may be integrated into the platform 602 and / or the display 604.

For example, when enabled, system 600 may include a driver (not shown) that includes a technique that allows a user to instantly turn on or off platform 602 upon a touch of a button after initial bootup . The program logic may cause the platform 602 to stream the content to the media adapter or other content service device (s) 606 or the content delivery device (s) 608 when the platform is turned "off ". In addition, the chipset 612 may include, for example, hardware and / or software support for 5.1 surround sound audio and / or high definition 7.1 surround sound audio. The driver may include a graphics driver for the integrated graphics platform. In some embodiments, the graphics driver includes a peripheral component interconnect express (PCIe) graphics card.

In various embodiments, any one or more of the components shown in system 600 may be integrated. For example, platform 602 and content service device (s) 606 may be integrated and platform 602 and content delivery device (s) 608 may be integrated, or platform 602, Content service device (s) 606 and content delivery device (s) 608 may be integrated. In some embodiments, platform 602 and display 604 are integrated units. For example, display 604 and content service device (s) 606 may be integrated, or display 604 and content delivery device (s) 608 may be integrated.

The system 600 may be implemented as a wireless system or a wired system. When implemented in a wireless system, the system 600 may include components and interfaces suitable for communicating over a wireless shared media, such as one or more antennas, a transmitter, a receiver, a transceiver, an amplifier, a filter, and control logic, An example of a wireless shared media may include a portion of the radio spectrum, such as the RF spectrum. When implemented as a wired system, the system 600 includes a wired communication media, such as an input / output (I / O) adapter, a physical connector connecting the I / O adapter to a corresponding wired communication medium, a network interface card ), A disk controller, a video controller, or an audio controller or the like. An example of a wired communications medium may include wires, cables, metal leads, printed circuit boards (PCBs), backplanes, switch fabrics, semiconductor materials, twisted-pair wires, coaxial cables,

The platform 602 may establish one or more logical or physical channels to convey information. The information may include media information and control information. The media information may refer to all data representing content created for the user. Examples of content include, for example, data from a voice conversation, video conferencing, streaming video, email (email) messages, voice mail messages, alphanumeric symbols, graphics, images, . The voice conversation data may include, for example, voice information, silence periods, background noise, comfort noise, and tones. The control information may refer to any data representing a command, a command word, or a control word made for an automation system. For example, the control information may be used to route the media information through the system, or it may be used to direct the node to process the media failure in a predetermined manner. However, the embodiment is not limited to the components or situations shown and described in Fig.

FIG. 7 is a schematic diagram of a small form factor device 700 in which the system 600 of FIG. 6 may be implemented. Items with like reference numerals are as described with respect to FIG. In some embodiments, for example, device 700 is implemented as a mobile computing device with wireless capabilities. A mobile computing device may refer to a processing system and any device having a mobile power or power supply, such as, for example, one or more batteries.

As described above, an example of a mobile computing device may be a personal computer (PC), a laptop computer, an ultra-laptop computer, a tablet, a touch pad, a portable computer, a handheld computer, a palmtop computer, a personal digital assistant (E.g., a smart phone, a smart tablet, or a smart television), a mobile Internet device (MID), a messaging device, or a data communication device, and the like.

An example of a mobile computing device may also be a computer configured to be worn by a person, such as a wrist computer, a finger computer, a ring computer, a spectacle computer, a belt-clip computer, an arm-band computer, ≪ / RTI > For example, a mobile computing device may be implemented as a smartphone capable of executing voice communications and / or data communications as well as computer applications. It should be appreciated that although some embodiments may be described for a mobile computing device implemented as an example smartphone, other embodiments may also be implemented using other wireless mobile computing devices.

7, device 700 may include a housing 702, a display 704, an input / output (I / O) device 706, and an antenna 708. The device 700 may also include a navigation feature 710. Display 704 may include all display units suitable for displaying information appropriate to a mobile computing device. The I / O device 706 may include any suitable I / O device suitable for inputting information to the mobile computing device. For example, the I / O device 706 may include an alphanumeric keyboard, a numeric keypad, a touchpad, an input key, a button, a switch, rocker switches, a microphone, a speaker, have. The information may also be input to the device 700 via a microphone. Such information may be digitized by the speech recognition device.

Example 1

A method for scheduling a media workload of a computing device is described herein. The method includes modeling features of the media workload. GPU utilization and memory bandwidth of the media workload can be determined. The method also includes scheduling the media workload by altering features of the media workload to adjust GPU utilization and memory bandwidth.

The features of the media workload can be modeled in the media feature modeling database. Media workloads can be converted to other formats using CODECs. In addition, kernel mode drives, GPUs, and memory counters can be used to determine GPU utilization and memory bandwidth. GPU utilization can be determined for each engine in the GPU. The media workload can be scheduled when the GPU utilization or memory bandwidth exceeds the threshold. Control of GPU utilization and memory bandwidth provides dynamic control of media workloads that control encoding and decoding of media and coordinating video playback.

Example 2

A computing device is described herein. The computing device includes a central processing unit (CPU) configured to execute stored instructions and a storage device that stores instructions. The storage device includes a processor running CODEC configured to model features of the media workload using the media feature modeling database when executed by the CPU. The GPU utilization and memory bandwidth of the media workload may be determined using a graphics processing unit (GPU) counter. The media workload can be scheduled by changing the characteristics of the media workload to adjust GPU utilization and memory bandwidth.

The computing device may include a camera, a video chat application, or a video conferencing application. The processor executable code may be configured to schedule the media workload using a comparator that determines whether the GPU utilization or memory bandwidth is above or below the threshold. Utilization can be determined for each GPU engine of the GPU. The CODEC may also be used to convert media workloads into other formats for processing by the GPU. Moreover, a GPU with a kernel mode drive and a GPU counter can be used to determine GPU utilization and memory bandwidth. Adjusting GPU utilization and memory bandwidth may include removing features from the media workload. The characteristics of the media workload may be modeled in the media feature modeling database of the computing device. The computing device of claim 9 of the claims also includes a radio and a display, wherein the radio and the display can be communicatively coupled to at least the central processing unit.

Example 3

At least one non-transitory machine-readable medium having stored thereon instructions is described herein. In response to being executed on the computing device, the instructions cause the computing device to model the characteristics of the media workload. GPU utilization and memory bandwidth of the media workload can be determined. The media workload can be scheduled by changing the characteristics of the media workload to adjust GPU utilization and memory bandwidth.

The media workload can be scheduled when the GPU utilization or memory bandwidth exceeds the threshold. Moreover, media workloads can be converted to other formats using CODECs. The features of the media workload can be modeled in the media feature modeling database. In addition, the GPU utilization may be determined for each engine of the GPU.

It goes without saying that the details of the foregoing examples may be used anywhere in one or more embodiments. For example, all of the optional features of the computing device described above may also be implemented for the methods or computer-readable media described herein. Moreover, although flow charts and / or state diagrams have been used herein to describe the embodiments, the present invention is not limited to these drawings or the corresponding description. For example, the flow does not need to travel through each illustrated box or state or in exactly the same order as illustrated and described herein.

The invention is not limited to the specific details set forth herein. Indeed, those of ordinary skill in the art having the benefit of this disclosure will recognize that many other modifications from the foregoing description and drawings can be made within the scope of the present invention. Accordingly, the following claims are inclusive of all modifications that define the scope of the invention.

Claims (18)

CLAIMS 1. A method for scheduling media workloads,
Modeling a feature of the media workload by a processor, the feature being a changeable processing aspect of the media workload;
Scheduling, by the processor, the media workload according to a default setting;
Determining, by the processor, a GPU utilization rate and a memory bandwidth of the media workload based on processing of the media workload according to the default setting;
Changing the characteristics of the media workload by the processor;
Scheduling the media workload according to the modified feature to adjust the GPU utilization or the memory bandwidth by the processor
Media workload scheduling method.
The method according to claim 1,
The media workload according to the modified feature is scheduled when the GPU utilization or the memory bandwidth exceeds a threshold
Media workload scheduling method.
The method according to claim 1,
Wherein adjusting the GPU utilization and the memory bandwidth comprises providing dynamic control of the media workload, adjusting encoding and decoding of the media, adjusting video playback, ≪ RTI ID = 0.0 >
Media workload scheduling method.
The method according to claim 1,
Wherein the features of the media workload are modeled in a media feature modeling database and the modeling of the features comprises accessing the media feature modeling database
Media workload scheduling method.
The method according to claim 1,
Wherein determining the GPU utilization and memory bandwidth of the media workload includes determining the GPU utilization and the memory bandwidth using the kernel mode driver, the GPU, and the memory counter by the processor
Media workload scheduling method.
The method according to claim 1,
Wherein the GPU utilization is determined for a first one of a plurality of engines of the GPU, and the features of the media workload include a feature that adjusts GPU utilization of the first engine
Media workload scheduling method.
The method according to claim 1,
Wherein scheduling the media workload comprises converting the media workload by the processor to another format using a CODEC
Media workload scheduling method.
As a computing device,
A central processing unit (CPU) configured to execute the stored instructions, and a storage device for storing the instructions,
The storage device, when executed by the CPU,
Modeling a feature of the media workload using a media feature modeling database, the feature being a changeable processing aspect of the media workload,
You can schedule media workloads by default,
Determining a GPU utilization and memory bandwidth of the media workload according to the default setting using a GPU (Graphic Processing Unit) counter,
Change the characteristics of the media workload,
And processor executable code configured to schedule the media workload according to the modified feature to adjust the GPU utilization or the memory bandwidth
Computing device.
9. The method of claim 8,
Further comprising a comparator to determine whether the GPU utilization is above or below a first threshold or whether the memory bandwidth is above or below a second threshold,
If the GPU utilization exceeds the first threshold or the memory bandwidth exceeds the second threshold, the CPU schedules a media workload with the modified feature
Computing device.
9. The method of claim 8,
Further comprising a GPU having a kernel mode driver and a GPU counter that determine the GPU utilization and memory bandwidth
Computing device.
9. The method of claim 8,
The CPU determines the utilization rate for each GPU engine of the GPU
Computing device.
9. The method of claim 8,
Further comprising a CODEC for converting the media workload to another format suitable for the GPU for processing
Computing device.
9. The method of claim 8,
The computing device may include a camera, a video chat application,
Computing device.
9. The method of claim 8,
A radio and a display, wherein the radio and the display are at least communicatively coupled to the CPU
Computing device.
At least one machine-readable medium having stored thereon instructions,
Wherein the instructions, in response to being executed on a computing device, cause the computing device to:
Modeling a feature of a media workload using a media feature modeling database, the feature being a changeable processing aspect of the media workload,
Scheduling the media workload according to a default setting,
Determine GPU utilization and memory bandwidth of the media workload according to the default setting,
Change the characteristics of the media workload,
And to schedule the media workload according to the modified feature to adjust the GPU utilization or the memory bandwidth
Machine readable medium.
16. The method of claim 15,
The media workload according to the modified feature is scheduled when the GPU utilization exceeds a first threshold or when the memory bandwidth exceeds a second threshold
Machine readable medium.
16. The method of claim 15,
In response to being executed on the computing device, cause the computing device to:
Determine a second GPU utilization and a second memory bandwidth of the media workload according to the modified feature,
Change a second characteristic of the media workload,
And to schedule the media workload according to the modified feature and the modified second feature
More instructions are stored
Machine readable medium.
16. The method of claim 15,
Wherein the GPU utilization is determined for a first one of a plurality of engines of the GPU, and wherein the characteristics of the media workload include a feature that regulates the GPU utilization of the first engine
Machine readable medium.

Images