KR20240045069A - Application processor, system on chip and method of operation thereof - Google Patents

Abstract

An application processor, system-on-chip, and method of operating the same are disclosed. The system-on-chip according to an exemplary embodiment of the present disclosure includes a first processor outputting a first access address, an access address received from the processor, or at least one IP (Intellectual Property) corresponding to a physical address area of the memory. a system bus that transmits the access address to the memory and, if the access address corresponds to a shadow physical address area other than the physical address area of the memory, transfers the access address to a processing circuit other than the memory; and receiving the first access address from the processor through the system bus, converting the first access address to a second access address corresponding to the physical address area, and using the second access address to access the memory. It may include a sub-processing circuit that transmits to the system bus.

Description

Application processor, system on chip and method of operation thereof {Application processor, system on chip and method of operation thereof}

The technical idea of the present disclosure relates to a system-on-chip, and more specifically, to a system-on-chip including a sub-processing circuit supporting an application executed by a processor, an application, and a method of operating the system-on-chip.

System on Chip (hereinafter referred to as SoC) is a technology that integrates a complex system with multiple functions into a single semiconductor chip. With the convergence trend of integrating computers, communications, broadcasting, etc., the demand for application specific semiconductors (Application Specific IC, ASIC) and standard products for specific purposes (Application Specific Standard Product, ASSP) is shifting to SoC. In addition, miniaturization and lightweighting of IT (Information Technology) devices are promoting SoC-related businesses.

As mobile applications evolve, processor and memory usage is increasing. An SoC that can support the use of service software for users while minimizing the increase in processor and memory usage within limited power usage is required.

The technical idea of the present disclosure is to provide a system-on-chip and an application processor that can distinguish and process memory use requests of software.

The system-on-chip according to an exemplary embodiment of the present disclosure includes a first processor outputting a first access address, an access address received from the processor, or at least one IP (Intellectual Property) corresponding to a physical address area of the memory. Then, a system bus that transmits the access address to the memory and, if the access address corresponds to a shadow physical address area other than the physical address area of the memory, transfers the access address to a processing circuit other than the memory, and Receiving the first access address from the processor through a system bus, converting the first access address to a second access address corresponding to the physical address area, and converting the second access address to access the memory It may include sub-processing circuitry for transmission to the system bus.

The application processor according to an exemplary embodiment of the present disclosure includes a first virtual address generated when executing an application, a physical address representing one of a physical address area and a shadow physical address area of a memory, and a virtual memory recognized by the application. A main processor, converting a first physical address to a first physical address using a first page table containing mapping information between virtual addresses representing an address area, and outputting a first access request including the first physical address, from the main processor. Receiving the first access request, and transmitting the first access request to the memory in response to the first physical address corresponding to the physical address area of the memory, and the first physical address corresponding to the shadow address area a router that outputs the first access request to an IP address other than the memory in response to a corresponding IP address; a sub-processing circuit that receives the first access request from the router, processes data related to the first access request, and converts the first physical address to a second virtual address; and a first MMU (Memory Management Unit) that converts the second virtual address into a second physical address corresponding to the physical address area of the memory,

The router receives a second access request including the second physical address from the first MMU, and sends the second access request in response to the second physical address corresponding to the physical address area of the memory. It can be transferred to the memory.

A method of operating a system-on-chip according to an exemplary embodiment of the present disclosure includes transmitting, by a processor, a first access request signal including a first physical address to a router, wherein the router determines that the first physical address is a physical address of a memory. In response to not corresponding to an address area, transmitting the first access request signal to a sub-processing circuit, wherein the sub-processing circuit converts the first physical address to a second access request signal corresponding to the physical address area of the memory. Converting to a physical address, the sub-processing circuit transmitting a second access request signal including the second physical address to the router, and the router transmitting the second access request signal to the memory. may include.

According to the application processor, system-on-chip, and operating method thereof according to the technical idea of the present disclosure, the system bus can distinguish and process memory use requests of software, so that the processor executing the software indirectly accesses the memory through a sub-processing circuit. You can access it. Since the sub-processing circuit can support functions such as compression/decompression, encryption/decryption, and pre-fetch that are not provided by the processor, the system-on-chip can support various functions without modifying the processor or increasing the load. Software can be supported and the bandwidth of memory usage can be reduced. Accordingly, the performance of the system-on-chip can increase and power consumption can be reduced.

1 is a block diagram schematically showing a system on a chip according to an exemplary embodiment of the present disclosure.
FIG. 2A shows a page table according to an exemplary embodiment of the present disclosure, and FIG. 2B shows address areas according to an exemplary embodiment of the present disclosure.
Figure 3 is a block diagram schematically showing a system-on-chip according to an exemplary embodiment of the present disclosure.
4A and 4B show a path through which a processor accesses memory in a system-on-chip according to an exemplary embodiment of the present disclosure.
FIG. 5 is a flowchart illustrating a method for a processor to access memory in a system-on-chip according to an exemplary embodiment of the present disclosure.
Figure 6 shows operation of a system-on-chip according to an example embodiment of the present disclosure.
Figure 7 shows the operation of the system-on-chip 100d according to an exemplary embodiment of the present disclosure.
FIGS. 8A and 8B illustrate a read operation of the system-on-chip 100e according to an exemplary embodiment of the present disclosure.
Figure 9 exemplarily shows compressed image data (CDT) stored in a compression buffer (CBUF).
Figure 10 shows the software layer of system-on-chip 100 according to an example embodiment of the present disclosure.
FIG. 11A shows matching of a virtual address space and an effective physical address space according to a comparative example of the present disclosure, and FIG. 11B shows a memory access method and results according to a comparative example of the present disclosure.
FIG. 12A shows matching of a virtual address space and an effective physical address space according to an exemplary embodiment of the present disclosure, and FIG. 12B illustrates a virtual address space based on an API (Application Programming Interface) according to an exemplary embodiment of the present disclosure. It shows the matching of the and effective physical address space.
Figure 13 shows an example implementation of an application processor 200 according to an example embodiment of the present disclosure.
14A and 14B illustrate read operations and write operations of the application processor 200 according to an exemplary embodiment of the present disclosure.
Figure 15 shows an example of an address matching table according to an exemplary embodiment of the present disclosure.
Figure 16 is a block diagram showing a system-on-chip according to an exemplary embodiment of the present disclosure.
Figure 17 is a block diagram showing an electronic device equipped with an application processor according to an exemplary embodiment of the present disclosure.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. Components described with reference to terms such as unit, module, block, etc. used in the detailed description and functional blocks shown in the drawings are software, hardware, or a combination thereof. It can be implemented in the form By way of example, software may be machine code, firmware, embedded code, and application software. For example, the hardware may include an electrical circuit, an electronic circuit, a processor, a computer, an integrated circuit, integrated circuit cores, a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), a passive component, or a combination thereof. .

1 is a block diagram schematically showing a system on a chip according to an exemplary embodiment of the present disclosure.

The system-on-chip 100 may be mounted on an electronic device, for example, a smart phone, a tablet personal computer (PC), a personal digital assistant (PDA), a portable multimedia player (PMP), or a laptop computer. , wearable devices, GPS (Global Positional system) devices, e-readers, digital broadcasting terminals, MP3 players, digital cameras, wearable computers, navigation, drones, etc. may be mobile devices. For example, the electronic device may be an internet of things (IoT) device, a home appliance, or an Advanced Drivers Assistance System (ADAS). The system-on-chip 100 may be a controller or processor that controls the operation of an electronic device. The system-on-chip 100 may refer to an application processor (AP), a mobile AP, or a control chip.

Cameras, smartphones, wearable devices, Internet of Things (IoT) devices, home appliances, tablet PCs (Personal Computers), PDA (Personal Digital Assistants), PMPs (portable multimedia players), navigation, drones It can be installed in electronic devices such as drones and advanced driver assistance systems (ADAS). Additionally, the image sensor 100 can be mounted on electronic devices that are included as components in vehicles, furniture, manufacturing facilities, doors, and various measuring devices.

Referring to FIG. 1, the system-on-chip 100 may include a processor 110, a sub-processing circuit 120, a memory 130, a plurality of IPs 140 and 150, and a system bus 160. . In addition to this, the system-on-chip 100 may further include a communication function module, an image sensor module, etc. Components of the system on chip 100, such as the sub-processing circuit 120, memory 130, and a plurality of IPs 140 and 150, may transmit and receive data through the system bus 160.

The processor 110 may be the main processor of the system-on-chip 100 and may control the overall operation of the system-on-chip 100. The processor 110 can run an operating system (OS) and various applications (application software) of an electronic device on which the system-on-chip 100 is mounted. Processor 110 may process various types of arithmetic operations and/or logical operations. To this end, the processor 110 may include one processor core (single core) or a plurality of processor cores (Multi-Core). The processor 110 may include cache memories necessary for each of one or more processor cores to perform various operations. Cache memories may temporarily store instructions and/or parameter values required for the processor 110 to execute an application.

The processor 110 may store data in the memory 130 or read data from the memory 130 while executing the operating system and applications. For example, an application may read data from the memory 130, process the read data, and store the processed data back in the memory 130. The processor 110 may read data from the memory 130 and transmit an access request for writing or reading to the memory 130 to write the processed data into the memory 130. The access request may be sent to the memory 130. ) may include a physical address indicating the area where data is stored or to be written among the storage areas.

At this time, the application can read data from a virtually given virtual address space or write data into the virtual address space, and the processor 110 uses a page table (e.g., PGTB in FIG. 2). , Converting a virtual address (or referred to as a logical address) representing a virtual address space into a physical address representing one of a plurality of address areas of the effective physical address space of the memory 130 in which actual data is stored or stored. can do.

In this embodiment, the page table stores a virtual address in the effective physical address space of the memory 130, that is, a physical address representing one of the address areas included in the actual physical address space, and an address area outside the effective physical address space ( For example, it can be mapped to one of the physical addresses representing one of the shadow physical address spaces. Hereinafter, in the present disclosure, the physical address corresponding to the effective physical address space of the memory 130 is referred to as an 'effective physical address', and the physical address corresponding to the shadow physical address space is referred to as a 'shadow physical address'. It will be referred to as ‘(shadow Physical Address)’.

FIG. 2A shows a page table according to an exemplary embodiment of the present disclosure, and FIG. 2B shows address areas according to an exemplary embodiment of the present disclosure.

Referring to FIG. 2A, the page table (PGTB) includes a virtual address (VA) and a physical address (PA). The virtual address (VA) represents address areas of the virtual address space (VAS), and the physical address (PA) represents a plurality of address areas of the effective physical address space and the shadow physical address space. Indicates one address area among multiple address areas of (Shadow Physical Address Space). The virtual address space (VAS) is an address space provided by a virtual memory technique, and is an address space that can be recognized by the operating system and applications in the processor (110 in FIG. 1). The effective physical address space (EPAS) may have the same size as the system's memory, for example, memory 130. Shadow physical address space (SPAS) is an area that exceeds the size of the system's memory.

Referring to FIG. 2B, when the memory 130 has a capacity of 2GB (gigabyte) and the system-on-chip 100 is a 32-bit system, the address area is a 2GB first address area (AR1) with addresses from 0x0000_0000 to 0x7fff_ffff. ), a second address area (AR2) of 1 GB with addresses from 0x8000_0000 to 0xbfff_ffff, and a third address area (AR3) of 1 GB with addresses from 0xc000_0000 to 0xffff_ffff. The first address area (AR1) is set to the effective physical address space (EPAS), and one of the address areas outside the first address area (AR1), for example, the third address area (AR3), is a shadow physical address space. It can be set to (SPAS).

In an embodiment, the virtual address space (VAS) may be divided into a plurality of pages (PN0 to PNn, n is an integer of 3 or more), and each of the plurality of pages (PN0 to PNn) is represented by a virtual address (VA). This is the address area. The size of each of the plurality of pages (PN0 to PNn) may be 4 kilobytes (KB), but is not limited thereto.

The effective physical address space (EPAS) and shadow physical address space (SPAS) can be divided into a plurality of frames (FN0 to FNn), and the physical address (PA) corresponding to the virtual address (VA) is divided into a plurality of frames (FN0). to FNn). Some of the plurality of frames (FN0 to FNn) are address areas of the effective physical address space (EPAS) (hereinafter referred to as effective physical address areas), and other parts are address areas of the shadow physical address space (SPAS) (hereinafter referred to as effective physical address areas). (referred to as a shadow physical address area). For example, frames FN1, FN7, and FN8 shown in Figure 2A may be valid physical address areas, and frames FNn-1 and FNn may be shadow physical address areas. Among the physical addresses (PA), a valid physical address represents one of the valid physical address areas, such as frames FN1, FN7 and FN8, and a shadow physical address represents one of the shadow physical address areas, such as frames FNn-1 and FNn. It can be expressed.

Virtual addresses (VA) of the page table (PGTB) sequentially correspond to a plurality of pages (PN0 to PNn), and physical addresses (PA) mapped to the virtual addresses (VA) correspond to a plurality of frames (FN0 to FNn). ) can be responded to sequentially or non-sequentially. The connection between virtual addresses (VA) and physical addresses (PA) is not permanent and may be disconnected or adjusted depending on various events.

Continuing to refer to FIG. 1, when the processor 110 wants to access the memory 130, the processor 110 uses an effective physical address or a shadow physical address mapped to a virtual address based on the page table (PGTB in FIG. 2A) as an access address. An access request including an access address can be generated and output to the system bus 160. The processor 110 may directly access the memory 130 based on a valid physical address. Additionally, the processor 110 may indirectly access the memory 130 through the sub-processing circuit 120 based on the shadow physical address. This will be described in detail later with reference to FIGS. 4A to 15.

The sub-processing circuit 120 may support functions that the processor 110 does not provide, such as data processing, for data that an application wants to read from the memory 130 or data to be written to the memory 130. The sub-processing circuit 120 may convert the shadow physical address into a valid physical address representing the physical address area of the memory 130 and read data from the memory 130 based on the valid physical address. The sub-processing circuit 120 may process data and transmit the processed data to the processor 110. In addition, the sub-processing circuit 120 processes data requested to be written from the processor 110 along with the shadow physical address, converts the shadow physical address into a valid physical address, and stores data in the memory 130 based on the valid physical address. Processed data can be entered (stored).

In embodiments, sub-processing circuitry 120 may compress or decompress received data. If the data stored in the memory 130 that the application wants to read is compressed data and the processor 110 does not provide a decompression function, the application cannot use the data even if the data is read from the memory 130. does not exist. The sub-processing circuit 120 may read data from the memory 130 based on the shadow physical address, decompress it, and transmit the decompressed data to the processor 130. Alternatively, the sub-processing circuit 120 may compress data provided from the processor 110 along with the shadow physical address, and write (store) the compressed data in the memory 130.

In embodiments, sub-processing circuit 120 may encrypt or decrypt received data. For example, the sub-processing circuit 120 may encrypt data provided from the processor 110, that is, data requested by an application to be written to the memory 130, before storing it in the memory 130. When data is encrypted before being stored in the memory 130, data can be protected from attacks such as cold-boot attacks. Since the encryption and decryption processing of the sub-processing circuit 120 is a process separate from the operation of the processor 110, there is no performance degradation in application execution that may occur due to security functions, and by using dedicated hardware, the processor Power efficiency may be higher than if 110 provides security functions.

In an embodiment, the sub-processing circuit 120 is expected to be accessed by the processor 110 through a separate channel to which the cache coherence protocol is applied between the system bus 160 and the sub-processing circuit 120. Data can be pre-fetched.

The memory 130 may temporarily store data processed or to be processed by the processor 110, the sub-processing circuit 120, and the plurality of IPs 140 and 150. The memory 130 may include volatile memory such as Dynamic Random Access Memory (DRAM), Static RAM (SRAM), Synchronous RAM (SDRAM), and/or Phase-change RAM (PRAM), Magneto-resistive RAM (MRAM), and ReRAM ( It may include non-volatile memory such as Resistive RAM (RAM), Ferroelectric RAM (FRAM), etc. However, for ease of explanation, it is assumed hereinafter that the memory 130 is DRAM. In FIG. 1, the memory 130 is shown as being mounted within the system-on-chip 100, but is not limited thereto. The memory 130 is implemented as a separate chip from the system-on-chip 100, and is not limited to this. Data can be transmitted and received with other components of the system-on-chip 100 through a memory controller provided in 100.

Memory 130 may be system memory. An operating system (OS), applications, and/or firmware are loaded into the memory 130 during booting. For example, when an electronic device equipped with the system-on-chip 100 is booted, the OS image stored in the storage may be loaded into the memory 130 according to the boot sequence. Overall input/output operations of the system-on-chip 100 may be supported by an operating system (OS). Furthermore, applications and/or firmware (eg, related to graphics processing) may be loaded into the memory 130 to select by the user or to provide basic services.

Each of the plurality of IPs 140 and 150 may be a unit module or a combination of unit modules designed to perform a specific function within the system on chip 100. IP may be referred to as a functional module or processing circuit. A plurality of IPs 140 and 150, for example, the first IP 140 and the second IP 150, include a graphics processing unit (GPU), an image signal processor (ISP), a digital signal processor (DSP), and a power management unit (PMU). ), CMU (clock management unit), USB (universal serial bus), PCI (peripheral component interconnect), wireless interface, controller, embedded software, codec, video module ( For example, camera interface, JPEG (Joint Photographic Experts Group) processor, video processor, or mixer, etc.), 3-dimentional graphics core, audio system ), or driver, etc. The plurality of IPs 140 and 150 may be implemented as hardware, software (or firmware), or a combination of hardware and software. In FIG. 1, the processor 110 and the sub-processing circuit 120 are shown as separate configurations from the plurality of IPs 140 and 150, but the processor 110 and the sub-processing circuit 120 are also configured as IPs. can be referred to.

The system bus 160 connects components within the system-on-chip 100, such as the processor 110, the sub-processing circuit 20, the memory 130, and a plurality of IPs 140 and 150. It can provide a transmission path for data or signals between devices.

In an embodiment, the system bus 160 may be implemented in a Network-on-Chip (NoC) manner. The NoC method is a method of connecting processing circuits within a semiconductor chip by applying packet or circuit network technology between general computers or communication devices to the semiconductor chip. The system bus 160 transfers data and signals between processing circuits in a system-on-chip (SoC), that is, the processor 110, the sub-processing circuit 120, the memory 130, and a plurality of IPs 140 and 150. It may include routers and switching circuits to provide a transmission path.

In an embodiment, the system bus 160 may be implemented in the form of a network-on-chip to which a protocol having a predetermined standard bus standard is applied. For example, as a standard bus standard, the Advanced Microcontroller Bus Architecture (AMBA) protocol of Advanced RISC Machine (ARM) may be applied. Bus types of the AMBA protocol may include Advanced High-Performance Bus (AHB), Advanced Peripheral Bus (APB), Advanced eXtensible Interface (AXI), AXI4, and AXI Coherency Extensions (ACE). Among the aforementioned bus types, AXI is an interface protocol between functional blocks and provides multiple outstanding address functions and data interleaving functions. In addition, other types of protocols, such as SONICs Inc.'s uNetwork, IBM's CoreConnect, or OCP-IP's Open Core Protocol, may be applied to the system bus 160.

The system bus 160 receives access requests from one component of the system on chip 100, for example, the processor 110, the sub-processing circuit 120, the first IP 140, and the second IP 150. The access request may be received and transmitted based on the physical address included in the access request, that is, the access address, to a component having the corresponding physical address, for example, the memory 130. Additionally, the system bus 160 may transmit a response to the access request to the configuration that provided the access request.

In the system-on-chip 100 according to the present disclosure, the system bus 160 sends the access request to the memory 130 if the physical address included in the access request received from the processor 110, that is, the access address is a valid physical address. It can be sent to . Accordingly, the processor 110 can directly access the memory 130 based on the valid physical address. In the present disclosure, 'directly accessing' the memory 130 means not going through another processing circuit, and includes access through the system bus 160.

If the physical address is a shadow physical address, the system bus 160 may transmit an access request to the sub-processing circuit 120. As described above, the sub-processing circuit 120 may convert the shadow physical address to a valid physical address. The system bus 160 may receive an access request including a valid physical address from the sub-processing circuit 120 and transmit the access request to the memory 130 . Accordingly, the processor 110 can indirectly access the memory 130 through the sub-processing circuit 120 based on the shadow physical address. In the present disclosure, 'accessing' the memory 130 indirectly means going through a processing circuit, such as the sub-processing circuit 120.

As such, in the system-on-chip 100 according to an exemplary embodiment of the present disclosure, the page table (PGTB in FIG. 2A) includes an effective physical address corresponding to the physical address area, and a shadow physical address space that is an address space other than the physical address area. Contains a shadow physical address corresponding to the address area, and the processor 110 accesses the memory 130 directly through the system bus 160 or indirectly through the system bus 160 and the sub-processing circuit 120. The memory 130 can be accessed. When executing an application, the sub-processing circuit 120 may provide functions that the processor 110 does not provide, such as compression/decompression functions and/or encryption/decryption functions. Accordingly, various applications can be supported without modification or increase in load on the processor 110. Additionally, by providing a compression/decompression function, the bandwidth of memory 130 usage can be reduced, and thus the performance of the system-on-chip 100 can be increased and power consumption can be reduced.

Figure 3 is a block diagram schematically showing a system-on-chip according to an exemplary embodiment of the present disclosure.

Referring to FIG. 3, the system-on-chip 100a may include a processor 110, a sub-processing circuit 120, a plurality of IPs 140 and 150, a system bus 160, and a memory controller 170. there is. FIG. 3 is a modified example of the system-on-chip of FIG. 1. Therefore, overlapping descriptions will be omitted.

In this embodiment, the memory 200 may be implemented as a separate chip outside the system-on-chip 100a. The memory 200 may be a system memory, and various types of memories that can be applied to the memory 130 of FIG. 1 can be applied to the memory 200.

The memory controller 170 may receive an access request including an access address from the system bus 160 and transmit the access request to the memory 200 . Additionally, the memory controller 170 may transmit a processing result or response according to an access request to the system bus 160.

FIGS. 4A and 4B show a path through which a processor accesses memory in the system-on-chip 100b according to an exemplary embodiment of the present disclosure, and FIG. 5 illustrates a path for the system-on-chip 100b according to an exemplary embodiment of the present disclosure. This is a flowchart showing how the processor accesses memory.

Referring to FIGS. 4A and 4B , the system-on-chip 100b may include a processor 110, a sub-processing circuit 120, a memory 130, and a router 161. Router 161 may be included in system bus 160 of FIG. 1 . The system-on-chip 100b may further include other components described with reference to FIG. 1 . In this embodiment, the memory 130 is shown as being provided within the system-on-chip 100b, but is not limited thereto. As explained with reference to FIG. 3, the memory 130 is external to the system-on-chip 100b. It can be implemented as a separate chip. At this time, the system-on-chip 100b may further include a memory controller (170 in FIG. 3) that provides a communication path with the memory 130.

Referring to FIG. 5, the processor 110 may generate a first access request signal including a first access address (S110). Processor 110 may execute an application and operate a page table (e.g., PGTB in FIG. 2A) according to an exemplary embodiment of the present disclosure corresponding to the application. The processor 110 may convert the virtual address (VA in FIG. 2A) issued by the application to access the memory 130 into a physical address (PA in FIG. 2A) using the page table (PGTB in FIG. 2A). . At this time, the physical address (PA) may be a valid physical address or a shadow physical address. The processor 110 may generate a first access request signal including a valid physical address or a shadow physical address as the first access address (AA1).

The processor 110 may transmit a first access request signal (for example, a write request command or a read request command) for the memory 130 including the first access address (AA1) to the system bus (160 in FIG. 1). There is (S120). The first access request signal may be transmitted to the router 161 within the system bus 160.

The router 161 may determine whether the first access address (AA1) corresponds to the shadow physical address area (S130). The router 161 may include information about the shadow physical address corresponding to the shadow physical address area, and the router 161 determines whether the first access address is a shadow physical address based on the information about the shadow physical address. can do.

If it is determined that the first access address (AA1) does not correspond to the shadow physical address area, the router 161 may transmit a first access request signal to the memory (S140). The router 161 may transmit the received access address to the memory 130 or the sub-processing circuit 120.

If the router 161 determines that the first access address (AA1) corresponds to the physical address area (PAS in FIG. 2A), the router 161 transmits the first access address (AA1) to the memory 130, as shown in FIG. 4A. You can. In other words, the router 161 may transmit the first access address (AA1) to the memory 130 if the first access address (AA1) corresponds to a valid physical address.

If the router 161 determines that the first access address (AA1) corresponds to the shadow physical address area (SPAS in FIG. 2A), the router 161 may transmit the first access request signal to the sub-processing circuit 120 (S150). If the router 161 determines that the first access address (AA1) corresponds to the shadow physical address area, in other words, if it determines that the first access address (AA1) corresponds to the shadow address area, the first access address (AA1) is determined as shown in FIG. 4B. 1 The access address (AA1) may be transmitted to the sub-processing circuit 120 rather than the memory 130. For example, if the first access address (AA1) corresponds to a shadow physical address, the router 161 may transmit the first access address (AA1) to the sub-processing circuit 120. The following steps are explained with reference to FIG. 4B.

The sub-processing circuit 120 may convert the first access address (AA1) into the second access address (AA2) corresponding to the physical address area (S160). In other words, the sub-processing circuit 120 can convert the first access address (AA1), which is a shadow physical address, into the second access address (AA2), which is a valid physical address.

The sub-processing circuit 120 may transmit a second access request signal including the second access address (AA2) to the system bus 160 (S170). In other words, the sub-processing circuit 120 may output the second access address to the router 161.

The system bus 160 may transmit a second access request signal to the memory 130 (S180). As described above with respect to step S130, the router 161 determines whether the received access address corresponds to the shadow physical address area, and since the second access address (AA2) is a valid physical address, the shadow physical address It does not correspond to the physical address area. Accordingly, the router 161 may transmit the second access address (AA2) to the memory 130.

As such, in the system-on-chip 100b according to an exemplary embodiment of the present disclosure, the processor 110 directly accesses the memory 130 based on the effective physical address and also accesses the sub-memory 130 based on the shadow physical address. The memory 130 may be indirectly accessed through the processing circuit 120.

Figure 6 shows the operation of the system-on-chip 100c according to an exemplary embodiment of the present disclosure.

Referring to FIG. 6, the system-on-chip 100c may include a processor 110, a sub-processing circuit 120, a memory 130, a router 161, and a memory management unit (MMU) 180. . FIG. 6 is a modified example of the system-on-chip 100b of FIGS. 4A and 4B, so redundant description will be omitted and the description will focus on the differences.

As described above, when the first access address included in the first access request signal generated by the processor 110 is included in the shadow physical address area, that is, when the first access address is a shadow physical address, the router 161 Can transmit a first access request signal to the sub-processing circuit 120.

The sub-processing circuit 120 may convert the first access address into a virtual address. In an embodiment, the sub-processing circuit 120 may include an address matching table containing mapping information for a virtual address (VA) corresponding to a shadow physical address (SPA), and the sub-processing circuit 120 Can convert a shadow physical address (SPA) to a virtual address (VA) using an address matching table. Accordingly, the sub-processing circuit 120 may convert the first access address, which is a shadow physical address (SPA), into a virtual address (VA).

The MMU 180 may convert the virtual address (VA) received from the sub-processing circuit 120 into an effective physical address (EPA). In an embodiment, the MMU 180 may include a page table containing mapping information for an effective physical address (EPA) corresponding to a virtual address (VA). The page table used by the MMU 180 is different from the page table used by the processor 110. For example, the MMU 180 is a system MMU that supports one or more processing circuits of the system-on-chip 100c, and the page table supports at least one other processing circuit (e.g., Graphics Processing Circuit (GPU)) of the system-on-chip 100c. Unit), ISP (Image Signal Processor), etc.) may be the same as the page table used.

The MMU 180 may generate a second access request including an effective physical address (EPA) as a second access address (AA2) and transmit the second access request to the router 161. The router 161 may transmit a second access request to the memory 130.

Figure 7 shows the operation of the system-on-chip 100d according to an exemplary embodiment of the present disclosure.

Referring to FIG. 7 , the system-on-chip 100d may include a processor 110, a sub-processing circuit 120, a memory 130, and a router 161. FIG. 7 is a modified example of the system-on-chip 100b of FIG. 4B, so redundant description will be omitted and the description will focus on the differences.

In this embodiment, the router 161 may include a cache (CC). A separate channel may be included for management of the cache (CC) between the sub-processing circuit 120 and the router 161, and the sub-processing circuit 120 supports the Cache Coherence Protocol. Accordingly, a cache management request signal (eg, a read command or a stash command, etc.) (QRC) for the cache (CC) may be transmitted to the router 161.

Router 161 may operate as a coherency controller. The router 161 may store data in the cache (CC) or output data stored in the cache in response to the cache management request signal (QRC) from the sub-processing circuit 120. The router 161 may also provide a path to read or write data stored in the memory 130 to the cache (CC). The router 161 is configured between the cache (CC) and at least one cache of the processor 110 (e.g., local caches and/or shared caches provided in the processor 110) or at least one cache of the processor 110 ( For example, it may be configured to maintain consistency among local caches and/or shared caches provided in the sub-processing circuit 120.

Meanwhile, in this embodiment, the cache (CC) is shown as being provided inside the router 161, but it is not limited thereto, and the cache (CC) is located within the system bus (160 in FIG. 1) of the router 161. It may be placed separately externally, or may be a shared cache between at least one processor.

The sub-processing circuit 120 may pre-fetch data expected to be used by the processor 110 into the cache (CC) based on the cache management request signal (QRC). The sub-processing circuit 120 may support a type of data pre-fetch that the processor 110 does not support. Accordingly, performance of the processor 110 when executing an application may be improved.

In an embodiment, as described with reference to FIG. 6, the system-on-chip 100d may include the MMU 180. At this time, the sub-processing circuit 120 may directly transmit a cache management request signal (e.g., read command or stash command, etc.) (QRC) to the router 161 and respond to the cache management request signal (QRC). The router 161 may transmit a response (eg, data according to a read request) to the sub-processing circuit 120.

FIGS. 8A and 8B illustrate a read operation of the system-on-chip 100e according to an exemplary embodiment of the present disclosure. Compressed image data is stored in the memory 130 of FIGS. 8A and 8B, and in response to the image data read request from the processor 110, the memory 130 is indirectly accessed through the sub-processing circuit 120 to compress the image data. An embodiment of reading image data will be described.

Referring to FIG. 8A, the processor 110 may execute an application that performs image processing (eg, face recognition and correction, image quality improvement, etc.). The application may request to read image data from the memory 130 and create a first virtual address indicating the area where the image data is stored.

The processor 110 may convert the first virtual address into a first physical address based on the first page table (PGTB1) set for the application. Here, the first page table may map a virtual address to a valid physical address or a shadow physical address, as described with reference to FIGS. 2A and 2B. The first physical address (PA1) may be a shadow physical address.

The memory 130 includes a compression buffer (CBUF), and compressed image data (CDT in FIG. 7B) may be stored in the compression buffer (CBUF).

Figure 9 exemplarily shows compressed image data (CDT) stored in a compression buffer (CBUF).

An image processing circuit (eg, ISP) may compress image data in subblock (SBL) units. For example, the sub-block SBL may include pixels arranged in a 4×4 matrix. One packet of compressed image data (CDT) may include a header (HD) and payload (PL). The payload (PL) may include a plurality of compressed sub-blocks (SBL). The header HD may have information on the storage order of each compressed sub-block (SBL) within the payload (PL) and information on the size of each compressed sub-block (SBL) as a header. Header information may further include the starting address of the payload (PL) (eg, the starting address of the first subblock).

The compression buffer (CBUF) may include a payload area (PLA) and a header area (HAD), and n payloads (PL) (n is a positive integer of 2 or more) are stored in the payload area (PLA). And, n headers (HD) can be stored in the header area (HAD). A footprint (PF) of the same size may be set for each payload (PL). Therefore, the start address (or end address) of the payload (PL) can be known according to the order of the payload (PL).

For example, an ISP may compress original image data received from an image sensor in sub-block units and store the compressed image data (CDT) in a compression buffer (CBUF). The ISP may store the compressed image data (CDT) in a valid physical address generated based on a second page table (PGTB2) established for the ISP, where the effective physical address is the compressed image data (CDT) where the compressed image data (CDT) will be stored. Corresponds to one area of the buffer (CBUF).

Continuing to refer to Figure 8A, processor 110 may not provide compression and decompression functionality for the application. Accordingly, when the processor 110 reads the compressed image data (CDT) directly from the memory 130, the application cannot use the compressed image data (CDT). The processor 110 may decompress compressed image data (CDT) using the sub-processing circuit 120 and receive the decompressed image data.

Therefore, the processor 110 uses the first page table (PGTB1) to convert the first virtual address into a shadow physical address rather than a valid physical address, and accesses the first physical address (PA1) corresponding to the shadow physical address. A first read request signal RD1 including an address may be generated.

Since the first physical address PA1 received from the processor 110 corresponds to the shadow physical address area rather than the physical address area of the memory 130, the router 161 sends the first read request signal RD1 to the memory ( It can be transmitted to the sub-processing circuit 120 rather than 130).

The sub-processing circuit 120 may include an address conversion circuit (ACC) that converts a physical address to a virtual address. In an embodiment, the address conversion circuit (ACC) may convert a physical address into a virtual address using an address matching table. The address matching table may include the virtual address of the payload to the physical address, and the address conversion circuit (ACC) may also provide information about the image data (e.g., height, width, format of the image data, etc.). It can be included. The sub-processing circuit 120 may convert the first physical address PA1 included in the first read request signal RD1 received from the router 161 into the second virtual address VA2. In an embodiment, the address conversion circuit (ACC) uses an address matching table to create a virtual address for each of the header (HD) and payload (PL) of the compressed image data (CDT) corresponding to the first physical address (PA1). The address can be calculated.

Sub-processing circuit 120 may also include a compressor (COMP). The compressor (COMP) can compress or decompress the received data, as will be described later with reference to FIG. 8B.

The MMU 180 may convert the second virtual address VA2 received from the sub-processing circuit 120 into the second physical address PA2 using the second page table PGTB2. The MMU 180 may transmit a second read request signal RD2 including a second physical address PA2 to the router 161. In an embodiment, the second page table (PGTB2) may be a page table set for another processing circuit (e.g., ISP) that compresses image data and stores the compressed image data (CDT) in the memory 130, The second page table (PGTB2) can map effective physical addresses to virtual addresses. The second physical address (PA2) is a valid physical address.

Since the second physical address PA2 corresponds to the physical address area of the memory 130, the router 161 may transmit the second read request signal RD2 to the memory 130.

Referring to FIG. 8B , compressed image data CDT may be read from the second physical address PA2 of the compression buffer CBUF of the memory 130. The router 161 may transmit compressed image data (CDT) to the sub-processing circuit 120. The compressor (COMP) of the sub-processing circuit 120 may decompress the compressed image data (CDT). The sub-processing circuit 120 may transmit uncompressed image data (CDT) to the processor 110 through the router 161.

The decompressed image data (CDT) may be more than the image data that the processor 110 has requested to read (eg, pixel values for some pixels included in the image data of one frame). The sub-processing circuit 120 may transmit image data corresponding to the first physical address PA2 received from the processor 110 among the decompressed image data DCDT to the processor 110 .

In an embodiment, as described with reference to FIG. 7, the sub-processing circuit 120 may include a cache (CC in FIG. 7). The sub-processing circuit 120 responds to the cache management request signal (QRC) received from the sub-processing circuit 120, and selects the remaining image data except for the part transmitted to the processor 110 among the decompressed image data (DCDT). Data can be stored in cache. The processor 110 is likely to request reading of consecutive image data in one frame. Accordingly, when image data that the processor 110 requests to be read later is stored in the cache, the sub-processing circuit 120 does not access the memory 130, but transfers the image data stored in the cache (CC) to the processor ( 110).

Figure 10 shows the software layer of system-on-chip 100 according to an example embodiment of the present disclosure. For convenience of explanation, hardware connected to the system-on-chip 100 is shown together.

The application 1020 and OS 1010 may be executed on a processor (110 in FIG. 1). Application 1020 refers to software (s/w) and services that implement specific functions. User 1030 refers to an object that uses the application 1020. The user 1030 may communicate with the application 1020 through a user interface (UI). The application 1020 is produced based on each service purpose and can communicate with the user 1030 through a user interface suitable for each purpose. The application 1020 performs the operation requested by the user 1030 and, if necessary, can call the contents of the API (Application Protocol Interface, 1016) and the library (Library, 1017).

The API 1016 and/or library 1017 may perform macro operations responsible for a specific function or provide an interface when communication with a lower layer is required. When the application 1020 requests an operation from the lower layer through the API 1016 and/or the library 1017, the API 1016 and/or the library 1017 sends the incoming request to security (Security, 1013), network ( It can be divided into Network, 1014) and Management (1015) fields. The API 1016 and/or library 1017 operates the necessary layers according to the requested field. For example, when the API 1016 requests a function related to the network 1014, the API 1016 can transmit necessary parameters to the network 1014 layer and call the related function. At this time, the network 1014 may communicate with lower layers to perform the requested task. If there is no corresponding lower layer, the API 1016 and/or the library 1017 may perform the corresponding task directly.

Driver 1011 can manage the hardware 1000, check its status, and receive classified requests from upper layers and deliver them to the hardware 1000 layer.

When the driver 1011 requests a task from the hardware 1000 layer, firmware 1012 can convert the request so that the hardware 1000 layer can accept it. The firmware 1012 that converts the request and transmits it to the hardware 1000 may be included in the driver 1011 or may be implemented to be included in the hardware 1000.

The system-on-chip (100 in FIG. 1) may include an API 1016, a driver 1011, and a firmware 1012, and an OS (Operating System, 1010) that manages them all. The OS 1010 may be stored in memory 330 in the form of control command codes and data.

Hardware 1000 may include a processor 1001, sub-processing circuit 1002, memory 1003, system MMU 1004, ISP 1005, GPU 1006, input/output display 1007, etc. . The hardware (1000) performs the requests (or commands) delivered by the driver (1011) and firmware (1012) in order (in-order) or out of order (out-of-order) and returns the performed results to the hardware (1000). ) It can be stored in an internal register or in the memory 300 connected to the hardware 1000. The stored results may be returned to the driver 1011 and firmware 1012.

The hardware 1000 may generate an interrupt and request a necessary operation from a higher layer. When an interrupt occurs, the hardware 1000 checks the interrupt in the management 1015 part of the OS 1010 and then communicates with the core part of the hardware 1000 to process the interrupt.

In this embodiment, the API 1016 may set an environment in which the processor 110 can indirectly access the memory (130 in FIG. 1) through a sub-processing circuit (120 in FIG. 1). The API 1016 creates a page table (e.g., PGTB in FIG. 2A and PGTB1 in FIG. 8A) corresponding to the application 1020 that uses the functionality of the sub-processing circuit 120 and controls the operation of the sub-processing circuit 120. Information such as address matching table, second page table (PGTB2 in FIG. 8A), etc. can be set.

Accordingly, when the application 1020 is executed in the processor 110, a page table corresponding to the application 1020, an address matching table for the sub-processing circuit 120, and a page table may be prepared, as shown in FIGS. 1 to 10. As described with reference to FIG. 8B, the processor 110 may access the memory 130 through the sub-processing circuit 120 based on the shadow physical address.

Meanwhile, in this embodiment, the API 1016 is shown as being included in the OS 1010, but it is not limited thereto. In the embodiment, the API 1016 may be provided by the application 1020.

FIG. 11A shows matching of a virtual address space and an effective physical address space according to a comparative example of the present disclosure, and FIG. 11B shows a memory access method and results according to a comparative example of the present disclosure.

Referring to FIG. 11A, the virtual address space (VAS) includes compressed buffers (comp_buf_o0, comp_buf_o1) as areas for applications. Compressed buffers (comp_buf_o0, comp_buf_o1) may be matched to valid physical address areas (valid_p0, valid_p1) of the effective physical address space (EPAS). The valid physical address areas (valid_p0, valid_p1) may be areas of the compression buffer (CBUF in FIG. 8A) of the memory (130 in FIG. 1).

Referring to FIG. 11B, code 1 (CD1) shows allocating compressed buffers (comp_buf_o0, comp_buf_o1) to the virtual address space (VAS). For example, the compressed buffers (comp_buf_o0, comp_buf_o1) may be a virtual address space (VAS) for the camera process. The first header of the virtual address corresponding to the compressed buffers (comp_buf_o0, comp_buf_o1) may indicate the image size, the first payload may be 0, and the second payload may indicate the size of the payload.

According to code 2 (CD2), when the application requests data from the compressed buffers (comp_buf_o0, comp_buf_o1) based on the solution function foo_sol, the processor (110 in FIG. 1) processes the memory 130 as described with reference to FIG. 4A. By accessing the valid physical address areas (valid_p0, valid_p1), compressed data stored in the valid physical address areas (valid_p0, valid_p1) can be read. Even if data in an empty area created by compression of image data is read or compressed data is read, the processor 110 does not provide a decompression function, so the application cannot interpret the compressed data. Therefore, an error occurs.

FIG. 12A shows matching of a virtual address space and a physical address space according to an exemplary embodiment of the present disclosure, and FIG. 12B illustrates a virtual address space based on an API (Application Programming Interface) according to an exemplary embodiment of the present disclosure. Shows matching of physical address space.

Referring to FIG. 12A, the virtual address space (VAS) may include compressed buffers (comp_buf_o0, comp_buf_o1) and corresponding uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1). Compressed buffers (comp_buf_o0, comp_buf_o1) are matched to valid physical address areas (valid_p0, valid_p1) of the effective physical address space (EPAS), and uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) are matched to the shadow physical address space. It can be matched to the shadow physical address areas (shadow_p0, shadow_p1) of (SPAS). The matching between the virtual address space (VAS) and the effective physical address space (EPAS) and shadow physical address space (SPAS) can be set by the API (1016 in Figure 10), and the matching information is stored in the page table (e.g., in Figure 2A). It can be created as PGTB (PGTB1 in Figure 8a).

Referring to FIG. 12b, compressed buffers (comp_buf_o0, comp_buf_o1) may be allocated to the virtual address space (VAS) according to code 1 (CD1).

Code 3 (CD3) shows applying the compression API to allocate uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) in the virtual address space (VAS). In other words, the compression API may provide an uncompressed view to the processor (110 in FIG. 1). The function c_api of the compression API creates uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) based on compressed buffers (comp_buf_o0, comp_buf_o1) and image information. For example, the uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) are the virtual address space of the operating system (OS), or the processor 110 temporarily uses the virtual address space of the application to access the uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1). Can be set to address space.

According to code 4 (CD4), requesting data for uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) based on the solution function foo_sol, as shown in code 5 (CD5), a function of the compression API for memory, e.g. *memory c_api can map uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) to shadow physical address areas (shadow_p0, shadow_p1) and create a page table. Additionally, a function of the compression API for memory can generate a matching table to be used in the sub-processing circuit (120 in FIG. 1). Here, shadow physical address areas (shadow_p0, shadow_p1) must be allocated consecutively for uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1). In other words, the buffer size (size of each shadow physical address area) added to the starting address representing the starting area shadow_p0 among the shadow physical address areas is the last area (for example, shadow_p1) among the shadow physical address areas. It can be calculated with the ending address indicated.

In this way, the processor 110 decompresses the compressed buffers (comp_buf_o0, comp_buf_o1) from the compression buffer (CBUF) of the memory (130 in FIG. 1) corresponding to the indirectly compressed buffers (comp_buf_o0, comp_buf_o1) through the sub-processing circuit (120 in FIG. 1). The data can be read. Applications can process uncompressed data.

When use of the uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) is completed, the uncompressed buffers (uncomp_buf_v0, uncomp_buf_v1) can be returned to the operating system (OS) according to the free function c_free of code 6 (CD6).

Figure 13 shows an example implementation of an application processor 200 according to an example embodiment of the present disclosure.

Referring to FIG. 13, the application processor (AP) 200 includes a CPU 210, a sub-processing circuit 220, a system interconnection circuit (SICC) 230, a network on chip (NoC) 240, and May include MMU (250). The AP 200 may further include other components, such as an ISP, GPU, communication module, RAM or ROM. The AP 200 may be implemented as a system-on-chip, and data between components within the chip, for example, the CPU 210, the sub-processing circuit 220, the MMU 250, and other processing function blocks, And signals may be transmitted and received through the SICC 230 and NoC 240. SICC 230 and NoC 240 may correspond to router 161 in the above-described embodiments. DRAM may be connected to the AP 200 as a memory of the AP 200. Although not shown, the AP 200 may include a DRAM controller, and the AP 20 may transmit and receive data with the DRAM through the DRAM controller.

The CPU 210 is the main processor of the AP 200 and can control the overall operation of the AP 200. The CPU 210 includes one processor core (Single Core) or a plurality of processor cores ( Multi-Core). CPU 210 may process or execute programs and/or data stored in RAM or ROM.

The CPU 210 may correspond to the processor 110 described with reference to FIG. 1 . The CPU 210 can run an operating system and an application, and can convert a virtual address generated by the application into a physical address. Based on the first page table (PGTB) set for the application, the CPU 210 converts the virtual address into a valid physical address corresponding to the physical address area of the DRAM 300 or a shadow physical address outside the physical address area of the DRAM 300. It can be converted to a shadow physical address corresponding to the address area. The CPU 210 may transmit an access request signal to the SICC 230 for the DRAM 300 including a physical address, that is, a valid physical address or a shadow physical address. The CPU 210 cannot support compression or decompression of data that the application wants to store in the DRAM 300 or data that the application wants to read from the DRAM 300. Accordingly, when an application wants to access the compression buffer (CBUF) of the DRAM 300, the CPU 210 accesses the compression buffer (CBUF) of the DRAM 300 through the sub-processing circuit 220 that supports compression or decompression. ) can be converted to a valid physical address based on the first page table (PGTB) to enable indirect access to the address. Here, the compression buffer (CBUF) can store compressed data (eg, compressed image data).

If the physical address included in the received access request signal is a valid physical address, the SICC 230 transmits the access request to the DRAM 300, and if the physical address is a shadow physical address, the SICC 230 transmits the access request signal to the sub-processing circuit 220. ) can be transmitted. If the physical address included in the access request signal received from the CPU 210 is a valid physical address, the SICC 230 transmits an access request signal to the DRAM 300 so that the CPU 210 can directly access the DRAM 300. If the physical address included in the access request signal is a shadow physical address, the CPU 210 sends the access request signal to the sub-processing circuit 220 to indirectly access the DRAM 300 through the sub-processing circuit 220. Can be transmitted.

In response to the access request signal, the SICC 230 may transmit data read from the DRAM 300 to the corresponding processing circuit (eg, CPU 210, sub-processing circuit 220) that requested access. SICC 230 may include a cache (CC) (e.g., system cache), and may store data read from the memory 300 in the cache (CC) or data from the cache (CC) in response to a cache management request signal. can be read. Additionally, SICC 230 is a cache that maintains data consistency between local caches and/or shared caches of processing circuits, such as CPU 210, sub-processing circuit 220, and other processing circuits. It can operate as a coherency controller.

Sub-processing circuitry 220 may compress or decompress the received data and may also buffer the data. The sub-processing circuit 220 may use an address matching table (AMT) to convert a shadow physical address included in a received access request into a virtual address matching a valid physical address. As described with reference to FIG. 12b, the address matching table (AMT) can be created by the compression API. The sub-processing circuit 220 may also output a cache management request signal provided to the SICC 230 to the SICC 230.

The MMU 250 may convert the virtual address received from the SICC 230 into a valid physical address. The MMU 250 may convert a virtual address into a valid physical address by referring to the second page table (PGTB2). In an embodiment, the second page table PGTB2 may be the same as the page table used by other processing circuits (eg, ISP, GSP, etc.) that access the compression buffer (CBUF) of the DRAM 300.

The NoC 240 may transmit a request signal output from the sub-processing circuit 220 and the MMU 250, for example, an access request signal or a cache management request signal, to the SICC 230, and the NoC 240 may perform sub-processing. It can operate as a data transmission/reception path between the circuit 220 and the SICC 230. NoC 240 may receive an access request including a valid physical address from MMU 250 and transmit the access request to SICC 230. The NoC 240 may transmit compressed data read from the compression buffer (CBUF) of the DRAM 300 to the sub-processing circuit 220 based on an access request that the SICC 230 receives from the MMU 250. Additionally, the NoC 240 may transmit the decompressed data decompressed by the sub-processing circuit 220 to the SICC 230, and the SICC 230 may transmit the decompressed data to the CPU 210.

14A and 14B illustrate read operations and write operations of the application processor 200 according to an exemplary embodiment of the present disclosure. FIG. 14A shows a process in which the CPU 210 of the AP 200 indirectly reads compressed image data from the compression buffer (CBUF) of the DRAM 300, and FIG. 14B shows the process in which the CPU 210 reads the compressed image data from the DRAM 300. It represents the process of indirectly writing image data into the compression buffer (CBUF).

Referring to FIG. 14A, the CPU 210 may output a read request signal including the shadow physical address A to the SICC 230 (operation ①). The read request signal may request to read a portion of image data from the DRAM 300, for example, pixel values corresponding to at least one area of image data of one frame.

The SICC 230 can determine whether the physical address included in the received access request signal is a shadow physical address or a valid physical address, and can transmit a read request signal including the shadow physical address A to the sub-processing circuit 220. There is (action ②).

The sub-processing circuit 220 may convert the shadow physical address A to a virtual address (VA) (operation ③). The sub-processing circuit 220 may convert the shadow physical address A into the virtual address B of the header and the virtual address C of the payload of the packet including the pixels requested to be read among the packets of compressed image data. The sub-processing circuit 220 may refer to an address matching table (e.g., AMT in FIG. 15) to translate the shadow physical address A into the virtual address B of the header and the virtual address C of the payload.

Figure 15 shows an example of an address matching table according to an exemplary embodiment of the present disclosure. It is assumed that the address matching table (AMT) of FIG. 15 is created based on the mapping relationship between the virtual address space (VAS) and the shadow physical address space (SPAS) of FIGS. 11A and 11B.

Referring to FIG. 15, the address matching table (AMT) contains the physical address for the decompressed view, that is, the shadow physical address (SPA _S ), the virtual address (VA _CB ) corresponding to the shadow physical address (SPA _S ) ( It may include the virtual address of the compressed buffer), the virtual address of the header (VA _HD ), and information about the image data (IF_IMG), such as the width, height, and format of the image data. Here, the shadow physical address (SPA _S ) represents the starting address of the shadow physical address area and will hereinafter be referred to as the starting address (SPA _S ). For example, comp_buf_o0 as a virtual address (VA _CB ) to the first shadow physical address shadow_p0, and comp_buf_o0+img0_size as the virtual address (VA _HD ) of the header are mapped, and the virtual address (VA _CB ), comp_buf_o1, and comp_buf_o1+img1_size can be mapped as the virtual address (VA _HD ) of the header. Here, img0_size and img1_size are the sizes of compressed image data stored in comp_buf_o0 and comp_buf_o1, which are virtual addresses (VA _CB ). The height, width and format of the first image data and the second image data (uncompressed or decompressed image data) stored at the first shadow physical address shadow_p0 and the second shadow physical address shadow_p1 are 3840, 2160 and NV12, respectively. You can.

Continuing to refer to FIG. 14A, the sub-processing circuit 220 determines the starting address (SPA _S ) and offset α from the shadow physical address A, and in the address matching table (AMT), the starting address of the shadow physical address A ( You can find the virtual address (VA _CB ), size of image data, and information about image data corresponding to SPA _S ). For example, if shadow physical address A represents one area of the first shadow physical address area (e.g., one subblock of image data), shadow physical address A is the first shadow physical address shadow_p0 plus offset α. It can have a value. At this time, offset α may be smaller than the size of the image data. The sub-processing circuit 220 has comp_buf_o0 as a virtual address (VA _CB ) corresponding to the first shadow physical address shadow_p0 in the address matching table (AMT), img0_size as the size of the image data, and a width of 3840 as information about the image data. , height 2160 and format NV12. The sub-processing circuit 220 may calculate the coordinates (x, y) of the image data according to the read request from the CPU 210 based on the offset α, the format, and the width of the image data. The sub-processing circuit 220 is a payload of a sub-block including pixels corresponding to the coordinates (x, y) of the image data based on comp_buf_o0, which is a virtual address (VA _CB ), and the coordinates (x, y) of the image data. The virtual address B of and the virtual address C of the header can be calculated.

The sub-processing circuit 220 may transmit the virtual address (VA) B of the payload and the virtual address C of the header to the MMU 250 (operation ④).

The MMU 250 may refer to the second page table (PGTB2) to convert the virtual address B of the payload and the virtual address C of the header into the physical address (PA) D of the payload and the physical address E of the header. The physical address D of the payload and the physical address E of the header are valid physical addresses. In an embodiment, the second page table PGTB2 may be a page table referenced by another processing circuit (eg, ISP, GPU, etc.) that processes image data within the AP 200. The MMU 250 may transmit read request signals including the physical address D of the payload and the physical address E of the header to the NoC 240, and the NoC 240 may transmit the read request signals to the SICC 230 ( Action ⑤). Since the read request signals from the NoC 240 include valid physical addresses, the SICC 230 can transmit the read request signals to the DRAM 300.

In the compression buffer (CBUF) of the DRAM 300, the header (HD) and payload (PL) of the compressed image data including pixel values of at least one region of the image data requested by the CPU 210 to be read are stored in the DRAM ( 300) and may be provided to the sub-processing circuit 220 through the SICC 230 and NoC 240 (operation ⑥).

The sub-processing circuit 220 may decompress the compressed image data (operation ⑦). The sub-processing circuit 220 may decompress the compressed image data by decoding the payload (PL) based on the compression information included in the header (HD). The sub-processing circuit 220 may transmit the decompressed image data (DCDT) to the CPU 210 through the NoC 240 and SICC 230 (operation ⑧). Decompressed image data (DCDT) is

In an embodiment, the sub-processing circuit 220 may transmit a cache management request signal (QRC) to the SICC 230 in relation to the decompressed image data (operation ⑨). For example, the decompressed image data (DCDT) may include pixel values of one sub-block including pixel values of at least one region requested by the CPU 210, and the sub-processing circuit 220 may ) A cache management request signal (QRC) requesting to store in the cache (CC) the remaining pixel values, excluding the pixel values provided in ), may be transmitted to the SICC 230. If the CPU 210 continuously requests to read pixel values within one sub-block, a cache hit may occur, and the pixel values already stored in the cache (CC) may be read by the CPU 210 without the need to access the DRAM 300. ) can be transmitted to. Accordingly, the hit ratio of the cache can be increased.

In an embodiment, the sub-processing circuit 220 analyzes read request signals from the CPU 210, determines a specific pattern of image data (e.g., a stripe pattern, etc. within the image data) requesting the read request signals, and , the CPU 210 may transmit a cache management request signal (QRC) to the SICC 230 to pre-fetch image data expected to be read from the DRAM 300.

Referring to FIG. 14b, the CPU 210 may output a write request signal including the shadow physical address A to the SICC 230 (operation ①). The write request signal may request to write (or store) a portion of image data, for example, pixel values corresponding to at least one area of image data of one frame, in the DRAM 300.

SICC 230 may transmit a write request signal including the shadow physical address A to the sub-processing circuit 220 (operation ②). The sub-processing circuit 220 may convert the shadow physical address A to a virtual address (VA) (operation ③). The sub-processing circuit 220 may refer to an address matching table (e.g., AMT in FIG. 15) to translate the shadow physical address A into the virtual address B of the header and the virtual address C of the payload. The sub-processing circuit 220 may transmit a cache management request signal (QRC) to the SICC 230 in relation to the sub-block containing the pixel values requested to be written (operation ④). In an embodiment, the sub-processing circuit 220 may transmit a 'clean invalid' request signal to the SICC 230 for a sub-block containing pixel values requested to be written. SICC 230 flushes dirty lines for caches for CPU 210, such as L1 cache, L2 cache, L3 cache, and LLC (last level cache), and sub-processing circuitry ( 220), for example, only the cache (CC) may have a flushed line.

In an embodiment, unless all pixel values of a sub-block containing write-requested pixel values are in the cache (CC), the sub-processing circuit 220 performs sub-processing according to steps ④ to ⑦ of FIG. 14A. The header (HD) and payload (PL) corresponding to the block are read from the compression buffer (CBUF) of the DRAM 300 (operation ⑥), and the payload (PL) is compressed based on the compression information of the header (HD). It can be canceled (action ⑦). The sub-processing circuit 220 updates the sub-block based on the pixel values requested to be written by the CPU 210, compresses the updated sub-block (operation ⑧), and produces compressed image data (CDT), that is, The header and payload of the subblock can be written back to the compression buffer (CBUF) of the DRAM 300 (operation ⑨).

Figure 16 is a block diagram showing a system-on-chip according to an exemplary embodiment of the present disclosure.

Referring to FIG. 16, the system-on-chip 400 (SoC) includes a CPU 410, random access memory (RAM) 420, multimedia IP 430, memory controller 440, sub-processing circuit 450, It may include a sensor interface 460 and a display controller 470. The system-on-chip 400 may further include other commonly used components such as a communication module, read only memory (ROM), etc. Components of the system-on-chip 400, such as CPU 410, random access memory (RAM) 420, multimedia IP 430, memory controller 440, sub-processing circuit 450, sensor interface ( 460) and the display controller 470 can transmit and receive data through the bus 480. As a standard for the bus 480, the AMBA protocol can be applied, and in addition, protocols such as uNetwork, CoreConnect, and OCP-IP's Open Core Protocol can be applied. In an embodiment, the bus 480 may be implemented in a network-on-chip format.

The CPU 410 may control the overall operation of the system-on-chip 400 and may correspond to the processor (110 in FIG. 1) and/or CPU (210 in FIG. 13) according to the above-described embodiments. The CPU 410 can run an operating system and applications, and can convert virtual addresses generated by applications into physical addresses. At this time, the CPU 410 determines the virtual address based on the first page table set for the application into an effective physical address corresponding to the physical address area of the memory 445 and a shadow address area outside the physical address area of the memory 445. It can be converted to one of the corresponding shadow physical addresses. The CPU 410 may output an access request signal (eg, a read request signal or a write request signal) to the memory 440 including a valid physical address or a shadow physical address. The access request signal may be transmitted to the memory controller 440 through the bus 480 or to the sub-processing circuit 450.

The RAM 420 may be implemented as a volatile memory such as DRAM or SRAM, and the RAM 420 may be implemented as a resistive memory such as PRAM, MRAM, ReRAM, or FRAM. RAM 420 may temporarily store programs, data, and/or instructions.

The multimedia IP 430 may perform image processing on image data, such as still images or moving images. For example, the multimedia IP 430 may include at least one of an ISP, GPU, video processing unit (VPU), display processing unit (DPU), and neural network processing unit (NPU).

The ISP can change the format of the received image data or correct the quality of the image data. For example, an ISP can receive RGB image data as input data and convert it into YUV image data. For example, the ISP can correct the image quality of the image data by performing image processing such as adjusting the gamma value of the received image data, adjusting the luminance, widening the DR (Dynamic Range), or removing noise. there is.

GPUs can compute and generate two-dimensional or three-dimensional graphics. GPU is specialized in processing graphics data and can process graphics data in parallel. Furthermore, GPUs can be used to perform complex operations such as geometric calculations, scalar and vector floating point calculations, etc. GPUs can execute various instructions that are encoded using APIs (Application Programming Interfaces) such as OpenCL, OpenGL, WebGL, etc.

The VPU can correct the quality of received video images or perform video recording and playback, such as camcoding and playback of video images.

The DPU may perform image processing to display received image data on the display device 475. For example, the DPU may change the format of the received image data to a format suitable for displaying on the display, or may correct the image data based on a gamma value corresponding to the display.

The NPU performs image processing on received image data based on a learned neural network, derives multiple features from image data, and identifies objects, backgrounds, etc. included in the image data based on the multiple features. It can be recognized. NPU is specialized in neural network operations and can process image data in parallel.

The memory controller 440 may interface data or commands between the system-on-chip 400 and the memory 445 . The memory controller 440 may transmit the access request signal received from the bus 480 to the memory 445. As described above with reference to FIG. 1, the memory 455 may be implemented as volatile memory such as DRAM, SRAM, or SDRAM, or non-volatile memory such as PRAM, MRAM, ReRAM, FeRAM, or NAND flash. The memory 455 may be implemented as a memory card (eg, MMC, eMMC, SD, micro SD). The memory 455 may include a compression buffer and may store compressed image data.

The multimedia IP 430 may compress image-processed data and store the compressed data in the memory 445. The multimedia IP 430 may include an MMU, and the MMU determines the virtual address where the compressed data will be stored based on the second page table (different from the first page table used by the CPU) to the physical address of the memory 445. It can be converted to a valid physical address corresponding to the area. The multimedia IP 430 may transmit an access request signal to the memory 440 including a valid physical address to the memory controller 440 through the bus 480.

The sub-processing circuit 450 may support functions that the CPU 410 does not provide, such as data processing, for data read from the memory 445 by an application or data to be written to the memory 445. The sub-processing circuit 450 converts the shadow physical address containing the access request transmitted from the CPU 410 into a valid physical address representing the physical address area of the memory 455, and stores the memory 445 based on the effective physical address. ), data can be read from. Accordingly, the CPU 410 can indirectly access the memory 445 through the sub-processing circuit 450.

In an embodiment, the sub-processing circuit 450 converts a shadow physical address into a virtual address using an address matching table created together when creating the first page table, and converts the shadow physical address to a virtual address and the second page set for the multimedia IP 430. You can convert a virtual address into a valid physical address using a table. In an embodiment, the system MMU, which converts a virtual address to a valid physical address using the second page table, may be implemented as a separate circuit from the sub-processing circuit 450. In an embodiment, the MMU of Multimedia IP 430 may be used as the system MMU.

The sub-processing circuit 450 may correspond to the sub-processing circuit (120 in FIG. 1 and 220 in FIG. 13) according to the above-described embodiments. Therefore, detailed description of the sub-processing circuit 450 will be omitted.

The sensor interface 460 may interface data or commands between the system-on-chip 400 and the image sensor 465, and may receive image data from the image sensor 465. Image data received from the image sensor 465 may be image-processed in at least one processing circuit of the multimedia IP 430, and may also be image-processed by an application running on the CPU 410. Image data received from the image sensor 465 or image-processed image data may be stored in the memory 445.

The display controller 470 may interface data (eg, image data) output to the display device 475. The display device 475 may output image or video data through a display such as liquid-crystal display (LCD) or active matrix organic light emitting diodes (AMOLED).

Figure 17 is a block diagram showing an electronic device equipped with an application processor according to an exemplary embodiment of the present disclosure.

Electronic devices (2000) may be mobile devices such as smart phones, tablet PCs, laptop computers, wearable devices, GPS devices, e-readers, MP3 players, digital cameras, navigation, drones, etc., and may be IoT devices, home appliances, ADAS, etc. It may be possible. Additionally, the electronic device 200 may be provided as a component in vehicles, furniture, manufacturing facilities, doors, various measuring devices, etc.

Referring to FIG. 17, the electronic device 2000 includes an application processor (AP) 2100, a camera module 2200, a working memory 2300, a storage 2400, a display device 2500, a communication module 2600, and a user interface 2700. The electronic device 2000 may further include other general-purpose components.

The AP 2100 controls the overall operation of the electronic device 2000 and may be implemented as a system-on-chip (SoC) that runs an application program, operating system, etc. The AP 2100 can image process image data provided from the camera module 2200, provide the image data to the display device 2500, or store the image data in the storage 2400. SoC (100 in FIG. 1, 100a in FIG. 3) and AP (200 in FIG. 13, 300 in FIG. 16) according to the above-described embodiments can be applied as the AP (2100).

In an embodiment, the AP 2100 may include a CPU 2101, a system bus 2102, and a sub-processing circuit 2103. The system bus 2102 determines whether the physical address included in the first access request signal for the working memory 2300 received from the CPU 2101 is a valid physical address or a shadow physical address, and if it is a valid physical address, the first access The request signal can be transmitted to the working memory 2300, and if it is a shadow physical address, the first access request signal can be transmitted to the sub-processing circuit 2103. The sub-processing circuit 2103 may convert the physical address included in the first access request signal into a valid physical address and transmit a second access request signal including the valid physical address to the system bus 2102. The system bus 2102 may transmit the second access request signal from the sub-processing circuit 2013 to the working memory 2300. In an embodiment, the sub-processing circuit 2103 operates the working memory 2300 in response to data received together with the first access request signal of the CPU 2101 or the second access request signal transmitted to the system bus 2102. ) can be compressed or decompressed, or encrypted or decrypted, and data expected to be accessed by the CPU 2101 can be pre-fetched from the working memory 2300. The CPU 2101 can indirectly access the working memory 2300 through the sub-processing circuit 2103, and the sub-processing circuit 2103 performs functions not supported by the CPU 2101, thereby supporting the CPU ( 2101) can support applications running on

The camera module 2200 may generate image data and transmit the image data to the AP (2100). The camera module 2200 may include at least one camera, and the camera may include an image sensor and an objective lens. An image sensor can convert optical signals received through an objective lens into image data. In an embodiment, the camera module 2200 may include a plurality of cameras with different angles of view. In an embodiment, the camera module 2200 may generate a plurality of image data with different exposures and transmit the plurality of image data to the AP 2100. The AP 2100 can generate a high dynamic range (HDR) image by merging a plurality of image data.

The working memory 2300 may be implemented as volatile memory such as DRAM or SRMA, or non-volatile resistive memory such as FeRAM or RRAM PRAM. An operation program or application program stored in the storage 2400 may be loaded into the working memory 2300 and executed. Additionally, data generated during operation of the electronic device 2000 may be temporarily stored in the working memory 2300. The working memory 2300 may store programs and/or data that the application processor 2100 processes or executes. For example, the application processor 2100 may image-process image data provided from the camera module 2200, compress the image-processed image data, and temporarily store the compressed image data in the working memory 2300. .

The storage 2400 may be implemented as a non-volatile memory device such as NADN flash or resistive memory. For example, the storage 2400 may be provided as a memory card (MMC, eMMC, SD, micro SD), etc. The storage 2400 may store data provided from the application processor 2100. For example, the application processor 2100 may store image-processed image data. Additionally, the storage 2400 may store operating programs and application programs of the electronic device 2000.

The wireless transceiver 2600 may include a transceiver 2610, a modem 2620, and an antenna 2630. The wireless transceiver 2600 performs wireless communication with an external device and can receive data from or transmit data to the external device.

The user interface 2700 may be implemented with various devices capable of receiving user input, such as a keyboard, curtain key panel, touch panel, fingerprint sensor, and microphone. The user interface 2700 may receive a user input and provide a signal corresponding to the received user input to the AP 2100.

As above, exemplary embodiments have been disclosed in the drawings and specification. Although embodiments have been described in this specification using specific terms, this is only used for the purpose of explaining the technical idea of the present disclosure and is not used to limit the meaning or scope of the present disclosure as set forth in the claims. . Therefore, those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present disclosure should be determined by the technical spirit of the attached patent claims.

100, 100a, 100b, 100c, 100d, 100e, 400: System-on-Chip
110: processor 120, 450: sub-processing circuit
130, 300: memory 200: application processor

Claims (20)

a first processor outputting a first access address;
If the access address received from the processor or at least one IP (Intellectual Property) corresponds to a physical address area of the memory, the access address is transmitted to the memory, and the access address is a shadow other than the physical address area of the memory. a system bus that transmits the access address to a processing circuit other than the memory if it corresponds to a physical address area; and
Receive the first access address from the processor through the system bus, convert the first access address to a second access address corresponding to the physical address area, and use the second access address to access the memory. A system-on-chip including a sub-processing circuit that transmits to the system bus. The method of claim 1, wherein the first processor:
A first page table includes mapping information between a virtual address, a first physical address, and a second physical address, wherein the first physical address corresponds to the physical address area and the second physical address corresponds to the shadow physical address area. For reference, a system-on-chip characterized in that the first physical address or the second physical address is generated as the first access address based on a virtual address requested from an application. The system-on-chip of claim 1, wherein the first processor executes an application on an operating system (OS). The method of claim 1, further comprising a second processor outputting a third access address for accessing the memory, wherein the third access address corresponds to the first physical address corresponding to the first physical address area. A system-on-chip, characterized in that. According to claim 1,
The system bus is,
Includes a cache for caching data stored in the memory,
The sub-processing circuit,
A system on chip, characterized in that transmitting a management request signal for the cache to the system bus in relation to data stored in the address area of the memory corresponding to the second access address. The method of claim 1, wherein the sub-processing circuit:
A system-on-chip comprising a first functional block that processes data received in response to an access request including the first access address. The method of claim 6, wherein the first functional block is:
A system-on-chip, comprising a compressor that compresses or decompresses the data. The method of claim 6, wherein the first functional block is:
A system-on-chip, comprising an encryptor that encrypts or decrypts the data. The method of claim 6, wherein the sub-processing circuit:
a first address conversion circuit that converts the first access address into a first virtual address, wherein the first virtual address matches a first physical address corresponding to the physical address area; and
A system-on-chip, comprising a second address conversion circuit that converts the first virtual address into the first physical address based on a second page table. The method of claim 9, wherein the second page table is:
A system-on-chip, characterized in that it is the same as the page table used by another processor that performs data processing, stores processed data in the memory, or reads from the memory, to convert a virtual address to a physical address of the memory. . The first virtual address generated when executing the application includes mapping information between a physical address representing one of the physical address area and the shadow physical address area of the memory and a virtual address representing the address area of the virtual memory recognized by the application. a main processor that converts a first physical address using a first page table and outputs a first access request including the first physical address;
Receiving the first access request from the main processor, transmitting the first access request to the memory in response to the first physical address corresponding to the physical address area of the memory, and the first physical address being a router that outputs the first access request to an IP other than the memory in response to one corresponding to the shadow address area;
a sub-processing circuit that receives the first access request from the router, processes data related to the first access request, and converts the first physical address to a second virtual address; and
A first MMU (Memory Management Unit) that converts the second virtual address into a second physical address corresponding to the physical address area of the memory,
The router receives a second access request including the second physical address from the first MMU, and sends the second access request in response to the second physical address corresponding to the physical address area of the memory. An application processor, characterized in that transfer to the memory. According to claim 11,
The application processor, wherein the first sub-processing circuit includes a compressor that compresses or decompresses the data. According to claim 11,
an image signal processor (ISP) that performs image processing on image data, compresses the processed image data, and generates a third virtual address of a virtual memory where the compressed image data will be stored;
The third virtual address is converted to the physical address of the memory using a second page table containing mapping information between a physical address representing the physical address area of the memory and a virtual address representing the address area of the virtual memory recognized by the ISP. The application processor further comprising a second MMU that converts the compressed image data into a third physical address corresponding to a compression buffer in which the compressed image data is to be stored. According to claim 13,
The first MMU refers to the second page table and converts the second virtual address into the second physical address. According to claim 13,
The router is,
Includes a cache for caching data stored in the memory,
The sub-processing circuit,
Characterized in that, transmitting a management request signal for the cache to the router in relation to data stored in an area corresponding to the second physical address of the compression buffer. According to claim 15,
The sub-processing circuit,
Transmitting the management request signal to the router to pre-fetch data expected to be requested by the processor from the memory to the cache based on the pattern of data for which the processor requests access. Characterized by an application processor. A processor transmitting a first access request signal including a first physical address to a router;
transmitting, by the router, the first access request signal to a sub-processing circuit in response to the first physical address not corresponding to a physical address area of a memory;
converting, by the sub-processing circuit, the first physical address into a second physical address corresponding to the physical address area of the memory;
transmitting, by the sub-processing circuit, a second access request signal including the second physical address to the router; and
A method of operating a system-on-chip including the step of the router transmitting the second access request signal to the memory. According to claim 17,
Further comprising the step of converting, by the processor, a first virtual address into the first physical address based on a page table,
The page table includes mapping information between a virtual address, a valid physical address, and a shadow physical address, wherein the effective physical address corresponds to the physical address area and the shadow physical address corresponds to a shadow physical address other than the physical address area of the memory. A method of operating a system-on-chip, characterized in that it corresponds to an address area. According to claim 17,
Decompressing, by the sub-processing circuit, compressed data output from the memory in response to the second access request; and
A method of operating a system on a chip further comprising transmitting at least a portion of the decompressed data by the sub-processing circuit to the processor through the router. According to claim 17,
Decrypting, by the sub-processing circuit, encrypted data output from the memory in response to the second access request; and
A method of operating a system on a chip further comprising transmitting at least a portion of the data decrypted by the sub-processing circuit to the processor through the router.