JP2006085393A - High-speed transfer method for data from device, and data processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To secure a continuous physical memory space on an operating system using a virtual memory method to realize high-speed data transfer technology. <P>SOLUTION: A continuous address space larger than a maximum continuous address space used for a virtual memory space of the operating system is allocated to configure a reference memory space inside the physical memory space 15, and data are transferred by DMA transfer from a device connected by an input/output bus thereto by a device driver 11. The reference memory space is memory-mapped and referred to from an application 16 executed on the operating system. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method for transferring data inside a computer, and more particularly to a technique for acquiring data of devices connected by an input / output bus at high speed in an application.

  As the CPU performance of personal computers, workstations, and the like improves, real-time processing of moving images is performed. At that time, the development of transfer technology for how to use large-capacity data recorded on a device such as a hard disk in an application is an issue.

For example, in a system that acquires data at an ultra-high speed exceeding 1 Gbps, it is difficult to perform a stable operation under the control of an operating system (OS) that is currently used. For this reason, a method using an expansion board with a dedicated memory, hard disk, etc. is known.
In such a system, since it operates separately from the OS, it is impossible to process captured data in real time by an application operating under the OS. It is enough.

  In addition, in order to use data from a device in an application, data transfer between the device, memory, and CPU is complicated in the conventional method, and an excessive load is applied to the CPU. For this reason, there is a problem that a processing speed sufficient for executing the application cannot be secured and the computer resources are not utilized.

Specifically, the following two methods have been used in conventional data transmission. As one of them, as shown in FIG. 5, input data (30) from the outside is input to a data acquisition device (32) in a PC (personal computer) main body (31).
The data acquisition device (32) is provided with a large-capacity data storage memory (33) so that ultrahigh-speed data is temporarily stored in the memory. After the data acquisition is completed, the contents of the data storage memory (33) are transferred from the OS to the memory (35) connected by the internal bus (37). Application execution processing and such transfer processing are performed by the CPU (34). A hard disk (36) and the like are also connected to the internal bus (37).
This method has a problem that the amount of data that can be acquired at one time is limited by the memory size of the data storage memory (33), and the acquired data cannot be accessed from the OS during data acquisition.

In another method, as shown in FIG. 6, input data (40) is input to a data acquisition device (41) and stored in a hard disk (43) connected by a separate dedicated bus (42). It has also been done.
In the PC main body (44), the CPU (45) etc. and the hard disk (47) connected to the internal bus (46) are separated from each other in the system. ) And memory (48).
Although this method relaxes the limit on the amount of data, it cannot be used in an application that performs real-time processing because the acquired data cannot be accessed in real time as described above.

  On the other hand, in systems that do not perform ultra-high-speed data acquisition, systems that realize real-time processing are widely known. The most frequently used is a video capture system, which displays data on a display while acquiring data and simultaneously compresses and decompresses the data.

  In such a system, as shown in FIG. 7, data is directly transferred from the data acquisition device (50) to the memory (52) of the PC main body (51). Usually, the transfer at this time is performed at a high speed by using DMA so as not to cause a load on the CPU (53).

  The DMA transfer is performed with respect to the physical memory (52) having a continuous address space, and the transfer rate itself shows a sufficient value even in the ultra high-speed data transfer. However, this physical memory having a continuous address space is an obstacle to data transfer.

This is because a normal OS employs a virtual memory system, and thus a large continuous physical memory space cannot be secured.
That is, since an OS that supports virtual memory allocates a part of physical memory to the application every time the application is started, the physical memory is increasingly fragmented with time of use. If an application that requires a large continuous area on a physical memory such as DMA transfer is started in a state where the fragmentation has progressed, memory allocation fails.

  As an OS that supports virtual memory, in Linux (registered trademark), the maximum continuous address space when physical memory is divided into small memory spaces is set to 2 Mbytes, and this small memory is used for high-speed data transfer. This is supported by multiple DMA transfers using space.

FIG. 8 shows a general DMA transfer procedure. A device (60) such as a hard disk in the PC main body operates on the CPU and transfers data to the physical memory (62) by a device driver (61) related to device control.
When the DMA transfer is enabled from the device (60), for example, when reading of the previous data is completed, the DMA transfer enable interrupt process (63) is performed on the CPU. Based on this, the device driver (61) issues a DMA transfer command (64), and the device (60) transfers (65) data directly to the physical memory (62) through the internal bus. As described above, the transfer speed at this time is sufficient for high-speed data transfer.

  In this way, the OS attempts to effectively use the memory by dividing the physical memory into small blocks. However, when using a large amount of data, the division of the block becomes an obstacle to transfer. Although FIG. 8 shows a process of performing two DMA transfers to the continuous physical memory (62) (62), a very large number of interrupt processing (63) and transfer are actually performed in high-speed data transfer. Command processing (64) is required.

The OS that controls the DMA transfer needs to issue a transfer command each time. In particular, the load is increased by task switching, and the efficiency is very low in the ultra-high-speed data transfer. Further, the cause of the further significant speed reduction is the data transfer from the physical memory (62) having the continuous address space to the virtual space memory (67) that can be recognized by the normal application (66).
Here, DMA transfer cannot be used, and transfer is performed by the CPU.

  At this time, the OS needs to switch the task of the application (66) and the device driver (61) at high speed, and when other heavy processing such as data processing is performed at the same time, this task switching is performed smoothly. You may lose some data.

In order to cope with such a problem, various high-speed transfer methods from the device to the memory have been proposed. For example, Patent Document 1 discloses a method for transmitting data from the target device without interruption. The bus transfer efficiency is improved by enabling burst output without inserting a wait even for a read request having a burst length exceeding the buffer capacity (number) of the.
However, this method prevents a decrease in transfer efficiency in the device, and is not a technique for avoiding the capacity limitation of the continuous physical memory in the memory of the PC main body as described above.

Japanese Patent Laid-Open No. 2003-132008

  The present invention was created in view of the above-described problems of the prior art, and is a technique that secures a continuous physical memory space and enables high-speed data transfer on an operating system using a virtual memory system. The purpose is to create.

In order to solve the above problems, the present invention provides the following high-speed data transfer method.
That is, the invention according to claim 1 provides a high-speed data transfer method for acquiring data at high speed from a device connected by an input / output bus when referring to data by an application. The method includes a reference memory space allocation step in which the memory space allocation means allocates a continuous address space larger than the maximum continuous address space used for the virtual memory space of the operating system in the physical memory space, DMA (Direct Memory Access) A DMA transfer step in which a transfer means transfers data by DMA transfer from a device connected to the reference memory space by an input / output bus; and a memory mapping means performs memory mapping of the reference memory space on an operating system. The memory mapping step for enabling reference from the application to be executed, and the application execution means include each step of a data reference step for referring to the reference memory space.

  The invention according to claim 2 is characterized in that the reference memory space allocation step is performed before the operating system is started to secure a large continuous address space in advance.

  The invention according to claim 3 is characterized in that, in the reference memory space allocation step according to claim 1 or 2, the memory capacity used in the operating system is set to be smaller than the actual installed memory capacity. Is.

  The invention according to claim 4 is characterized in that the memory capacity of the continuous address space is 2 Mbytes or more.

The present invention can also be provided as the following data processing apparatus.
That is, the invention according to claim 5 is a data processing apparatus capable of high-speed data transfer from a device connected by an input / output bus when referring to data by an application.
The apparatus includes a device and a physical memory, reference memory space allocating means for allocating a continuous address space larger than a maximum continuous address space used for a virtual memory space of an operating system in the physical memory space, and the reference memory space DMA transfer means for transferring data by DMA (Direct Memory Access) transfer from a device connected by an I / O bus, and the reference memory space can be referenced from an application executed on the operating system by memory mapping And a memory mapping means for executing the application while referring to the reference memory space.

  According to the sixth aspect of the present invention, it is possible to provide a configuration in which a reference memory space allocation step is performed before the operating system is started to secure a large continuous address space in advance.

  The invention described in claim 7 is characterized in that the reference memory space allocation means in the invention described in claim 5 or 6 sets the memory capacity used in the operating system to be smaller than the actual installed memory capacity. .

  The invention described in claim 8 is characterized in that the memory capacity of the continuous address space is 2 Mbytes or more.

By providing the above means, the present invention has the following effects.
That is, according to the method and apparatus of claim 1 or 5, a large amount of data is transferred at a high speed to a continuous address space larger than the maximum continuous address space used for the virtual memory space of the OS, and by memory mapping, Since direct reference from an application is possible, an ultra-high-speed data transfer technology that contributes to real-time processing of high-definition moving images and measurement data is provided.

  According to the method and apparatus of the second or sixth aspect, since a large continuous address space is secured before the operating system is started, processing can be performed without being affected by memory fragmentation by the operating system.

  According to the method and apparatus of claim 3 or 7, as a method for securing a large continuous address space, the physical memory is forcibly made unused by reducing the memory capacity used in the operating system. Can do.

  According to the method and apparatus of the fourth or eighth aspect, by using a continuous memory space of 2 Mbytes or more, sufficient high-speed transfer that cannot be transferred by the current operating system can be realized.

Hereinafter, embodiments of the present invention will be described based on examples shown in the drawings. The embodiment is not limited to the following.
FIG. 1 is an explanatory diagram showing a basic configuration of a data transfer method according to the present invention. This configuration allows high-speed data transfer between a device connected by a known bus and a memory in the PC main body, and enables reference from an application.

A PCI (Peripheral Component Interconnect) bus is widely used as the bus. Since the system used in the present invention requires a transfer rate of 2048 Mbps or higher, 64bit / 66 MHz is used as the PCI bus.
A device (10), a device driver (11), a CPU that executes an application (16), and a physical memory (15) are connected by a PCI bus.

In the present invention, first, a large continuous memory space is secured in the physical memory (15).
Here, the virtual memory method adopted by the operating system will be briefly described. In the virtual memory method, the contents of memory pages that are used infrequently are saved to the disk and loaded from the disk to the memory when necessary, so that even a large program exceeding the physical memory size can be executed. become.

In a 32-bit system, memory space up to 4 Gbytes can be accessed as one segment, and the first 3 Gbytes are allocated as process space.
A dynamically generated process space between addresses 0x0 to 0xC0000000 is generated as an independent multiple virtual space for each process being executed.
In the virtual memory method, even if the page is valid in the virtual space, a page on the physical memory (physical page) is not always assigned to the page, and a method of mapping physical memory is used. ing.

Each process executed on the CPU is given an independent process space. One process space is managed by one memory management structure (represented by an mm_struct structure in Linux (registered trademark)).
The process space in an effective state is composed of a plurality of non-contiguous areas, and is managed as a continuous virtual space by a virtual space management structure (same as the vm_area_struct structure).

  The effective space that can be used by one process is the sum of multiple virtual space management structures. At the same time, the memory management structure uses a table (PGD, PTE area) that converts the virtual space addresses of processes into physical memory addresses. The virtual memory space and the physical memory space are converted and referred to by management. The conversion process is performed by, for example, a CPU MMU (Memory Management Unit).

As described above, the MMU manages the physical memory by dividing it into blocks called pages or segments. By mapping processing assigned to the virtual space address, a memory space larger than the actual memory capacity can be provided to the application, and swapped to secondary storage means such as a hard disk as necessary.
At the same time, by separating the virtual space addresses of each application and OS, it contributes to improving security and protecting the memory.

By the way, in the execution of an application, when processing such as continuously transferring a large amount of data such as a high-definition image from the outside, the above-mentioned virtual memory method divides it into pages or segments. It cannot be secured.
In addition, since the virtual memory space and the physical memory space are converted, a large load is generated on the CPU, and the processing speed is greatly reduced.

In order to solve this problem, it is proposed in the present invention to take a configuration as shown in FIG. 2 and improve the transfer control method in the CPU.
FIG. 2 shows an image processing apparatus (20) as an example of a data processing apparatus according to the present invention, which is constituted by a general personal computer or the like. A video image from a video camera, a video player or the like (not shown) is AD-converted as necessary, and is input to the CPU (22) by the video capture interface (21).

The interface (21), CPU (22), memory (23), and hard disk (24) are connected to each other by a PCI bus (25).
In the CPU (22), an MMU unit (26) arranged exclusively as hardware, a DMA transfer control unit (27) that functions by OS processing, and an application execution unit (28) that executes applications such as image processing Is included.
The MMU unit (26) may be configured by software on the CPU (22).

  In the present invention, first, the capacity of the physical memory (23) used by the OS is made smaller than the actual memory capacity by setting on the OS. For example, in the case of an actual capacity of 2 Gbytes, if the installed memory capacity is set to 1 Gbyte, only the first 1 Gbyte is recognized and the latter 1 Gbyte is an unused area.

The actual setting method in linux (registered trademark) indicates a smaller amount of memory than the actual setting file at startup (grub.conf, lilo.conf, etc.) or startup options.
For example, the contents of grub.conf when only 1024 Mbytes are used on a PC with 2048 Mbytes are defined as shown in FIG.
When linux (registered trademark) is started with this setting, up to (0x00000000-0x40000000) in the 2GB total memory space (0x00000000-0x80000000) is under the management of linux (registered trademark) and the latter half (0x40000000-0x80000000) It can be operated so that it is not used at all.

In addition, linux (registered trademark) has a special device file (/ dev / mem, / dev / kmem) that can read and write physical memory directly by the application. The area (0x40000000-0x80000000) is described as shown in Fig. 4. It can be provided to the application side with a map function.
By such processing, a continuous 1 Gbyte address space (hereinafter referred to as reference memory space) is secured in the physical memory space.

  According to the above setting, since a large continuous address space is secured before the operating system is started, high-speed data transfer can be performed without being affected by memory fragmentation by the operating system.

  The DMA transfer control unit (27) directly writes the transfer data from the interface (21) into the reference memory space by DMA transfer. At this time, as shown in FIG. 1, when a DMA transfer enable interrupt (12) is generated from the interface (21), the DMA transfer control unit (27) issues a DMA transfer command, and then the interface (21) is connected to the reference memory. DMA transfer to space.

  Since the reference memory space is a continuous physical memory and does not require conversion processing using a table as described above, high-speed transfer close to the theoretical transfer rate value of the PCI bus is possible. However, since the reference memory space is not reserved by the OS, the data is simply stored in the memory as it is, and cannot be referenced from the application.

Therefore, the MMU unit (26) performs memory mapping so that it can be referred to. The memory mapping is performed by dividing into blocks in the OS process space, but in the memory space separated from the OS, by mapping the space below the start address, it can be recognized as one large block. become.
According to this method, it is possible to refer directly from the application execution unit (28), and data transfer can be performed at a much higher speed than before.

The method for allocating continuous physical memory space is not limited to the above, and an optimum method can be used depending on the OS to be executed. The above is a preferred method that is easily executed in Linux (registered trademark).
Further, the continuous physical memory space needs to be larger than at least the block size by which the OS divides the physical memory. For example, in the case of the current Linux (registered trademark), since the maximum continuous address space is 2 Mbytes, it is desirable to exceed it.

  The acquired data can be stored in the hard disk (24), for example. The hard disk can perform real-time processing by using a hard disk that is capable of high-speed data transfer using a RAID system as is well known.

Finally, the load on the CPU is compared with the conventional method.
For example, when the maximum size of a block that can be allocated by the OS is 2 Mbytes, the conventional configuration shown in FIG. 8 requires many times of DMA transfer control. When data is input from the device (60) that acquired the data at a data rate of 1 Gbps, DMA transfer to the physical memory (62) requires 64 DMA transfers per second and data transfer to the virtual space memory.

At this time, the device driver and application tasks are switched 64 times. If task A that requires CPU processing time for one second is generated separately, the task management manager executes a driver, an application, and task A in a loop.
In linux (registered trademark), task switching is forcibly switched in 10 milliseconds unless the task itself completes processing. When such a task A occurs during data transfer, the task A is also performed 64 times while the DMA transfer is performed 64 times, and a waiting time of 10 milliseconds generated in the task A is performed for each DMA transfer. appear.
In this case, 640 milliseconds in one second is not used at all for the transfer time, and the transfer efficiency is reduced to 36%.

  On the other hand, according to the present invention, by preparing a 128 Mbyte continuous physical memory space, when data is input at a data rate of 1 Gbps, it is only necessary to perform DMA transfer once per second. As a result, real-time processing of data becomes possible and no data is lost.

Specifically, in this embodiment, eight 128 Mbyte memory spaces are secured and a round-robin buffer configuration is adopted. For this reason, even if the processing of the application is somewhat delayed (allowed up to several seconds), no missing occurs.
For example, some recent hard disks have a function of stopping the rotation of a disk that has not been used for a long time in order to reduce power consumption. When the disk stops rotating, it takes several seconds until the recording is actually started even if the recording is started. However, according to the present invention, the delay can be sufficiently absorbed by using the buffer for 8 seconds.

  The present invention can be generally used for data processing devices, but is particularly suitable for use in high-definition image capturing and processing devices, data acquisition devices that process a large amount of scientific and technological data, or measurement devices.

It is explanatory drawing which shows the basic composition of the data transfer method in this invention. 1 is a configuration diagram of an image processing apparatus according to the present invention. It is a figure which shows a setting as a memory allocation method when an operating system uses linux (trademark). It is a figure which shows a setting as a method of providing a continuous address space to the application side with a map function. It is explanatory drawing explaining the conventional data transfer method. It is explanatory drawing explaining the conventional data transfer method. It is explanatory drawing explaining the conventional data transfer method. It is explanatory drawing explaining the conventional data transfer method (method using DMA).

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Data acquisition device 11 Device driver 12 DMA transfer possible interrupt 13 DMA transfer instruction 14 DMA transfer 15 Physical memory 16 Application

Claims (8)

When referring to data in an application, it is a high-speed data transfer method for acquiring data at high speed from a device connected by an input / output bus.
A reference memory space allocation step in which the memory space allocation means allocates a continuous address space larger than the maximum continuous address space used for the virtual memory space of the operating system in the physical memory space;
A DMA transfer step in which DMA (Direct Memory Access) transfer means transfers data by DMA transfer from a device connected to the reference memory space by an input / output bus;
A memory mapping step in which a memory mapping means makes the reference memory space memory-mapped so that it can be referenced from an application executed on an operating system;
A high-speed data transfer method from a device, wherein the application execution means includes a data reference step of referring to the reference memory space. The high-speed data transfer method from the device according to claim 1, wherein the reference memory space allocation step is performed before the operating system is started to secure a large continuous address space in advance. In the reference memory space allocation step,
The high-speed data transfer method from the device according to claim 1, wherein a memory capacity used in the operating system is set smaller than an actual mounted memory capacity. The high-speed data transfer method from the device according to claim 1, wherein a memory capacity of the continuous address space is 2 Mbytes or more. A data processing apparatus capable of high-speed data transfer from a device connected by an input / output bus when referring to data in an application, together with the device and physical memory,
A reference memory space allocating means for allocating a continuous address space larger than a maximum continuous address space used for a virtual memory space of the operating system in the physical memory space;
DMA transfer means for transferring data by DMA (Direct Memory Access) transfer from a device connected to the reference memory space by an input / output bus;
Memory mapping means for mapping the reference memory space so that it can be referenced from an application executed on an operating system;
Application processing means for executing an application while referring to the reference memory space. The data processing apparatus from the device according to claim 5, wherein the reference memory space allocating unit secures a large continuous address space in advance before starting the operating system. The reference memory space allocation means is
The data processing device according to claim 5, wherein a memory capacity used in the operating system is set smaller than an actual mounted memory capacity. The data processing device according to claim 5, wherein a memory capacity of the continuous address space is 2 Mbytes or more.

Images