JP2020035263A - Memory controller - Google Patents

Abstract

To extend a power-saving mode without wasting the processing margin of a bus master.SOLUTION: A memory controller comprises: a plurality of command control parts for controlling transfer commands in units of shared memory; and a power-saving mode control part for controlling power-saving modes of a plurality of shared memories. The command control part controls a transfer command to the shared memory by an arbiter that arbitrates a transfer request to the shared memory and a command queue that temporarily stores the transfer command arbitrated by the arbiter. If there are transfer constraints added for the plurality of transfers when a bus master issues a plurality of transfer requests to the plurality of shared memories, the power-saving mode control part calculates end times of the plurality of transfers to which the transfer constraints are added, and controls the power-saving modes of the shared memories so that the transfer end time of a faster transfer end side is delayed within a range not exceeding the transfer end time of a slower transfer end side.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a memory controller.

  2. Description of the Related Art Conventionally, by constructing a system by combining a volatile memory such as an SDRAM and an LSI, the memory capacity (circuit scale) inside the LSI has been reduced. In such a system, a memory controller for arbitrating when transfer to the shared memory conflicts is used. However, in recent years, with the advancement of semiconductor miniaturization technology, the scale of circuits that can be mounted on one LSI has been rapidly increasing.

  For this reason, even in an imaging apparatus such as a digital still camera or a video camera, a CPU for system control, a development processing unit that performs color conversion, resizing processing, and the like on an input image from a sensor, and efficiently recording on a recording medium. The data compression means and the like are mounted on one LSI, and the use of a shared memory by a plurality of bus masters is increasing. As a result, it is difficult to control the arbitration of the shared memory only by the priority between the bus masters.

  In addition, the clock frequency operating inside the LSI has been increased due to the advance of the miniaturization technology, and the power consumption has been increased due to the increase in the circuit scale and the increase in the clock frequency. Therefore, reduction of the power consumption has been demanded. Therefore, an imaging apparatus having a power saving mode for suppressing power consumption of a bus system and an external memory has been proposed. However, switching between ON and OFF of the power saving mode requires time to execute a predetermined sequence, and transfer to the memory cannot be performed during switching of the power saving mode. For this reason, if the power saving mode is frequently switched, the transfer band of the memory is lost, and the power reduction effect is reduced.

  Therefore, Patent Document 1 discloses that when a transfer request from the bus master occurs during the power saving mode, the return from the power saving mode is delayed for a predetermined time within a range not exceeding the latency request of the bus master, thereby reducing the period of the power saving mode. Have been proposed.

JP-A-2006-81357

  In the imaging apparatus, there are a bus master that executes processing in units of one image and a bus master that executes processing in units of one horizontal line in one image. The latency request per operation of each bus master is obtained by dividing the processing margin per processing unit by the number of transfer requests. Here, since a plurality of bus masters asynchronously transfer data to the shared memory during one image processing period, the congestion of the shared memory varies. For this reason, the bus master that processes in units of one image stores a margin during a period when the entire system is sparse, and discharges the stored margin during a period when the entire system is dense to adjust the overall balance.

However, in the method of Patent Literature 1, the return from the power saving mode is delayed within the range of the latency request in one transfer request unit, so that it is difficult to adjust the margin. If the next transfer does not occur after the elapse of the predetermined time and the transfer request that has been waiting has been executed, the mode is switched to the power saving mode again. In this case, no power reduction effect is obtained, but the processing margin of the bus master is lost.

  SUMMARY OF THE INVENTION It is an object of the present invention to obtain a power saving effect by extending a power saving mode without wasting a processing margin of a bus master.

In order to achieve the above object, a memory controller according to the present invention comprises:
A memory controller for controlling transfer from a plurality of bus masters to a plurality of shared memories, a plurality of command controllers for controlling transfer commands on a shared memory basis, and a power saving mode control for controlling a power saving mode of the plurality of shared memories. The command control unit controls a transfer command to the shared memory by an arbiter for arbitrating a transfer request to the shared memory, and a command queue for temporarily storing a transfer command arbitrated by the arbiter, When a plurality of transfer requests are issued from the bus master to a plurality of shared memories, if a transfer constraint is added between the plurality of transfers, the power saving mode control unit The transfer end time of the earlier transfer end is calculated as long as the transfer end time does not exceed the transfer end time of the later transfer end. And controlling the power saving mode of the shared memory to delay time.

  ADVANTAGE OF THE INVENTION According to the memory controller which concerns on this invention, the power saving effect by lengthening a power saving mode without wasting the processing margin of a bus master by controlling a power saving mode according to the transfer situation between several shared memories Is obtained.

It is a block diagram of an imaging device. It is a flowchart of a power saving mode control. 5 is a timing chart of power saving mode control. 9 is a timing chart of conventional power saving mode control.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

[Embodiment 1]
FIG. 1 is a diagram illustrating a configuration of an embodiment of an imaging device of the present invention. Hereinafter, FIG. 1 will be described.

  In FIG. 1, an LSI 100 for performing image processing, an imaging sensor 120 for photoelectrically converting a subject image, a display unit 130 for displaying a processed image, a recording medium 140 for recording a processed still image or moving image, storage of a program and image It comprises an SDRAM 150a and an SDRAM 150b for temporarily recording an intermediate image or the like generated by the processing means. The CPU 101 reads a program from the SDRAM 150a, and sets a cycle for the timing control means 102 according to the shooting mode.

  The timing control unit 102 outputs a horizontal / vertical synchronization signal to the sensor control unit 103 and the display control unit 104 according to the set cycle. The sensor control unit 103 controls the output of the image sensor 120 according to the input synchronization signal. The imaging processing unit 105 performs defective pixel correction, shading correction, and the like on the image data output from the image sensor 120. The processed image data is recorded on the SDRAM 150b via the memory bus 106 and the memory controller 110.

  The development processing unit 107 reads out image data after the imaging processing recorded in the SDRAM 150b via the memory bus 106 and the memory controller 110, and performs resizing processing such as pixel interpolation, filtering processing, and reduction, and color conversion processing, for example, compressed image processing. Development processing such as conversion to a YCbCr format, which is the most suitable format for storing data, is performed, and the processed image is recorded on the SDRAM 150a and the SDRAM 150b via the memory bus 106 and the memory controller 110.

  Here, the image after development is divided into Y, which is a luminance signal, and CbCr, which is a color difference signal, and the luminance signal is recorded on the SDRAM 150a and the color difference signal is recorded on the SDRAM 150b. The recognition processing unit 108 reads a luminance signal from the SDRAM 150a via the memory bus 106 and the memory controller 110, and compares the luminance signal with preset face information of a person to determine whether a specific person is included.

  The display control unit 104 reads the image data after the development processing from the SDRAM 150 a and the SDRAM 150 b via the memory bus 106 and the memory controller 110 in accordance with the synchronization signal, and outputs the read image data to the display unit 130. The encoding unit 109 reads the image after the development processing from the SDRAM 150a and the SDRAM 150b via the memory bus 106 and the memory controller 110, and 264 and the like, and is recorded on the recording medium 140. Next, the configuration inside the memory controller 110 will be described.

  The memory controller 110 includes command control means 111a and 111b for controlling commands for the SDRAMs 150a and 150b, and a power saving mode control unit 112 for controlling the power saving modes of the SDRAMs 150a and 150b. The command control units 111a and 111b include arbitration units 113a and 113b for arbitrating transfer requests from the bus master and command queues 114a and 114b for temporarily storing transfer commands after arbitration.

  The power saving mode control unit 112 controls switching of the power saving mode according to the command storage status of the command queues 114a and 114b and the power saving mode status of the SDRAMs 150a and 150b. The power saving mode control unit 112 includes a timer (not shown), and performs control to delay switching timing of the power saving mode using the timer.

  FIG. 2 is a control flowchart of the power saving mode control unit 112. The power saving mode control according to the present embodiment will be described with reference to FIG. The power saving mode control means 112 controls the power saving mode for each SDRAM, and a new command is input to each command queue, or a command being executed in each SDRAM ends, and a new command can be issued. Or when the time set in the timer prepared for each SDRAM in the power saving mode control unit elapses, the process of the flowchart is started for each SDRAM.

  When the process of the flowchart is started, it is determined whether the time set in the timer has elapsed (STEP 201). If the time has elapsed, the power saving mode is switched from ON to OFF (STEP 202). If the time has not elapsed, it is determined whether the power saving mode on the side where the command can be issued is ON (STEP 203). If the power saving mode is ON, it is determined whether a transfer constraint between a plurality of transfer requests is added (STEP 204). If not, the power saving mode is switched from ON to OFF (STEP 202).

  When the transfer constraint is added, the time until the transfer on the side to which the new command is input and the time until the other transfer is completed are calculated (STEP 205), and the transfer end time of the new command is calculated. Is greater than the end time of the other transfer (STEP 206). If the transfer end time of the new command is longer than the other transfer time, the power saving mode is switched to OFF (STEP 202). If the transfer end time of the new command is smaller than the other transfer time, a difference time is set, a timer is started (STEP 207), and the process is ended.

  If the power saving mode is OFF in STEP 203, it is determined whether or not a command is stored in the command queue (STEP 208). If a command is stored, the process ends. If no command is stored, the power saving mode is switched from OFF to ON (STEP 209), and the process ends.

  FIG. 4 is a timing chart of the conventional power saving control. In FIG. 4, the transfer request from the bus master is limited to two bus masters in order to simplify the description, and it is assumed that the transfer request does not conflict in the arbitration unit. In FIG. 4A, the time axis is an index of the processing time, and the processing of T0 to T1 is processing time: 1, and the processing of T2 to T4 is processing time: 2. FIG. 4B shows the timing of the transfer request of the development processing unit 107, and the transfer request for the luminance signal and the transfer request for the color difference signal are issued in parallel. FIG. 4C shows the timing of the transfer request of the recognition processing unit 108. 4 (2) and 4 (3), the start of the transfer request is indicated by the rise of the signal, and the transfer request is accepted by the fall of the signal. Address information (a: SDRAM 150a, b: SDRAM 150b) is added as a transfer request specification.

  In FIG. 4, (4) to (7) show operation timings for the SDRAM 150a, and (8) to (11) show operation timings for the SDRAM 150b. FIGS. 4 (4) and (8) show the types and numbers of commands stored in the command queues 112a and 112b. FIGS. 4 (5) and (9) show the count of the timer for delaying the timing of switching the power saving mode from OFF to ON. At the timing when the count changes from '1' to '0', the timer enters the power saving mode. Initiate the switch.

  FIGS. 4 (6) and (10) show the state of the power saving mode for the SDRAMs 150a and 150b, and the data is transferred to the SDRAM only during the OFF period. FIGS. 4 (7) and (11) show the operation status of the SDRAMs 150a and 150b. The period in which the SDRAM is in the power saving state is standby, and the transition period from the power saving state to the state in which transfer can be started is marked with a triangle. ↓ indicates the transition period from the transfer state to the standby state, and the bus master name indicates that the transfer process is being performed on each bus master. Here, the transfer time with respect to the bus master is set to processing time: 4 for all bus masters, and each transition period is set to processing time: 1.

  The transition period is a period during which data cannot be transferred to the SDRAM, but since the SDRAM is not in the standby state, no power reduction effect can be obtained. For this reason, the transfer band is lost as the number of transitions increases.

  Next, the flow of processing will be described based on the time axis. In FIG. 4, at T0, the SDRAM 150a is in a standby state, and the command queue 112a is also empty. The transfer of the encoding unit 109 to the SDRAM 150b is started, and the transfer of the imaging processing unit 105 is stored in the command queue 112b. At T1, the development processing unit 107 issues a transfer request of a luminance signal to the SDRAM 150a and a transfer request of a color difference signal to the SDRAM 150b.

  At T2, since there is free space in the command queues 112a and 112b, the transfer requests of the development processing unit 107 are accepted by the arbitration units 113a and 113b and stored in the command queues 114a and 114b. Here, since the SDRAM 150a is in the standby state, it is necessary to switch the power saving mode from ON to OFF in order to transfer to the SDRAM. However, a timer a is set to wait for a preset time in order to prevent the power saving mode from being fragmented. Here, the waiting time is set to the processing time: 2, and カ ウ ン タ 2 ’is set in the counter.

  At T3, the count of the timer a is decremented to “1”. At T4, the timer a changes from “1” to “0” and the waiting time has elapsed, so that the switching of the power saving mode of the SDRAM 150a is started. Further, the transfer of the encoding unit 109 of the SDRAM 150b ends, a transfer command to the imaging processing unit 105 is read from the command queue 114b, and the transfer to the SDRAM 150b is started.

  At T5, the transfer to the SDRAM 150a becomes possible, so the transfer of the developing unit 107 is started from the command queue 114a. At T8, the transfer of the SDRAM 150b to the imaging processing unit 105 ends, and the transfer of the development processing unit 107 starts from 114b in the command queue. At T9, the transfer to the development processing unit 107 of the SDRAM 150a ends. Since the command queue 114a is empty, the power saving mode is started to be switched from OFF to ON in order to reduce the power of the SDRAM 150a.

  At T10, the recognition processing unit 108 issues a transfer request to the SDRAM 150a. Further, the transition of the power saving mode of the SDRAM 150a is completed, and the SDRAM 150a enters a standby state. At T11, since there is free space in the command queue 114a, the transfer request of the recognition processing unit 108 is accepted by the arbitration unit 113a and stored in the command queue 114a. Here, since the SDRAM 150a is in the standby state, "2" is set to the timer a similarly to T2.

  At T12, the count of the timer a is decremented to “1”. Also, the transfer of the SDRAM 150b to the development processing unit 107 ends. Since the command queue 114b is empty, the power saving mode is switched from OFF to ON in order to reduce the power of the SDRAM 150b.

  At T13, the timer a changes from "1" to "0" and the waiting time has elapsed, so that the switching of the power saving mode of the SDRAM 150a is started. Further, the transition of the SDRAM 150b to the power saving mode is completed, and the SDRAM 150b enters the standby state.

At T14, transfer to the SDRAM 150a becomes possible, so transfer of the recognition processing unit 108 is started from the command queue 114a.
At T18, the transfer to the recognition processing unit 108 of the SDRAM 150a ends. Since the command queue 114a is empty, the power saving mode starts to be switched from OFF to ON in order to reduce the power of the SDRAM 150a. At T19, the transition of the SDRAM 150a to the power saving mode is completed, and the SDRAM 150a enters the standby state.

  In FIG. 4, when the transfer request is received, the SDRAM is controlled to wait for a predetermined time when the SDRAM is in the power saving state. However, since the state of a plurality of SDRAMs is not referred to, switching of the power saving mode occurs when the next transfer request does not occur within the range of the latency set for each transfer request.

  FIG. 3 shows timings when a transfer request specification among a plurality of transfer requests is used. 3 (1) to 3 (11) show the same power saving control timings as those of FIGS. 4 (1) to 4 (11), and a description thereof will be omitted.

  3 (2) and 3 (3), the rise of the signal indicates that the transfer request has started, and the fall indicates that the transfer request has been accepted. Further, as a transfer request specification, a mark “●” is added as a flag indicating that address information (a: SDRAM 150a, b: SDRAM 150b) and transfer conditions are set between a plurality of transfer requests. FIGS. 3 (4) and (8) show the types and numbers of commands stored in the command queues 112a and 112b. Also, a mark “●” is added as a flag indicating that the transfer condition is set between a plurality of transfer requests.

  The operation of the power saving mode control of this embodiment will be described with reference to the flowchart of the power saving mode control of FIG. 2 and FIG. At T0, the SDRAM 150a is in a standby state, and the command queue 112a is also empty. The transfer of the encoding unit 109 to the SDRAM 150b is started, and the transfer of the imaging processing unit 105 is stored in the command queue 112b.

  At T1, the development processing unit 107 issues a transfer request of a luminance signal to the SDRAM 150a and a transfer request of a color difference signal to the SDRAM 150b. Here, in the development process, the luminance signal and the color difference signal are processed in synchronization on a pixel-by-pixel basis, so that even if one transfer ends, the next process cannot be started unless the other transfer ends. For this reason, no loss occurs even if the transfer execution timing is delayed, as long as the transfer on the side where transfer ends earlier is within the range until the transfer on the later side ends. Therefore, a flag is added for controlling the transfer of the luminance signal to the SDRAM 150a and the transfer request of the chrominance signal to the SDRAM 150b so as to delay the transfer of the earlier transfer within the range of the transfer end time of the later transfer end. .

  At T2, since there is free space in the command queues 112a and 112b, the transfer requests of the development processing unit 107 are accepted by the arbitration units 113a and 113b and stored in the command queues 114a and 114b. Here, since a new command is stored in the command queue 114a, the control method of the power saving mode for the SDRAM 150a is determined. The power saving mode of the SDRAM 150a is ON, and a transfer constraint between a plurality of transfers is added. Therefore, in the flowchart of FIG.

  Here, the SDRAM 150a calculates the time required for the transfer of the development processing unit in the command queue 114a to be completed by itself and the time required for the SDRAM 150a to complete the transfer of the development processing unit in the command queue 114b. In this case, the transfer of its own is ‘5’ as the total value of the power saving mode switching time of the SDRAM 150a: 1 and the transfer time of the development processing unit: 4. On the other hand, in the other transfer, the remaining time of the transfer of the encoding unit during transfer: 2, the transfer time of the imaging processing unit to be transferred next in the command queue: 4, and the transfer time of the development processing unit: 4 Is "10" in total.

  For this reason, STEP 206 → 207, and the difference time “5” is set in the timer a. Further, since a new command is also stored in the command queue 114b, the control method of the power saving mode for the SDRAM 150b is determined. Since the power saving mode of the SDRAM 150b is OFF and the SDRAM 150b is in the process of being transferred, STEP 201 → 203 → 208, and the power saving mode is not controlled.

  At T3, the count of the timer a is decremented to "4". At T4, the count of the timer a is decremented to "3". Further, the transfer of the encoding unit 109 of the SDRAM 150b ends, a transfer command to the imaging processing unit 105 is read from the command queue 114b, and the transfer to the SDRAM 150b is started. In addition, since the transfer of the SDRAM 150b ends, the control method of the power saving mode for the SDRAM 150b is determined. Since the power saving mode of the SDRAM 150b is OFF and the SDRAM 150b is in the process of being transferred, STEP 201 → 203 → 208, and the power saving mode is not controlled.

  At T5, the count of the timer a is decremented to "2". At T6, the count of the timer a is decremented to "1". At T7, the count of the timer a is decremented to “0”. Here, since the time set in the timer a of the SDRAM 150a has elapsed, the control method of the power saving mode for the SDRAM 150a is determined. Here, since the time set in the timer a has elapsed, STEP 201 changes to STEP 202, and the power saving mode is switched from OFF to ON.

  At T8, transfer to the SDRAM 150a becomes possible, so transfer of the development processing unit 107 is started from the command queue 114a. Here, since the transfer of the SDRAM 150a ends, the control method of the power saving mode for the SDRAM 150a is determined. Since the SDRAM 150a is transferring data, STEP 201 → 203 → 208, and the power saving mode is not controlled. Further, the transfer of the imaging processing unit 105 of the SDRAM 150b ends, a transfer command for the developing unit 107 is read from the command queue 114b, and the transfer to the SDRAM 150b is started. In addition, since the transfer of the SDRAM 150b ends, the control method of the power saving mode for the SDRAM 150b is determined. Since the power saving mode of the SDRAM 150b is OFF and the SDRAM 150b is in the process of being transferred, STEP 201 → 203 → 208, and the power saving mode is not controlled.

  At T10, the recognition processing unit 108 issues a transfer request to the SDRAM 150a. At T11, since there is free space in the command queue 114a, the transfer request of the recognition processing unit 108 is accepted by the arbitration unit 113a and stored in the command queue 114a. Here, since a new command is stored in the command queue 114a, the control method of the power saving mode for the SDRAM 150a is determined. The power saving mode of the SDRAM 150a is OFF, and since the SDRAM 150a is transferring data, STEP 201 → 203 → 208, and the power saving mode is not controlled.

  At T12, the transfer of the development processing unit 107 of the SDRAM 150a ends, a transfer command to the recognition processing unit 108 is read from the command queue 114a, and transfer to the SDRAM 150a is started. In addition, since the transfer of the SDRAM 150a ends, the control method of the power saving mode for the SDRAM 150a is determined. The power saving mode of the SDRAM 150a is OFF, and since the SDRAM 150a is transferring data, STEP 201 → 203 → 208, and the power saving mode is not controlled.

  In addition, since the transfer of the SDRAM 150b ends, the control method of the power saving mode for the SDRAM 150b is determined. Since the power saving mode of the SDRAM 150b is OFF and the SDRAM 150b is not transferring, the power saving mode is switched from OFF to ON in STEP 201 → 203 → 208 → 209.

  At T13, the timer a changes from "1" to "0" and the waiting time has elapsed, so that the switching of the power saving mode of the SDRAM 150a is started. Further, the transition of the SDRAM 150b to the power saving mode is completed, and the SDRAM 150b enters the standby state.

  At T16, the transfer of the recognition processing unit 108 of the SDRAM 150a ends. Here, since the transfer of the SDRAM 150a ends, the control method of the power saving mode for the SDRAM 150a is determined. Since the power saving mode of the SDRAM 150a is OFF and the SDRAM 150a is not transferring, the power saving mode is switched from OFF to ON in STEP 201 → 203 → 208 → 209. At T17, the transition of the SDRAM 150a to the power saving mode is completed, and the SDRAM 150a enters a standby state.

  In FIG. 3, the transfer end times of two transfer requests issued in parallel to the two SDRAMs from the development processing unit are compared with each other, and a return from the power saving mode is performed within a range not exceeding the transfer end time on the late side. Is controlled to delay. As a result, compared to the case where the return from the power saving mode is delayed for a fixed time, the number of times the power saving mode is switched can be reduced without delaying the transfer end time in response to the transfer request from the development processing unit.

  As described above, in the power saving mode control according to the present embodiment, a plurality of transfers are performed in parallel to a plurality of shared memories, and when there is a transfer constraint between the plurality of transfers, By delaying the control timing of the power saving mode within a range that is within the transfer constraint between a plurality of transfers according to the status of the queue, the number of times the power saving mode is switched can be reduced without causing a loss to the bus master.

  As a result, the period during which the shared memory is in the power saving state is lengthened, and the power reduction effect can be enhanced.

101 CPU, 120 Image sensor, 130 Display unit

Claims (1)

A memory controller that controls transfer from a plurality of bus masters to a plurality of shared memories,
A plurality of command controllers for controlling transfer commands in units of shared memory;
A power saving mode control unit that controls a power saving mode of the plurality of shared memories,
The command control unit controls a transfer command to the shared memory by an arbiter that arbitrates a transfer request to the shared memory and a command queue that temporarily stores a transfer command arbitrated by the arbiter,
When a plurality of transfer requests are issued to a plurality of shared memories from the bus master, the power saving mode control unit adds a transfer constraint when a transfer constraint is added between the plurality of transfers. Calculate the end time of multiple transfers and control the power saving mode of the shared memory so that the transfer end time of the earlier transfer end is delayed within the transfer end time of the later transfer end. Features memory controller.