JP5381409B2 - Image processing apparatus, image processing method, and control program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a control program that read and process video data of an input video signal at an internal synchronization signal read timing.

  The video signal may have a different frequency (frame speed) depending on a video source (for example, a video recorder or a camera). For this reason, the image processing apparatus generally generates a synchronization signal from a stable clock inside the apparatus itself, and transfers the video data to the synchronization signal to process the video signal.

  In particular, since the synchronization signal of the video signal input from the outside depends on the video source, it is not exactly the same in frequency as the synchronization signal generated inside the image processing apparatus. For this reason, in order to continuously output a stable video from the image processing apparatus, it is necessary to skip (discard) or repeat (re-output) a part of the video signal input from the outside.

  For example, when the video input has a higher frame rate than the video output, the number of frames of the input video data exceeds the number of frames of the output video data, and the memory capacity of the image processing apparatus that stores the frames is insufficient. . For this reason, the image processing apparatus skips (discards) the frame because a frame that cannot be output occurs at a certain time.

  In addition, when the video input has a slower frame rate than the video output, the number of frames of the input video data is less than the number of frames of the output video data, and the amount of video data stored in the memory of the image processing apparatus that stores the frames Is lacking. For this reason, the image processing apparatus has a shortage of frames to be output at a certain point in time, and thus repeats (re-outputs) the frames that have been output once to compensate for the frames to be output.

  For example, a conventional image processing apparatus includes a memory bank having a storage area in which an index is set for each field unit. The image processing apparatus also controls input / output control of data in a memory bank by an input management queue for managing an index of a free area in the memory bank and an output management queue for managing an index of a data area that can be output in the memory bank. Provide a circuit. Further, the input / output control circuit determines whether the number of fields that can be output by the output management queue, means for comparing the determined number of fields with a threshold value, and whether skip processing or repeat processing is performed based on the comparison result. Means for determining (see, for example, Patent Document 1).

  In the conventional image processing apparatus, the moving image data shot by the imaging device is stored in the moving image data storage area of the storage device. The CPU detects a scene turning point at which the scene changes greatly from the moving image data, and partially extracts the moving image data based on the scene turning point. Then, the extracted partial video is stored as a thumbnail video in the thumbnail storage area in association with the video data. This thumbnail moving image corresponds to a scene change and has a dynamic element unlike a still image. Therefore, it is possible to immediately determine which scene is taken (see, for example, Patent Document 2).

JP 2007-259218 A Japanese Patent Laid-Open No. 2004-166131

  However, the conventional image processing apparatus discards the digital image signal that has been stored in the memory bank for the longest as skip processing, and outputs again the same signal as the last output digital image signal as repeat processing. For this reason, the conventional image processing apparatus allows a viewer who has dropped frames or stopped for a moment when a frame to be skipped or repeated is a frame of a portion of a digital image signal that is moving rapidly. There is a problem that the video gives a sense of incongruity.

  Accordingly, it is an object of the present invention to provide an image processing apparatus or the like that can output a video that does not make the viewer feel uncomfortable by making the skip and repeat of frames generated by changing the synchronization signal of the video signal inconspicuous on the display screen. .

An image processing apparatus includes: a motion amount detection unit that detects a frame in which a motion amount between frames of video data is lower than a first threshold; a writing unit that writes input video data in units of frames to a memory; A reading unit that reads out video data in units and a threshold setting unit that stores information indicating a predetermined allowable memory range are provided. In addition, the image processing apparatus outputs an instruction signal for instructing restriction on writing of a low frame to the memory to the writing unit based on the free capacity or the accumulation amount of the memory, and also the free space of the memory or the video in the memory. A capacity monitoring unit that outputs a capacity notification signal for notifying the accumulated amount of data to the threshold setting unit is provided, the threshold setting unit adjusts the memory allowable range based on the storage capacity notification signal, and the writing unit receives the instruction signal. Based on this, it limits the writing of low frames to the memory.

  The disclosed image processing apparatus has an effect of outputting a video that does not give the viewer a sense of incongruity by skipping a frame with a small amount of motion between video frames.

1 is a block diagram illustrating an example of a schematic configuration of an image processing apparatus according to a first embodiment. FIG. 2 is an explanatory diagram for explaining an embodiment of an operation of a synchronization signal transfer method in the image processing apparatus shown in FIG. 1, (a) shows input video data input to a motion amount detection unit, (b) ) Shows the identification result of whether or not it is a low motion frame by the motion amount detection unit, (c) shows write control, and (d) shows read control. 3 is a flowchart illustrating an example of a processing procedure in the image processing apparatus illustrated in FIG. 1. FIG. 4 is a flowchart showing a continuation of the flowchart shown in FIG. 3. It is a block diagram which shows the other Example of schematic structure of the image processing apparatus which concerns on 1st Embodiment.

  Here, the present invention can be implemented in many different forms. Therefore, it should not be interpreted only by the description of the following embodiment. Also, the same reference numerals are given to the same elements throughout the embodiment.

  In the embodiment, the apparatus will be mainly described. However, as will be apparent to those skilled in the art, the present invention can also be implemented as a program and method usable on a computer. In addition, the present invention can be implemented in hardware, software, or software and hardware embodiments.

  The program can be recorded on any computer-readable medium such as a hard disk, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), an optical storage device, or a magnetic storage device. Furthermore, the program can be recorded on another computer via a network.

  The image processing apparatus according to the embodiment includes a motion amount detection unit, a writing unit, a reading unit, and a capacity monitoring unit. The motion detection unit detects a frame in which the motion amount between frames of the input video data is lower than the first threshold. The writing unit writes the input video data in frame units to the memory, and the reading unit reads video data from the memory in frame units. The capacity monitoring unit outputs an instruction signal for instructing restriction on writing of a low frame to the memory to the writing unit based on the free capacity or the accumulation amount of the memory. Further, the writing unit limits writing of the low frame to the memory based on an instruction signal from the capacity monitoring unit.

(First embodiment)
In FIG. 1, an image processing apparatus 10 is an apparatus that processes a video signal input from the outside as a video signal including video data synchronized with an internal synchronization signal. The image processing apparatus 10 also includes a video input unit 1, a threshold setting unit 2, a motion amount detection unit 3, a capacity monitoring unit 4, a write control unit 5, a read control unit 6, a memory unit 7, and a video processing unit 8. ing.

The video input unit 1 is an example of video input means. Image input unit 1, the input video signal S _IN input from the outside, and the input image data S ₁ including a plurality of frames, timing signals, including the sync signal and the clock for the input image data S ₁ (hereinafter, input timing signal separating the referred) and S _2. Further, the video input unit 1 outputs the input video data S ₁ to the motion amount detection unit 3, and outputs the input timing signal S ₂ to the motion amount detection unit 3, the write control unit 5, and the memory unit 7.

The synchronization signal is a signal for transmitting the writing timing of each frame to the motion amount detection unit 3, the write control unit 5, and the memory unit 7, and the clock is the motion amount detection unit 3, the write control unit 5, and the memory unit. 7 is a signal generated periodically so that the operation of step 7 is in tune. The input video data S ₁ is data synchronized with the input timing signal S ₂ .

The threshold setting unit 2 is an example of threshold setting means. Threshold setting unit 2 stores information indicating the motion amount of the threshold for determining the "low action frame" described later, the motion amount of the first threshold signal S ₃ carrying the information indicating the threshold value for motion-estimation Output to part 3. Further, the threshold setting unit 2 stores the video data in the memory unit 7 when the video data storage amount in the memory unit 7 is empty, and the reference for stopping the frame writing from the memory unit 7 or the reference for starting the reading. The information indicating the threshold value of the accumulated amount is stored. Further, the threshold setting unit 2 is a reference for starting output of a skip instruction signal S ₇ and a repeat instruction signal S ₈ to be described later, and an allowable range for the free capacity of the memory unit 7 or the accumulation amount of video data in the memory unit 7 ( Hereinafter, information indicating a memory allowable range) is stored.

In addition, the threshold setting unit 2 outputs a second threshold signal S _{4 on} which information indicating the threshold of the storage amount and information indicating the allowable memory range is placed to the capacity monitoring unit 4. Note that the motion amount threshold value and the accumulation amount threshold value are appropriately set by the designer of the image processing apparatus 10, and the accumulation amount threshold value is, for example, the storage capacity of the memory unit 7 corresponding to 7 frames. When the video data S ₅ can be stored, it is conceivable to set the frame to 3 frames.

Further, the input video data S ₁ includes a plurality of frames that are continuous in time series, and the higher the correlation between the images in the preceding and succeeding frames, the smaller the amount of motion of the subsequent frames with respect to the previous frames. Also, the input video data S ₁ indicates that the lower the correlation between the images in the previous and subsequent frames, the greater the amount of movement of the subsequent frame relative to the previous frame. For this reason, when the motion amount of the frame is lower than the motion amount threshold, it can be determined that the change in the video scene is small, and one of the frames before and after this (for example, the previous frame) is defined as a “low motion frame”. To do.

The motion amount detection unit 3 is an example of a motion amount detection unit. The motion amount detection unit 3 detects the motion amount of each frame by comparing consecutive frames of the input video data S ₁ , and the motion amount in the first threshold signal S ₃ input from the threshold setting unit 2. A frame lower than the motion amount threshold is identified as a low motion frame. Further, the motion amount detection unit 3 outputs the video data S ₅ from which the motion amount has been detected to the memory unit 7 in synchronization with the input timing signal S ₂ . Further, the motion amount detection unit 3 uses the first identification signal S _{6 on} which information indicating the identification result as to whether or not it is a low motion frame (hereinafter referred to as low motion frame identification information) as an input timing signal S _2. Are output to the write control unit 5 and the memory unit 7 in synchronization with each other. That is, the motion amount detection section 3 and the first identification signal S ₆ for each frame and each frame of the video data S ₅ in a pair, in synchronization with the input timing signal S _2, the image data S ₅ and the first The identification signal S ₆ is output to the memory unit 7.

  The amount of motion can be detected by, for example, comparing data at an arbitrary position in a certain frame with data located at the same position in the next frame and its surroundings. The detection of the amount of motion is a method performed in inter-frame prediction in an image compression technique such as MPEG (Moving Picture Experts Group), but is not limited to this method.

The capacity monitoring unit 4 is an example of capacity monitoring means. The capacity monitoring unit 4 is based on the difference in the number of inputs between a write completion notification signal S _{12 (} described later) input from the write control unit 5 and a read completion notification signal S _{15 (} described later) input from the read control unit 6. calculating the accumulated amount of free space or the video data S ₅ of the memory unit 7. That is, the capacity monitoring unit 4, the write completion notice signal S _12, to recognize that the accumulation amount of the video data in the memory unit 7 is increased one frame, the read completion notification signal S _15, the image in the memory unit 7 It can be recognized that the amount of accumulated data has decreased by one frame. For this reason, the capacity monitoring unit 4 can calculate the amount of video data stored in the memory unit 7 based on the difference in the number of inputs of the write completion notification signal S ₁₂ and the read completion notification signal S _15. By subtracting the accumulated amount from the above, the free capacity of the memory unit 7 can be calculated.

Further, the capacity monitoring unit 4 writes a skip instruction signal S _{7 on} which information for instructing the limitation of writing of the low operation frame to the memory unit 7 is placed when the free capacity of the memory unit 7 is below the allowable memory range. Output to the control unit 5. Also, the capacity monitoring unit 4, when the free space of the memory unit 7 exceeds the memory tolerance, the read control the repeat instruction signal S ₈ carrying the information indicating the re-reading of the low action frame from the memory unit 7 Output to unit 6.

The allowable memory range is set as appropriate by the designer of the image processing apparatus 10. For example, when the storage capacity of the memory unit 7 can store the video data S ₅ for 7 frames, the memory allowable range starts from 3 frames. Setting up to 5 frames is conceivable.

  In particular, if real time processing is required to process video data input from the outside and output it immediately, the image processing apparatus 10 reduces the storage capacity of the memory unit 7 and sets the lower limit of the memory allowable range. By increasing the value, it is conceivable to frequently use low-motion frame skip processing. If real-time processing is not required, the image processing apparatus 10 increases the storage capacity of the memory unit 7, lowers the lower limit value of the memory allowable range, and increases the upper limit value. Processing can be reduced, and video data input from the outside can be faithfully reproduced.

In the following description, the case where the capacity monitoring unit 4 monitors the free capacity of the memory unit 7 when outputting the skip instruction signal S ₇ and the repeat instruction signal S ₈ will be described. The accumulated amount may be monitored. That is, the capacity monitoring unit 4 includes a skip instruction signal S _{7 on} which information for instructing the limitation of writing of the low operation frame to the memory unit 7 is placed when the amount of video data stored in the memory unit 7 exceeds the allowable memory range. Is output to the writing control unit 5. The capacity monitoring unit 4 also includes a repeat instruction signal S _{8 on} which information for instructing to re-read a low-operation frame from the memory unit 7 when the amount of video data stored in the memory unit 7 falls below the allowable memory range. Is output to the read control unit 6.

Also, the capacity monitoring unit 4, when the accumulation amount of the video data in the memory unit 7 is empty, the read control the read stop instruction signal S ₉ carrying the information for instructing the stop of the reading frame from the memory unit 7 Output to unit 6. In addition, the capacity monitoring unit 4 instructs the start of frame reading from the memory unit 7 when the amount of video data stored in the memory unit 7 exceeds the threshold from the state where reading of the frame from the memory unit 7 is stopped. The read start instruction signal S ₁₀ carrying the information to be output is output to the read control unit 6.

  Note that when the amount of video data stored in the memory unit 7 is empty, for example, the state in which video data from outside immediately after the start of the image processing device 10 is not input to the memory unit 7 or the operation of the image processing device 10 In this state, the video data is empty without being input to the memory unit 7.

The write control unit 5 is an example of a write control unit. Write control unit 5, in synchronization with the input timing signal S ₂ of the synchronization signal, it controls the writing of a frame into the memory unit 7. That is, the writing control unit 5 gives an instruction for writing a frame to the memory unit 7 and an instruction for a storage area to be written for each frame of the video data S ₅ input from the motion amount detection unit 3 to the memory unit 7. A signal (hereinafter referred to as a write instruction signal S ₁₁ ) carrying the information shown is output to the memory unit 7.

In this case, while the skip instruction signal S ₇ is input from the capacity monitoring unit 4, the write control unit 5 performs identification based on the first identification signal S ₆ input from the motion amount detection unit 3. The write instruction signal S ₁₁ is not output to the memory unit 7 for the low operation frame. Accordingly, the write control unit 5 skips (discards) the low operation frame without writing the low operation frame to the memory unit 7 while the skip instruction signal S ₇ is input from the capacity monitoring unit 4. To do. For frames other than the low operation frame, the write instruction signal S ₁₁ is output from the write control unit 5, not the target of skip processing.

Further, every time the write control unit 5 outputs the write instruction signal S ₁₁ , the write control unit 5 outputs a write completion notification signal S ₁₂ with information for notifying the completion of writing to the capacity monitoring unit 4.
Here, the writing control of the frame to the memory unit 7 will be described with reference to FIG.

For example, the input video data S ₁ includes a plurality of frames (first frame (right arrow), second frame (down arrow), third frame (down arrow), and fourth frame (left arrow) shown in FIG. ), Video data including the fifth frame (upward arrow),.

Also, the input video data S ₁ is composed of a plurality of frames (101st frame (right arrow), 102nd frame (downward arrow), 103th frame (downward arrow), 104th frame (leftward arrow) shown in FIG. ), ...) is considered.

The motion amount detector 3 to which the input video data S ₁ shown in FIG. 2 (a) is inputted is compared with the previous frame (comparison between the 0th frame and the first frame) as shown in FIG. 2 (b). The amount of motion is detected by comparing the first frame with the second frame,. Then, the motion amount detection unit 3, and the detected motion amount is compared with the motion amount of the threshold value in the first threshold signal S ₃ which is inputted from the threshold setting unit 2, whether the lower operating frame Identify.

  In FIG. 2, the images of the second frame and the third frame match (the amount of motion is 0), the third frame is identified as the low motion frame, and the first frame, the second frame, The 4th frame and the 5th frame are identified as not being the low operation frames. In FIG. 2, the images of the 102nd frame and the 103rd frame match (the amount of motion is 0), and the 103rd frame is identified as a low motion frame, and the 101st frame, the 102nd frame, and the 104 frames are identified as not being low motion frames.

For example, consider a case where the skip instruction signal S ₇ is input from the capacity monitoring unit 4 to the write control unit 5 when the first frame is output from the motion amount detection unit 3 to the memory unit 7. Also, the 101 frame, the 102 frames, at the time of outputting from the amount detector 3 motion 103rd frame and 104th frame in the memory unit 7, the skip command signal S ₇ from the capacity monitoring unit 4 to the writing control section 5 Consider the case where it is not entered.

As shown in FIG. 2 (c), the write control section 5 is the skip instruction signal S ₇ at the time of determining whether to direct the writing of the first frame in the memory unit 7 is input However, since the first frame is not a low operation frame, the write instruction signal S ₁₁ is output to the memory unit 7. Thus, the first frame is stored in the storage area (M) of the memory unit 7 without being skipped. Similarly, since the second frame is not a low operation frame, the second frame is stored in the storage area (M + 1) of the memory unit 7 without being skipped.

On the other hand, the write control unit 5 receives the skip instruction signal S ₇ when it determines whether to instruct to write the third frame to the memory unit 7 and the third frame is low. Since it is an operation frame, the write instruction signal S ₁₁ is not output to the memory unit 7. As a result, the third frame is skipped and is not stored in the storage area of the memory unit 7.

Moreover, the write control unit 5, in order to skip instruction signal S ₇ at the time of determining whether to direct the writing of the fourth frame and the fifth frame to the memory section 7 is not input, the fourth frame The write instruction signal S ₁₁ is output to the memory unit 7 for the fifth frame. Thus, the fourth frame and the fifth frame are not skipped, and the fourth frame is stored in the storage area (M + 2) of the memory unit 7 and the fifth frame is stored in the storage area (M + 3) of the memory unit 7. Stored. In the FIG. 2 (c), after the third frame is skipping, frames are read from the memory unit 7 by the read instruction signal S ₁₃ to be described later by the read control unit 6, the free space of the memory unit 7 To increase. In addition, since the free capacity of the memory unit 7 is within the memory allowable range, the skip instruction signal S ₇ from the capacity monitoring unit 4 is not output.

Moreover, the write control unit 5, in order to skip instruction signal S ₇ at the time of determining whether to direct the writing of the 101 frames, second 104 frame to the memory section 7 is not input, the 101 frame The write instruction signal S ₁₁ is output to the memory unit 7 for the 104th frame. Accordingly, the 101st to 104th frames are sequentially stored in the storage areas (M + 1, M + 2, M + 3, M + 4) of the memory unit 7 without being skipped. Although the 103 frame is low operating frame, in order to skip instruction signal S ₇ at the time of determining whether an instruction to write the frame into the memory unit 7 is not input, to the 103 frame However, the write instruction signal S ₁₁ is output to the memory unit 7.

As described above, the write control unit 5 receives the skip instruction signal S ₇ when determining whether or not to instruct writing of a frame to the memory unit 7, and the frame is a low operation frame. If there is, the write instruction signal S ₁₁ is not output to the memory unit 7 for this low operation frame. In other words, when the write control unit 5 determines whether or not to instruct writing of a frame to the memory unit 7 when the skip instruction signal S ₇ is not input, or when the frame is not a low operation frame. The write instruction signal S ₁₁ is output to the memory unit 7 for this frame.

The read control unit 6 is an example of a read control unit. The read control unit 6 controls reading of frames from the memory unit 7 in synchronization with a synchronization signal of an output timing signal S ₁₇ described later. That is, the read control unit 6 puts a signal (indicated with information indicating a frame reading instruction from the memory unit 7 and a storage area instruction to be read for each frame of the video data S ₅ stored in the memory unit 7. Hereinafter, the read instruction signal S ₁₃ ) is output to the memory unit 7.

In this case, while the repeat instruction signal S ₈ is being input from the capacity monitoring unit 4, the read control unit 6 is identified based on a second identification signal S ₁₄ described later input from the memory unit 7. The read instruction signal S ₁₃ is output to the memory unit 7 again for the operation frame. Thus, the reading control unit 6, while the repeat instruction signal S ₈ is input from the capacity monitoring unit 4 reads out the low-action frame from the memory unit 7 again, to repeat low operating frame (reissued) process. Note that the frame other than the low action frame, rather than repeat the object of processing, the read instruction signal S ₁₃ again from the reading control unit 6 will not be output.

The second identification signal S ₁₄ is a signal on which low operation frame identification information is read together with the frame from the memory unit 7 by the read control unit 6. The second identification signal S ₁₄ is the same as the first identification signal S ₆ output from the motion amount detection unit 3 and input to the memory unit 7 although the timing of writing and reading is different. The same information is put on the frame.

Further, the reading control unit 6, the time for outputting the read instruction signal S _13, and outputs a read completion notification signal S ₁₅ carrying the information for notifying the completion of the reading capacity monitoring unit 4. However, the read completion notification signal S ₁₅ is not output to the capacity monitoring unit 4 in response to the read instruction signal S ₁₃ for the low operation frame (repeat frame) read again from the memory unit 7.

Further, the read control unit 6 stops reading the frame from the memory unit 7 based on the read stop instruction signal S ₉ input from the capacity monitoring unit 4. Further, the read control unit 6 starts reading frames from the memory unit 7 based on the read start instruction signal S ₁₀ input from the capacity monitoring unit 4.
Here, reading control of a frame from the memory unit 7 will be described with reference to FIG.

  Note that FIG. 2D is an explanatory diagram for explaining the read control for each frame for which the write control (skip process for the third frame only) shown in FIG.

For example, when the first frame, the second frame, the fourth frame, and the fifth frame are output from the memory unit 7 to the video processing unit 8, the repeat instruction signal S ₈ is input from the capacity monitoring unit 4 to the read control unit 6. Think if not. Consider a case where the repeat instruction signal S ₈ is input from the capacity monitoring unit 4 to the read control unit 6 when the 101st frame is output from the memory unit 7 to the video processing unit 8.

As shown in FIG. 2D, the read control unit 6 determines whether to instruct to read the first frame, the second frame, the fourth frame, and the fifth frame from the memory unit 7 again. in repeat instruction signal S ₈ is not being input. Therefore, readout control unit 6, the first frame, second frame, does not output the read instruction signal S ₁₃ again in the memory unit 7 with respect to the fourth frame and the fifth frame. Thus, the first frame, the second frame, the fourth frame, and the fifth frame are sequentially read from the storage areas (M, M + 1, M + 2, M + 3) of the memory unit 7 without being subjected to the repeat process.

The read control unit 6 receives the repeat instruction signal S ₈ when it determines whether to instruct to read the 101st frame from the memory unit 7 again. However, the reading control unit 6, for a 101 frame is not a low operating frame, does not output the read instruction signal S ₁₃ again in the memory unit 7. Similarly, the read control unit 6 does not output the read instruction signal S ₁₃ again to the memory unit 7 because the 102nd frame is not a low operation frame. As a result, the 101st frame and the 102nd frame are sequentially read from the storage areas (M + 1, M + 2) of the memory unit 7 without being subjected to the repeat process.

In contrast, the read controller 6, the repeat instruction signal S ₈ at the time of determining whether to instruct re-reading of the first 103 frames from the memory unit 7 is input, and the first 103 frames It is a low motion frame. For this purpose, the read control unit 6 outputs the read instruction signal S ₁₃ again to the memory unit 7. As a result, the 103rd frame is repeated, and is repeatedly read from the storage area (M + 3) of the memory unit 7 twice.

In addition, since the repeat control signal S ₈ is not input when the read control unit 6 determines whether or not to instruct to read the 104th frame again from the memory unit 7, does not output the read instruction signal S ₁₃ again in the memory unit 7. As a result, the 104th frame is read only once from the storage area (M + 2) of the memory unit 7 without being subjected to repeat processing. In the FIG. 2 (d), the after the 103 frame is repeating process, the memory unit 7 frame is written by the write instruction signal S ₁₁ by the write control unit 5, the free space of the memory unit 7 is reduced To do. Then, showing a state in which the free space of the memory unit 7 in order to become the memory tolerance, not output is repeat instruction signal S ₈ from the capacity monitoring unit 4.

As described above, the read controller 6, the repeat instruction signal S ₈ at the time of determining whether to instruct re-reading of the frame is input, and, if the frame is low action frame, the The read instruction signal S ₁₃ is output again to the memory unit 7 for the low operation frame. In other words, when the repeat control signal S ₈ is not input at the time of determining whether or not to instruct re-reading of the frame from the memory unit 7, the read control unit 6 performs the memory unit for this frame. 7 does not output the read instruction signal S ₁₃ again. Further, when the frame is not a low operation frame, the read control unit 6 does not output the read instruction signal S ₁₃ again to the memory unit 7 for this frame.

The memory unit 7 is an example of memory means. The memory unit 7 is discriminated for each frame acquired from each frame of the video data S ₅ input from the motion amount detection unit 3 and the first identification signal S ₆ input from the motion amount detection unit 3. And low-motion frame identification information.

Further, the memory unit 7 includes the input timing signal S ₂ in which the frame instructed to be written by the write instruction signal S ₁₁ from the write control unit 5 among the video data S ₅ input from the motion amount detection unit 3. Is written in the storage area designated by the write instruction signal S ₁₁ in synchronization with the synchronization signal. In this case, the frame to be written in the memory unit 7 is integrally written with the low operation frame identification information determined for this frame based on the first identification signal S ₆ input together with the frame. For example, when the memory unit 7 is a bank memory, when writing a frame in the storage area of the bank memory, to identify whether or not the frame is a low operation frame in the upper bits from the top (most significant bit) of the frame It is conceivable to give a flag of “1” for a low operation frame.

Further, the memory unit 7, the frame is instructed to read the read instruction signal S ₁₃ from the read control unit 6 in synchronization with the synchronization signal output timing signal S ₁₇ to be described later, is indicated by the read instruction signal S ₁₃ Read from the storage area. The frame read from the memory unit 7 is input to the video processing section 8 as the output image data S _16. In this case, the frame read from the memory unit 7 is read as a unit with the low motion frame identification information determined for this frame based on the second identification signal S ₁₄ read together with the frame. For example, when the memory unit 7 is a bank memory, when reading a frame from the storage area of the bank memory, it is determined whether or not the frame is a low operation frame that is given to higher bits than the head (most significant bit) of the frame. A flag for identification is read out.

The video processing unit 8 is an example of video processing means. The video processing unit 8 includes a clock oscillation circuit, and outputs a timing signal including a synchronization signal generated from the clock (hereinafter referred to as an output timing signal S ₁₇ ) to the read control unit 6 and the memory unit 7. Further, the video processing unit 8 outputs the output video data S ₁₆ input from the memory unit 7 synchronized with the output timing signal S ₁₇ to the outside as the output video signal S _OUT . Note that the video processing unit 8 receives the output video data S ₁₆ that has been transferred to the synchronization signal of the output timing signal S ₁₇ , and thus reliably processes the output video data S ₁₆ in accordance with the clock of the image processing apparatus 10. be able to.

The synchronous signal output timing signal S ₁₇ is a signal for transmitting the read timing of each frame reading controller 6 and the memory unit 7, the output timing signal S ₁₇ clocks, the read control unit 6 and the memory unit 7 This signal is generated periodically so that the operation is in tune.

Next, a processing procedure in the image processing apparatus 10 according to the present embodiment will be described with reference to FIGS. 3 and 4.
First, the image processing apparatus 10 is activated (step S1).

The threshold setting unit 2 outputs the first threshold signal S ₃ to the motion amount detection unit 3, sets the motion amount threshold value to the motion amount detection unit 3, and outputs the second threshold signal S ₄ to the capacity monitoring unit 4. Then, the threshold value and the memory allowable range are set in the capacity monitoring unit 4 (step S2).

Capacity monitoring unit 4 for storage of image data in the memory unit 7 is empty, and outputs the read stop instruction signal S ₉ to the read control unit 6 stops the reading frame from the memory unit 7 (step S3).

When the input video signal S _IN is input from the outside (Step S4), the video input unit 1 separates the input video data S ₁ and the input timing signal S ₂ included in the input video signal S _IN to obtain the input video signal. Data S ₁ is output to the motion amount detector 3. The video input unit 1, the motion amount detection unit 3 outputs the input timing signal S ₂ to the write control section 5 and the memory section 7 (step S5).

The motion amount detector 3 compares one frame (here, the first frame) of the input video data S ₁ with the next frame (here, the second frame), and determines the motion amount of the next frame. Is detected. Then, the motion amount detection unit 3 compares the detected motion amount with the motion amount threshold set by the threshold setting unit 2 to identify whether or not the next frame is a low motion frame (step) S6).

Then, the motion amount detection unit 3 synchronizes the next frame and the first identification signal S ₆ carrying the low motion frame identification information for this frame with the synchronization signal of the input timing signal S ₂ , and the memory unit 7 (step S7).

Further, the write control unit 5 outputs the write instruction signal S ₁₁ to the memory unit 7, whereby the next frame is stored in the designated storage area in the memory unit 7 together with the low operation frame identification information. (Step S8).
Then, the write control unit 5 outputs a write completion notice signal S ₁₂ to the capacity monitoring section 4 (step S9).

Capacity monitoring unit 4, based on the write completion notice signal S _12, or accumulated amount has increased one frame accumulation amount of the video data in the memory unit 7, it exceeds the threshold value of the set amounts of accumulated threshold setting unit 2 It is determined whether or not (step S10).

If it is determined in step S10 that the accumulated amount threshold has not been exceeded, the process returns to step S6, and steps S6 to S10 are repeated for the next frame.
Further, in step S10, if it is determined that exceeds the threshold value of accumulation amount, the capacity monitoring unit 4 outputs a read start instruction signal S ₁₀ to the read control section 6 (step S11).

The write control and the read control for the memory unit 7 are performed at the same time or at different timings corresponding to the write timing based on the synchronization signal of the input timing signal S ₂ and the read timing based on the synchronization signal of the output timing signal S _17. (Step S12).

First, write control will be described.
The motion detection unit 3 compares one frame of the input video data S ₁ (here, the next frame with respect to the frame when the threshold is exceeded in step S10) and the next frame with respect to this one frame. The amount of motion of one frame is detected. Then, the motion amount detection unit 3 compares the detected motion amount with the motion amount threshold value, and identifies whether one frame is a low motion frame (step S13).

Then, the motion amount detection unit 3 synchronizes the input timing signal S ₂ input from the video input unit 1 with one frame and the first identification signal S ₆ carrying the low motion frame identification information for this frame. The data is output to the memory unit 7 in synchronization with the signal. The motion amount detection unit 3, a first identification signal S ₆ carrying the low operating frame identification information for the frame, and outputs to the write control section 5 (step S14).
Then, the write control unit 5, based on the low operating frame identification information in the first identification signal S _6, it is determined whether one frame is low operating frame (step S15).

In step S15, when it is determined that the low operating frame, the write control unit 5, the skip instruction signal S ₇ determines whether or not input from the capacity monitoring unit 4 (step S16).

In step S16, if it is determined that the skip instruction signal S ₇ is input, the write control unit 5, without outputting a write instruction signal S ₁₁ to the memory unit 7, the flow returns to step S12. As a result, this low operation frame is skipped.

Further, in step S16, if the skip instruction signal S ₇ is determined not to be input, and in step S15, when it is determined that it is not the low operating frame, the write control section 5, the write instruction signal S ₁₁ is output to the memory unit 7 (step S17). Thereby, this one frame is stored in the memory unit 7 together with the low operation frame identification information.
Then, the write control unit 5 outputs a write completion notice signal S ₁₂ to the capacity monitoring unit 4 (step S18).

Based on the write completion notification signal S ₁₂ , the capacity monitoring unit 4 calculates a storage amount obtained by increasing the storage amount of the video data in the memory unit 7 by one frame, and subtracts the calculated storage amount from the storage capacity of the memory unit 7. As a result, the free capacity of the memory unit 7 is calculated. Then, the capacity monitoring unit 4 determines whether or not the calculated free capacity is below the allowable memory range set in the capacity monitoring unit 4 (step S19).

In step S19, if the free space is determined to be below the memory tolerance, capacity monitoring unit 4, to the write control unit 5 starts the output of the skip instruction signal S ₇ (Step S20), Step S12 Return to.

  If it is determined in step S19 that the free capacity does not fall below the allowable memory range, the capacity monitoring unit 4 determines whether the calculated free capacity is within the allowable memory range set in the capacity monitoring unit 4. Is determined (step S21).

In step S21, if the free space is determined to be within the memory tolerance, capacity monitoring unit 4, if being output the repeat instruction signal S ₈ to the reading control unit 6 stops the repeat instruction signal S ₈ (Step S22), the process returns to step S12.
If it is determined in step S21 that the free space is not within the memory allowable range, the process returns to step S12.

Next, read control will be described.
The read control unit 6 outputs the read instruction signal S ₁₃ to the memory unit 7 in synchronization with the synchronization signal of the output timing signal S ₁₇ input from the video processing unit 8, and is stored in the designated storage area. Read the frame. Further, the reading control unit 6 reads out put low operating frame identification information for the one frame to the second identification signal S ₁₄ (step S23).

The output image data S ₁₆ including a read frame, as the output video signal S _OUT, output from the video processor 8 to the outside (step S24).
Further, the reading control unit 6 outputs a read completion notification signal S ₁₅ to the capacity monitoring unit 4 (step S25).

Then, the reading control unit 6, the repeat instruction signal S ₈ to determine whether or not input from the capacity monitoring unit 4 (step S26).
In step S26, if it is determined that the repeat instruction signal S ₈ is input, the process proceeds to step S31 to be described later.

Further, in step S26, if the repeat instruction signal S ₈ is determined not to be input, the capacity monitoring unit 4, based on the read completion notification signal S _15, one frame accumulation amount of the video data in the memory unit 7 Calculate the reduced accumulated amount. Then, the capacity monitoring unit 4 subtracts the calculated storage amount from the storage capacity of the memory unit 7 to calculate the free capacity of the memory unit 7, and the calculated free capacity is stored in the memory set in the capacity monitoring unit 4. It is determined whether or not the allowable range is exceeded (step S27).

  When it is determined in step S27 that the free capacity does not exceed the memory allowable range, the capacity monitoring unit 4 determines whether the calculated free capacity is within the memory allowable range set in the capacity monitoring unit 4. (Step S28).

In step S28, if the free space is determined to be within the memory tolerance, capacity monitoring unit 4, if being output skip instruction signal S ₇ to the write control unit 5, stops the skip instruction signal S ₇ Then (step S29), the process returns to step S12.

If it is determined in step S28 that the free space is not within the memory allowable range, the process returns to step S12.
Further, in step S27 described above, if the free space is determined to exceed the memory tolerance, capacity monitoring unit 4, to the read control section 6 starts outputting the repeat instruction signal S ₈ (step S30) .
Then, the reading control unit 6, based on the low operating frame identification information of the second identification signal S _14, it is determined whether one frame is low operating frame (step S31).

If it is determined in step S31 that the frame is not a low operation frame, the process returns to step S12.
Further, in step S31, when it is determined that the low operating frame, the read control unit 6, against the one frame, and outputs the read instruction signal S ₁₃ again in the memory unit 7, the specified storage area One frame stored in is read (step S32).
The output image data S ₁₆ including a read again frame, as the output video signal S _OUT, output from the video processor 8 to the outside (step S33), the flow returns to step S12.

The input is completed the input video signal S _IN from the outside, with the write target frame into the memory unit 7 is eliminated, there is no read target frame from the memory unit 7, the output video signal S _OUT to the outside When the output ends, the processing by the image processing apparatus 10 ends.

  As described above, the image processing apparatus 10 according to the present embodiment performs frame skip processing or repeat processing, which is generated by changing the synchronization signal of a video signal, as a “low motion frame that is a frame with a small change in the video scene. "Is processed. As a result, the image processing apparatus 10 can output an image that does not make the viewer feel the frame skip or repeat despite the occurrence of the frame skip or repeat.

  In particular, the image processing apparatus 10 according to the present embodiment skips “low motion frames” when the free space of the memory unit 4 or the amount of video data stored in the memory unit 4 is outside a predetermined allowable range. Processing or repeat processing is performed. For this reason, not all of the “low motion frames” are skipped or repeated, but the video can be reproduced smoothly by skipping or repeating the minimum necessary frames. There is an effect.

  In the image processing apparatus 10 according to the present embodiment, the case where the memory allowable range is a fixed range set in advance has been described, but the memory allowable range may be one of an upper limit value and a lower limit value as necessary. May be changed.

Specifically, as shown in FIG. 5, the capacity monitoring unit 4 changes the free capacity of the memory unit 7 or the storage amount of video data in the memory unit 7 every time the value of the free capacity or storage amount is updated. A capacity notification signal S ₁₈ to be notified is output to the threshold setting unit 2. Thereby, the threshold value setting unit 2 can recognize the free space of the memory unit 7 or the amount of video data stored in the memory unit 7 in real time. The threshold setting unit 2, based on the volume notification signal S _18, is adjusted so as to shift the memory tolerance.

  Here, when skip processing is performed on a frame, the speed (frame speed) of the video signal before switching to the synchronization signal generated inside the image processing apparatus 10 is faster than the speed of the video signal after switching. To occur. That is, the skip process is performed so that the number of frame writes to the memory unit 7 exceeds the number of read times, and the free space of the memory unit 7 is not short.

Therefore, the threshold value setting section 2, recognized that based on the available capacity or the accumulation amount of information obtained from the capacitance notification signal S _18, the accumulation amount of the video data in the memory unit 7 is full of state are frequently it can. If the threshold setting unit 2 can recognize that the full state frequently occurs, the threshold setting unit 2 raises the lower limit value of the memory allowable range for the free capacity of the memory unit 7 or the memory allowable range for the storage amount of video data in the memory unit 7. Set to lower the upper limit. In the threshold setting unit 2, the output frequency of the skip instruction signal S ₇ is reduced more than necessary by raising the lower limit value of the memory allowable range or lowering the upper limit value.

  In addition, when the repeat process is performed on the frame, it occurs because the speed of the video signal after switching to the synchronization signal generated inside the image processing apparatus 10 is faster than the speed of the video signal before switching. In other words, the repeat process is performed so that the number of times the frame is read from the memory unit 7 exceeds the number of writes, and the amount of video data stored in the memory unit 7 is not short.

Therefore, the threshold value setting section 2 based on the information obtained from the capacitance notification signal S _18, if recognizing that the accumulation amount of the video data is empty state frequently occurs in the memory unit 7, the repeat instruction signal S The output frequency of ₈ is considered to be less than necessary. On the other hand, the threshold value setting unit 2 lowers the upper limit value of the memory allowable range with respect to the free space of the memory unit 7 or increases the lower limit value of the memory allowable range with respect to the amount of video data stored in the memory unit 7.

As described above, the image processing apparatus 10 according to this embodiment, by changing as needed the value of either the upper limit value and the lower limit value of the memory tolerance skip instruction signal S ₇ and the repeat instruction signal S The output frequency of ₈ can be adjusted, and the free capacity of the memory unit 7 can be kept substantially constant.

[Appendix] The following appendices are further disclosed with respect to the embodiment including the above examples.
(Supplementary Note 1) In an image processing apparatus that processes video data, a motion amount detection unit that detects a frame in which the motion amount between frames of the video data is lower than a first threshold, and the input video data in units of frames A writing unit for writing to the memory, a reading unit for reading the video data from the memory in units of frames, and an instruction to limit the writing of the low frame to the memory based on the free capacity or the accumulation amount of the memory A capacity monitoring unit that outputs an instruction signal to the writing unit, and the writing unit restricts writing of the low frame to the memory based on the instruction signal. .

(Supplementary Note 2) In the image processing apparatus according to Supplementary Note 1, in the case where the capacity monitoring unit has a free capacity of the memory exceeding a predetermined allowable range, or a storage amount of video data in the memory is a predetermined allowable range. The repeat instruction signal for instructing to read again the low operation frame from the memory is output to the read control unit, and the read control unit is configured to output the low operation frame based on the presence or absence of the repeat instruction signal. An image processing apparatus for determining whether or not to read again from the memory.

(Supplementary Note 3) In the image processing apparatus according to Supplementary Note 1 or 2, the capacity monitoring unit inputs a write completion notification signal for notifying completion of writing of the frame to the memory from the writing unit, A read completion notification signal for notifying completion of reading of a frame from the memory is input from the reading unit, and based on a difference between the number of times the write completion notification signal and the read completion notification signal are input, the free space of the memory or the memory An image processing apparatus for calculating the amount of video data stored in the computer.

(Supplementary Note 4) In the image processing apparatus according to any one of Supplementary Notes 1 to 3, when the amount of video data stored in the memory is empty, the capacity monitoring unit stops reading frames from the memory. When the amount of video data stored in the memory exceeds a predetermined threshold from the state in which reading of frames from the memory is stopped, the readout stop instruction signal for instructing is output from the memory. The readout start instruction signal for instructing the start of reading of the frame is output to the readout unit, and the readout unit controls the readout of the frame from the memory based on the readout stop instruction signal and the readout start instruction signal Processing equipment.

(Supplementary Note 5) In the image processing apparatus according to any one of Supplementary Notes 1 to 4, the image processing apparatus includes a threshold setting unit that stores information indicating the predetermined allowable range, and the capacity monitoring unit includes a free space in the memory unit. Alternatively, the image processing apparatus outputs a capacity notification signal for notifying an accumulation amount of video data in the memory unit to the threshold setting unit, and the threshold setting unit adjusts the allowable range based on the capacity notification signal.

(Supplementary Note 6) In an image processing method for processing video data, a motion amount detection step of detecting a frame in which a motion amount between frames of the video data is lower than a first threshold value; and the input video data in units of frames A writing step for writing to the memory, a reading step for reading the video data from the memory in units of frames, and an instruction to limit the writing of the low frame to the memory based on the free capacity or accumulation amount of the memory A capacity monitoring step of outputting an instruction signal to the writing step, and the writing step restricts writing of the low frame to the memory based on the instruction signal. Method.

(Supplementary Note 7) In a control program for causing a computer of an image processing apparatus that processes video data to function, the computer detects a motion amount detection unit that detects a frame in which the motion amount between frames of the video data is lower than a first threshold value; Write means for writing the input video data in frame units into the memory, read out means for reading out the video data from the memory in frame units, and writing to the memory based on the free capacity or storage amount of the memory An instruction signal for instructing restriction on writing of the low frame is caused to function as a capacity monitoring means for outputting to the writing means, and the writing means restricts writing of the low frame to the memory based on the instruction signal. A control program characterized by:

DESCRIPTION OF SYMBOLS 1 Video input part 2 Threshold setting part 3 Motion amount detection part 4 Capacity | capacitance monitoring part 5 Write control part 6 Read control part 7 Memory part 8 Video processing part 10 Image processing device S ₁ Input video data S ₂ Input timing signal S ₃ 1st 1 threshold signal S ₄ second threshold signal S ₅ video data S ₆ first identification signal S ₇ skip instruction signal S ₈ repeat instruction signal S ₉ read stop instruction signal S ₁₀ read start instruction signal S ₁₁ write instruction signal S ₁₂ Write completion notification signal S ₁₃ Read instruction signal S ₁₄ Second identification signal S ₁₅ Read completion notification signal S ₁₆ output video data S ₁₇ output timing signal S ₁₈ capacity notification signal S _IN input video signal S _OUT output video signal

Claims (4)

In an image processing apparatus that processes video data,
A motion amount detection unit for detecting a frame in which a motion amount between frames of the video data is lower than a first threshold;
A writing unit that writes the input video data in units of frames to a memory;
A reading unit that reads the video data from the memory in units of frames;
A threshold setting unit for storing information indicating a predetermined allowable memory range;
Based on the free space or accumulated amount of the memory, an instruction signal for instructing the restriction of writing of the low frame to the memory is output to the writing unit, and the free space of the memory or the video data in the memory is output. A capacity monitoring unit that outputs a capacity notification signal for notifying the accumulation amount to the threshold setting unit ;
With
The threshold setting unit adjusts the memory allowable range based on the capacity notification signal,
The image processing apparatus, wherein the writing unit restricts writing of the low frame to the memory based on the instruction signal. The image processing apparatus according to claim 1,
When the free capacity of the memory exceeds a predetermined memory allowable range, or when the amount of video data stored in the memory is lower than a predetermined memory allowable range, the capacity monitoring unit detects a low operation frame from the memory. outputs the repeat instruction signal instructing to again read the read out unit,
The image processing apparatus in which the read out unit, based on the presence or absence of the repeat instruction signal, it is determined whether re-read the lower operating frame from the memory.
In an image processing method executed by an image processing apparatus that processes video data,
A motion amount detection step of detecting a frame in which the motion amount between frames of the video data is lower than a first threshold;
A writing unit provided in the image processing apparatus, a writing step of writing the input video data in units of frames into a memory;
A step of reading out the video data from the memory in units of frames;
Based on the free space or accumulated amount of the memory, an instruction signal for instructing the restriction of writing of the low frame to the memory is output to the writing unit, and the free space of the memory or the video data in the memory is output. A capacity monitoring step for outputting a capacity notification signal for notifying the accumulation amount to a threshold setting unit for storing information indicating a predetermined memory allowable range ;
The threshold setting unit adjusting the memory allowable range based on the capacity notification signal ;
Have
The image processing method, wherein the writing step limits writing of the low frame to the memory based on the instruction signal. In a control program for causing a computer of an image processing apparatus that processes video data to function ,
From the memory, a motion amount detecting means for detecting a frame in which the motion amount between frames of the video data is lower than a first threshold, a writing means for writing the input video data in units of frames into the memory, A reading means for reading out the video data in units of frames, a threshold setting means for storing information indicating a predetermined memory allowable range, and the low frame of the memory to the memory based on a free capacity or an accumulation amount of the memory. Capacity monitoring means for outputting an instruction signal for instructing the limitation of writing to the writing means, and for outputting a capacity notification signal for notifying the free capacity of the memory or the amount of video data stored in the memory to the threshold setting means ; To function ,
The threshold setting means adjusts the memory allowable range based on the capacity notification signal;
The control program , wherein the writing means limits writing of the low frame to the memory based on the instruction signal.

Images