JP4010742B2 - Moving image playback display device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving image reproduction display device that decodes and reproduces a moving image encoded using digital technology.
[0002]
[Prior art]
In general, in a moving image encoding process in a moving image reproduction display device, an I-frame encoded based on information in a frame, and a P-frame encoded with difference information between frames using motion compensation Is used. P-frames are encoded by extrapolation from past frames, but B-frames that contain I-frames and P-frames may be used.
[0003]
When moving image data encoded as described above is stored in a file, reproduction of this image data must be performed from the position of the I-frame. This is because a P-frame consists of difference information from a past frame, so that the frame cannot be completely reproduced only from that information.
[0004]
For example, Japanese Patent Laid-Open No. 7-203416 discloses a technique for realizing fast forward and rewind as well as normal reproduction of encoded moving image data. In this moving image distribution method, fast-forward and rewind are realized by quickly moving the playback position to the position of the I-frame in the image data.
[0005]
[Problems to be solved by the invention]
However, when playing back video data in which only P-frames for a long time (for example, 5 minutes) are recorded continuously after I-frames, the scene once viewed is returned for a short time (for example, 10 seconds). If you try to play it, it will not play properly.
[0006]
The present invention provides a moving image reproduction display device that enables rewinding and fast-forwarding a short time before even when only a P-frame continues for a long time.
[0007]
[Means for Solving the Problems]
  The moving image reproduction / display apparatus according to the invention of claim 1 reproduces and displays encoded moving image data in which an I-frame including information on the entire frame is followed by a P-frame comprising difference information between the frames. In a moving image reproduction display apparatus, input control means for controlling input of encoded moving image data, and decoding means for decoding each frame from encoded moving image data sent from the input control means A frame recording means for adding a frame number and a time code to the frame decoded by the decoding means and recording a position in the moving image data as information of the entire frame; and a frame number and a time code of the frame to be reproduced. On the basis of the information of the whole corresponding frame from the frame recording means and set in the decoding means, and Reproduction control means for setting the input position of the moving image data in the input control means, and display means for displaying the frame decoded by the decoding means;The frame recording means has a two-stage configuration having two ring buffers for recording new frames after the predetermined number of frames to be recorded is overwritten. The frame recording period of the second stage ring buffer is longer than the frame recording period of the first stage ring buffer.
[0008]
  In the above aspect,Each frame is decoded by the decoding means from the encoded moving image data sent from the input control means, and the frame number and time code are added to the decoded frame, and the position in the moving image data is the information on the entire frame. Is recorded in the frame recording means. Based on the frame number and time code of the frame to be reproduced, the reproduction control means takes out the information of the entire corresponding frame from the frame recording means and sets it in the decoding means, and sets the input position of the moving image data in the input control means. The display means displays the frame decoded by the decoding means.
[0010]
  Also,In the ring buffer of the frame recording means, after reaching a predetermined number of frames to be recorded, a new frame is recorded on the first recorded frame.
[0012]
  further,Frames are recorded in the first-stage ring buffer at the same frame recording cycle as in claim 2, and frames are recorded in the second-stage ring buffer at a frame recording cycle longer than that of the first-stage ring buffer. Record. Therefore, many frames can be recorded in the second stage ring buffer in inverse proportion to the frame recording period.
[0015]
  The moving image reproduction display device according to the invention of claim 2 is the invention of claim 1,Second decoding for decoding moving image data encoded prior to the decoding means and transmitting the decoded frame, frame number, time code, and position in the moving image data to the frame recording means The frame recording means records the frame received from the second decoding means in the second stage ring buffer, and the frame from the decoding means is recorded in the first stage ring buffer. It is characterized by recording.
[0016]
  In the above aspect, the claim 1In addition to the operation of the invention, the second decoding means decodes the moving image data encoded prior to the decoding means, and in the decoded frame, frame number, time code, and moving image data The position is transmitted to the frame recording means. The frame recording means records the frame received from the second decoding means in the second stage ring buffer, and records the frame from the decoding means in the first stage ring buffer.
[0019]
  Claim3A moving image reproduction display device according to the present invention is as follows.2In the invention ofFrame display means for displaying a list of frames recorded in the frame recording means;Moving object detection means for determining whether the image in the block is stationary or moving for each block of the frame decoded by the second decoding means, and detecting the presence of the moving object from the static determination result; A frame recording timing determination unit that records information of the entire frame in the frame recording unit at a predetermined timing when the moving object is detected by the moving object detection unit; In addition to the ring buffer, a temporary recording ring buffer for temporarily and temporarily recording the frame decoded by the second decoding unit is provided, and the frame at the timing determined by the frame recording timing determination unit and the temporary The second stage ring buffer stores a frame that is a predetermined time before the frame recorded in the recording ring buffer. The frame display means displays the frame at the timing determined by the frame recording timing determination means, and the reproduction control means displays a frame a predetermined time before the timing determined by the frame recording timing determination means. The playback from is set.
[0020]
  In the above aspect, in addition to the operation of the invention of claim 2,The moving object detection unit determines whether the image in the block is stationary or moving for each block of the frame decoded by the second decoding unit, and detects the presence of the moving object from the static determination result. The frame recording timing determining means records information of the entire frame on the frame recording means at a predetermined timing when the moving object is detected by the moving object detecting means.
[0022]
  AlsoThe frame recording means records the frame at the timing determined by the frame recording timing judging means and the frame that is a predetermined time before the frame recorded in the temporary recording ring buffer in the second stage ring buffer, and displays the frame display. The means displays the frame at the timing determined by the frame recording timing determination means, and the reproduction control means sets the reproduction from the frame a predetermined time before the timing determined by the frame recording timing determination means.
[0023]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below. FIG. 1 is a block diagram of a moving image reproduction display device according to the first embodiment of the present invention. The encoded video data is recorded in the file 100, and the input control means 101 reads the encoded video data from the file 100 in accordance with the file position information from the reproduction control means 104. The decoding unit 102 decodes the encoded moving image data sent from the input control unit 101 and restores the frame.
[0024]
The frame storage means 103 adds a frame number and a time code to the frame decoded by the decoding means 102 and records the position in the moving image data as information of the entire frame. Further, the reproduction control means 104 extracts information on the entire corresponding frame from the frame recording means 103 based on the frame number and time code of the frame to be reproduced, and sets and inputs the information on the entire frame to the decoding means 102. A moving image data input position is set in the control means 101. Then, the frame decoded by the decoding unit 102 is displayed on the display unit 105.
[0025]
FIG. 2 is a block configuration diagram of the decoding unit 102.
The encoded moving image data sent from the input control means 101 is first stored once in the input buffer 201 of the decoding means 102 and sent to the variable length decoding unit 202, where the variable length of information of each syntax Decode the code.
[0026]
In the variable length decoding unit 202, if the frame mode is I (I-frame), the mode changeover switch 207 is turned off to perform decoding. The quantized DCT coefficient information decoded by the variable length decoding unit 202 is input to the inverse quantization unit 203 and inversely quantized. Then, the IDCT unit 204 performs inverse discrete cosine transform processing to generate a reproduced image signal. The reproduced image signal is stored as a reference image in the frame memory 208, and is output to the outside (the frame storage unit 103 and the display unit 105) via the output control unit 209.
[0027]
On the other hand, if the frame mode is P (P-frame), the processing differs for each macroblock. At this time, if the mode of the macroblock is INTRA, the mode changeover switch 207 is selected to be off, and the quantized DCT coefficient information decoded by the variable length decoding unit 202 is dequantized by the inverse quantization unit 203. Then, the IDCT unit 204 performs an inverse discrete cosine transform process to generate a reproduced image signal. The reproduced image signal is stored as a reference image in the frame memory 208 and is output to the outside through the output control unit 209.
[0028]
If the mode of the macroblock is INTER and NOT_CODED, the mode changeover switch 207 is selected to be on, and the quantized DCT coefficient information decoded by the variable length decoding unit 202 is dequantized by the inverse quantization unit 203. The IDCT unit 204 performs inverse discrete cosine transform processing, and based on the motion vector information decoded by the variable-length decoding unit 202, the motion compensation unit 205 performs motion compensation, and the adder 206 adds them. A reproduction image signal is generated. The reproduced image signal is stored as a reference image in the frame memory 208, and is output to the outside (the frame storage unit 103 and the display unit 105) via the output control unit 209.
[0029]
The output control unit 209 usually displays the position in the file of the encoded data of the frame that accompanies the reproduced image signal, the frame number, and the time code that represents the elapsed time from the first frame in milliseconds. Output to the frame recording means 103.
[0030]
Note that the variable length decoding unit 202, the inverse quantization unit 203, the IDCT unit 204, the adder 206, and the motion compensation unit 205 in the decoding unit 102 are sequentially processed, and the time of the output destination display unit 105 as described later. The output may be blocked by the control. When the output is blocked by the time control of the display unit 105, the entire process of the decoding unit 102 is blocked. The output control unit 209 also has a function of omitting output to the display unit 105. In this case, since the process is not blocked, the decoding unit 102 can perform decoding at high speed. Using this mechanism, the decoding means 102 performs a real-time mode for decoding in real time according to the time control of the display means 105, and a high-speed mode for decoding at high speed without being restricted by the time control of the display means 105. Realize.
[0031]
FIG. 3 is a block configuration diagram of the display means 105. The reproduced image signal sent from the decoding means 102 is input to the video buffer 220 and displayed on the display 221. When receiving the playback start signal from the playback control unit 104, the time control unit 222 records the time code of the frame that has arrived first from the decoding unit 102, and then the time code of the frame to be displayed and the playback start. The input to the video buffer 220 is controlled so that the difference from the time code of the hour frame does not exceed the time of the timer 106. When the input to the video buffer 220 is controlled, the output of the decoding means 102 is suppressed, and the decoding speed is adjusted to the reproduction speed. The timer 223 is a clock with a unit of 1 millisecond, and is reset to 0 when receiving a playback start signal from the playback control means 104, and then counts up in real time.
[0032]
Next, the frame recording means 103 converts the reproduced frame output from the decoding means 102 at a preset time interval, for example, 1 second, into a frame number, an accompanying time code, and a position in the moving image data. Record with.
[0033]
FIG. 4 is an explanatory diagram of a record recorded in the frame recording means 103. The record includes a frame number 301, a time code 302, a position 303 in the moving image data, and a frame 304. A plurality of records are recorded in the frame storage means 103. The reason why the time code 302 is recorded in addition to the frame number 301 as a record is to cope with the case where the time code 302 is not proportional to the frame number 301. Further, a pointer 305 for recording the position of the last record is provided. Among these, the frame number 301, the time code 302, and the position 303 in the moving image data are 4-byte integer data, and the frame 304 is image data in the YUV 4: 2: 0 format.
[0034]
When the frame 304 has a CIF size of 354 pixels × 288, the size of the frame 304 is 203904 bytes, and the size of the record is 203916 bytes. Therefore, when the frame 304 is recorded for 600 seconds at intervals of 1 second, the size becomes about 122 MByte, and the frame recording means 103 can be configured using a general hard disk or main memory. Further, the length of each frame 304 is constant, and random access to the top of the record is easy.
[0035]
Note that the frame recording unit 103 does not record a frame 304 having a time code 302 that is earlier than the latest time code 302 among the recorded records when the reproduced frame 304 is rewound and reproduced.
[0036]
The frame recording means 103 has a function of searching for a corresponding record using the frame number 301 and the time code 302 (time) as keys. FIG. 5 shows search processing when a search request for a frame 304 is received from a playback control means 104 (to be described later) using a time code 302 (time) as a key.
[0037]
In FIG. 5, when the process is started, it is determined whether the designated time is before or after the time code 302 of the last record (S1). If the time after the time code 302 of the last record is designated, failure is returned and the process is terminated (S2). Otherwise, the first frame number 0 and the last record number are entered in the two variables Before and After, respectively (S3).
[0038]
Then, it is determined whether After-Before is 1 or less (S4). If it is 1 or less, Before is returned and the process is terminated (S5). On the other hand, if it is greater than 1, (After + Before) / 2 is set to f (S6). In this case, for example, rounding is performed so that f is an integer.
[0039]
Next, the time code of f is compared with the given target time (S7). If the time code of f is smaller than the target time, f is entered in Before (S8), and the process returns to step S4. If the time code of f is larger than the target time, f is set in After (S9), and the process returns to step S4. If the time code of f matches the target time, f is returned and the process is terminated (S10).
[0040]
For example, when the last record number is “4” and the target time is 300, the time code of the last record is 400, and the designated time is 300. Therefore, the designated time is the time code of the last record. Since it is earlier, 0 is set to Before and 4 is set to After in step S3.
[0041]
In step S4, After-Before (= 4) is larger than 1, f (= 2) is calculated in step S6. Since the time code of f (= 2) is 200 and is smaller than the target time 300, 2 is set in Before in step S8, and the process returns to step S4.
[0042]
In the same manner, since After is 4 and Before is 2, f (= 3) is calculated in step S6. Since the time code of f (= 3) is 300, which is equal to the target time 300, in step S10 Returns f (= 2). That is, the record number “2” is returned.
[0043]
As described above, the record number having the corresponding time code is searched by the 2-minute search after step S4, and the result is returned at step S5 or step S10. When returning the result in step S5, the record number of the latest frame before the specified time is returned.
[0044]
Next, the reproduction control unit 104 controls the entire moving image reproduction display device. During normal reproduction, the reproduction control unit 104 sends the beginning of the file as the file reading position to the input control unit 101 and decodes it. The clearing signal of the frame memory 208 is sent to the converting means 102. Further, the display means 105 is notified of the start of reproduction. A clear signal for data recorded therein is sent to the frame recording means 103. Thereby, the reproduction operation is started.
[0045]
On the other hand, when reproducing from a frame at a specific time, the reproduction control means 104 inquires the frame recording means 103 for a record having a corresponding time code. When the corresponding record is found by the processing of FIG. 5, the position 303 in the moving image data of the record is sent to the input control means 101, the frame 304 of the record is written in the frame memory 208 of the decoding means 102, and the display means 105 Informs the start of playback. As a result, the input control means 101 reads the P-frames after the designated time thereafter, but the contents of the frame memory 208 are normally the same as the state in which the P-frames at the same time were first reproduced. Played.
[0046]
Here, when a frame corresponding to the given time is not found and failure is returned in step 2 shown in FIG. 5, the target time is sent to the decoding means 102 and the decoding means 102 is operated at high speed. Set to mode and proceed with decryption up to the target time. When the decoding unit 102 finishes decoding until the target time, the decoding unit 102 returns to the real time mode, and notifies the display unit 105 of the start of reproduction through the reproduction control unit 104.
[0047]
As described above, since the frame storage means 103 has a function of searching for a record, quick rewind playback and fast forward playback are possible.
[0048]
FIG. 6 is an explanatory diagram of the frame recording means 103 when a ring buffer is adopted as the frame recording means 103. Since the storage capacity of the frame recording means 103 has a limit, it is not possible to record a record exceeding the limit.
[0049]
Therefore, when the maximum number of record records 600 of the frame recording means 103 is exceeded, the 601st frame (frame number 6000) is recorded in the 0th record and recorded in the 602th frame (frame number 6010). Thereafter, records are counted up in the same manner. In this way, the recording is made into a ring shape. Thus, the past 600 frames are recorded by erasing the most recently recorded frame. When the ring buffer is employed, the first record number is one greater than the last record number, and therefore the process in FIG. 5 needs to be changed. Specifically, the process of step S3 is changed as follows.
[0050]
Last + 1 → Before
Last + buffer size → After
Furthermore, when referring to a record using the temporary record number calculated in the process of FIG. 5, the remainder obtained by dividing the temporary record number by the buffer size is set as the record number. By adopting this ring buffer, it is possible to rewind up to 600 seconds before the frame being played back.
[0051]
Next, when rewinding is possible from 600 frames before the frame being played back, the ring buffer is duplicated as shown in FIG. FIG. 7A shows the first ring buffer, and FIG. 7B shows the second ring buffer. The first ring buffer is the same as the ring buffer shown in FIG. 6, but a second ring buffer having the same structure is prepared.
[0052]
Similar to the first ring buffer, a second ring buffer indicated by a frame number 601, a time code 602, a position 603 in moving image data, a frame 604, and a pointer 605 is prepared, and 1/10 of the first ring buffer. Record at a period, that is, every 10 seconds. Since the period is 1/10, 10 times the time can be recorded with the same storage capacity as the first ring buffer. When the rewinding time is designated, the frame recording means 103 first searches the first ring buffer, and if not found, searches the second ring buffer. When a frame at the target time is found in the second ring buffer, the playback control means 104 initializes the first ring buffer and starts playback.
[0053]
In this way, it is possible to rewind to the past 10 times using the frame recording means 103 having a recording capacity twice that of the case where a single ring buffer is used. Of course, although the accuracy of the time that can be set in the second ring buffer is lowered, there is no practical problem because the time sense of human becomes ambiguous as it goes back in the past. The user can also specify the recording timing to the second ring buffer.
[0054]
Next, a second embodiment of the present invention will be described. FIG. 8 is a block diagram of a moving image reproduction display device according to the second embodiment of the present invention. In the second embodiment, a second decoding unit 700 is added to the first embodiment shown in FIG. 1, and the second decoding unit 700 includes a decoding unit. The moving image data encoded prior to 102 is decoded, and the decoded frame, frame number, time code, and position in the moving image data are transmitted to the frame recording means 103. . Since the other configuration is the same as that of the first embodiment shown in FIG. 1, the same components are denoted by the same reference numerals, and redundant description is omitted.
[0055]
The second decoding unit 700 decodes the image data sent from the input control unit 101 at a high speed without being restricted by the time control of the display unit 105. The internal structure of the second decoding unit 700 is the same as that of the first decoding unit 102. Prior to the frame sent from the decoding unit 102, the frame recording unit 103 records the frame from the second decoding unit 700 at a constant period.
[0056]
When the frame recording means 103 having a two-stage ring buffer is used, the frame from the first decoding means 102 is transferred to the first-stage ring buffer of the frame recording means 103 and from the second decoding means 700. Are recorded in the second-stage ring buffer.
[0057]
Accordingly, since the decoded frame can be recorded in the frame recording unit 103 prior to the decoding unit 102 that performs the decoding in real time, it is possible to quickly forward.
[0058]
Next, a third embodiment of the present invention will be described. FIG. 9 is a block diagram of a moving image reproduction display apparatus according to the third embodiment of the present invention. This third embodiment is different from the second embodiment shown in FIG. 8 in that a frame display means 800 for displaying a list of frames recorded in the frame recording means 103 is provided. The frame recording means 103 is a frame display means. When a frame displayed on 800 is designated, a record related to the frame is searched, and the playback control means 104 plays back from the searched frame. Since the other configuration is the same as that of the second embodiment shown in FIG. 8, the same elements are denoted by the same reference numerals, and redundant description is omitted.
[0059]
The frame display means 800 has a display similar to the display means 105. As shown in the screen example of FIG. 10, the frame display means 800 takes out the frames recorded at a constant period from the frame recording means 103 and displays a list on the display. . When the user designates a frame on the display with a pointing device such as a mouse, the frame display means 800 takes out the record corresponding to the designated frame from the frame recording means 103, the frame number 301 recorded in the record, the time The code 302 and the position 303 in the moving image data are transmitted to the reproduction control means 104, and the reproduction control means 104 reproduces and displays the moving image based on the received information.
[0060]
Note that when displaying a list on the display, the display may be reduced so that as many frames as possible can be displayed in a limited display area of the display. Further, the frame display means 800 may share and use the display 221 of the display means 105.
[0061]
Next, FIG. 11 is obtained by adding moving object detection means H101 and frame recording timing determination means H102 to the second encoding means 700 shown in FIGS.
[0062]
The moving object detection unit H101 includes a macroblock static motion determination unit H111, a moving object determination unit H113, and a subtracter H112. The macro block static determination unit H111 of the moving object detection unit H101 determines whether the image in the block is stationary or moving for each block of the frame, and the moving object determination unit H113 moves from the static determination result. Detect the presence of an object. When a moving object is detected by the moving object detection unit H101, the frame recording timing determination unit H102 records information on the entire frame in the frame recording unit 103 via the output control unit 209 at a predetermined timing.
[0063]
In the moving object detection means H101, the process shown in FIG. 12 is performed. That is, the macroblock static determination unit H111 performs macroblock static determination for each frame (S101), and the moving object determination unit H113 performs moving object determination (S102).
[0064]
In the macroblock static determination means H111, the mode information, motion vector information, DCT coefficient information decoded by the variable length decoding unit 202 of the second decoding means 700, and the playback at the time of the INTRA macroblock obtained by the subtractor H112 From the difference absolute value sum SAD between the image signal and the frame memory of the previous frame, static motion determination is performed for each macroblock in the frame.
[0065]
FIG. 13 is a flowchart showing specific contents of the macroblock static determination process (S101). Here, i and j represent the addresses of the vertical and horizontal macroblocks in the frame, respectively, and V_NMB and H_NMB represent the numbers of vertical and horizontal macroblocks in the frame. The two-dimensional array M [i] [j] is an array for storing information indicating whether each macro block is a dynamic macro block. If TRUE, a dynamic macro block is indicated, and if it is FALSE, a static macro block is indicated.
[0066]
Steps S201 to S209 (LOOP1) mean that i = 0 to V_NMB-1 is repeated, and steps S202 to S208 (LOOP2) mean that j = 0 to H_NMB-1 is repeated.
[0067]
First, the mode information MODE from the variable length decoding unit 202 of the second decoding unit 700 is determined for each macroblock (S203). If MODE is INTRA, the difference absolute value sum SAD between the reproduced image signal of the macroblock and the frame memory 208 one frame before is calculated and compared with the threshold value T0 (S204). If it is larger than the threshold value T0, it is determined that the macroblock is a moving macroblock, and TRUE is substituted for M [i] [j] (S206). On the other hand, if it is equal to or less than the threshold value T0, the macroblock is determined to be a static macroblock, and FALSE is assigned to M [i] [j] (S207).
[0068]
If MODE is INTER in the determination of step S203, the absolute value sum Σ | MV | of the motion vector of the macroblock and the absolute value of the DCT coefficient are obtained from the motion vector information and DCT coefficient information from the variable length decoding unit 202. The sum Σ | COF | is calculated and compared with the respective threshold values T1 and T2 (S205). If the absolute value sum of the motion vectors and the absolute value sum of the DCT coefficients are smaller than the threshold value, the macro block is determined to be a static macro block, and FALSE is assigned to M [i] [j] (S207 ). Otherwise, it is determined that the macroblock is a moving macroblock, and TRUE is assigned to M [i] [j] (S206).
[0069]
If MODE is NOT_CODED in the determination in step S203, the macroblock is determined to be a static macroblock, and FALSE is substituted for M [i] [j] (S207).
[0070]
Next, the moving object determination unit H113 determines the moving object from the static motion determination information for each macroblock of the macroblock static motion determination unit H111. FIG. 14 is a flowchart showing the processing contents of the moving object determination means H113. In the moving object determination means H113, two processes of a noise removal process (S301) and a moving object inclusion process (S302) are performed.
[0071]
In the noise removal process (S301), in order to prevent erroneous detection as a moving macroblock due to fluctuation of a small object in the background image or noise during image capture, a moving macro in which all the surrounding 8 macroblocks are stationary. The block is removed. The moving object inclusion process (S302) performs a process of detecting a minimum rectangle including a region where a moving macroblock is adjacent to the result of the static motion determination after removing the noise. With this process, a rectangle that encompasses the moving object can be found.
[0072]
FIG. 15 is a flowchart showing specific processing contents of the noise removal processing (S301). As in FIG. 13, i and j represent the addresses of macroblocks in the vertical and horizontal directions in the frame, respectively, and V_NMB and H_NMB represent the numbers of macroblocks in the vertical and horizontal directions in the frame. The two-dimensional array M [i] [j] is an array for storing information indicating whether each macro block is a dynamic macro block. If TRUE, a dynamic macro block is indicated, and if it is FALSE, a static macro block is indicated.
[0073]
Steps S401 to S408 (LOOP1) mean that i = 0 to V_NMB-1 is repeated, and Steps S402 to S407 (LOOP2) mean that j = 0 to H_NMB-1 is repeated.
[0074]
First, the static motion determination result for each macroblock is observed (S403). If the value of the two-dimensional array M [i] [j] is FALSE, that is, a static macro block, nothing is performed in the macro block, and the next macro block is reached by LOOP2 and LOOP1. If the value of the two-dimensional array M [i] [j] is TURE, that is, a dynamic macroblock, the static determination results of 8 macroblocks around the macroblock are checked (S405), and all are FALSE, that is, static. If it is a macro block, the macro block is determined to be noise and rewritten to a static macro block (S406). If even one of the surrounding 8 macroblocks has TURE, it is not determined as noise, and the next macroblock is moved by LOOP2 and LOOP1. The outside of the screen is assumed to be a static macro block.
[0075]
FIG. 16 is a flowchart showing specific processing contents of the moving object inclusion processing (S302). In step S501, n is a counter indicating the number of moving objects, S1 to S4 are parameters indicating a search range for a rectangle including the moving object, and S1 and S2 are the start point and end point of the address in the vertical direction. , S3, S4 are the start point and end point of the address in the horizontal direction.
[0076]
First, initialization is performed so that the entire frame is designated as a search range (S501). Next, a function “Rectangular” that searches for the smallest rectangle that encompasses the moving object in the designated search range is called (S502).
[0077]
17 and 18 are flowcharts showing the processing contents of the function Rectangular. The function Rectangular takes as input the search range S1 to S4, n indicating the number of moving objects, and the two-dimensional array M in which the static motion determination results of each macroblock are stored. The stored one-dimensional arrays B1 to B4 and n indicating the number of moving objects are output.
[0078]
Here, the one-dimensional array HV is a working array for creating a histogram of the number of moving macroblocks in the vertical direction, and the one-dimensional array HH is for creating a histogram of the number of moving macroblocks in the horizontal direction. It is a working arrangement. The variable VFLAG is a flag that changes to TRUE when the value of the histogram in the horizontal direction is not 0, and FALSE when the value is 0, and the variable HFLAG has a value of 0 as the histogram in the vertical direction. It is a flag that is changed to become TRUE when it is not, and FALSE when the value is 0.
[0079]
First, in FIG. 17, the range of S1 to S2, which is the search range of the work array HV for creating a histogram of the number of moving macroblocks in the vertical direction, is initialized with a value 0 (S601). In the next loop of LOOP1 (S602 to S607) and LOOP2 (S603 to S606), a histogram of the number of moving macroblocks in the vertical direction in the search range is created. That is, the value of the static motion determination result M [i] [j] for each macroblock is compared (S604). If the value is TURE, that is, if it is a dynamic macroblock, HV [i] is incremented by 1 (S605), If it is FALSE, the action is to do nothing.
[0080]
Next, the process proceeds to LOOP3 (S602 to S639), and a non-zero continuous portion is searched from the vertical histogram. First, the flag VFLAG is set to FALSE (S608). Next, in the order of the search range S1 to S2, it is checked whether the histogram HV is not 0 and VFLAG is FALSE (S610). This condition applies to the start point of a continuous portion where the histogram is not zero. Accordingly, since it becomes a candidate for the vertical start point of the searched rectangle, the address i is stored in the one-dimensional array B1 [n], and VFLAG is set to TURE (S611).
[0081]
Next, it is checked whether the histogram HV is 0 or the end of the search range and VFLAG is TURE (S612). This condition applies to the end portion of the continuous portion where the histogram is not zero. Therefore, since it is a candidate for the vertical end point of the rectangle being searched, if the histogram HV is 0, the address i-1 is stored in the one-dimensional array B2 [n] (S614), otherwise Stores the address i in the one-dimensional array B2 [n] (S615). Then, VFLAG is set to FALSE again (S616).
[0082]
Next, in FIG. 18, this time, the range of S3 to S4, which is the search range of the work array HV for creating a histogram of the number of moving macroblocks in the horizontal direction, is initialized with a value of 0 (S617). In the next loop of LOOP4 (S618 to S623) and LOOP5 (S619 to S622), a histogram of the number of horizontal moving macroblocks in the search range is created. That is, the value of the static motion determination result M [k] [j] for each macroblock is compared (S620), and if the value is TURE, that is, a dynamic macroblock, HH [j] is incremented by 1 (S621), If it is FALSE, the action is to do nothing.
[0083]
Next, a non-zero continuous portion is searched from the horizontal histogram. First, the flag HFLAG is set to FALSE (S624). Next, the processing proceeds to LOOP6 (S625 to S638), and it is checked whether the histogram HH is not 0 and HFLAG is FALSE in the order of the search ranges S3 to S4 (S626). This condition applies to the start point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the horizontal start point of the rectangle being searched, the address j is stored in the one-dimensional array B3 [n], and HFLAG is set to TURE (S627).
[0084]
Next, it is checked whether the histogram HH is 0 or the end of the search range and HFLAG is TURE (S628). This condition applies to the end portion of the continuous portion where the histogram is not zero. Accordingly, since it is a candidate for the horizontal end point of the rectangle being searched, it is determined whether or not the histogram HH is 0 (S629). If the histogram HH is 0, the one-dimensional array B4 [n] is stored. The address j-1 is stored (S630), and if not, the address j is stored in the one-dimensional array B4 [n] (S631). Then, HFLAG is set to FALSE again (S632).
[0085]
Here, the search using one vertical and horizontal histogram has been completed, and it is checked whether the search results B1 [n] to B4 [n] match the search range S1 to S4 (S633). If there is a match, there is no further search range, so it can be determined that the smallest rectangle has been obtained (S634). Then, n representing the number of moving objects is incremented by 1 (S635), and the next moving object is searched. If the search results B1 [n] to B4 [n] do not match the search ranges S1 to S4, there are still a plurality of moving objects in the search result range, so the search results B1 [n] to B4 [n ] Are transferred to S1 to S4 (S636), and the function Rectangular is called again (S637).
[0086]
The frame recording timing determination unit H102 determines the timing for recording a frame based on n and B1 [n] to B4 [n] as a result of being sent from the moving object detection unit H101 as follows.
[0087]
If n is 0, no frame is recorded. If the number of n is 1 or more, that is, if one or more moving objects are detected, if the time code of the frame is one second or more compared to the time code of the previously recorded frame, the frame is Record. Otherwise, do not record.
[0088]
The frame recording timing determination means H102 uses the rectangular coordinate data B1 [n] to B4 [n] including the moving object to detect the moving object at a more complicated determination condition, for example, a predesignated position. If it is determined that the moving object is moving at a certain speed or more, or if it is determined that the moving object is moving at a certain speed, the frame recording timing may be determined. it can.
[0089]
Based on the frame recording timing determined by the frame recording timing determination unit H102, the output control unit 209 records the frame in the frame recording unit 103.
[0090]
As described above, since the frame in which the moving object is detected is recorded, it is possible to quickly fast-forward and rewind to the frame in which the moving object is detected.
[0091]
Here, a temporary recording ring buffer for temporarily storing the frame decoded by the second decoding unit 700 may be provided in the frame recording unit 103. The structure of the temporary recording ring buffer is the same as that shown in FIG. 6, and the second decoding means 700 stores all the newly decoded frames while erasing the old frames. The frame is saved for 3 seconds, for example, and 30 records are recorded at 10 frames per second.
[0092]
When the frame recording timing determination unit H102 detects the moving object and makes the determination, the frame recording unit 103 detects the frame sent from the second decoding unit 700 and the oldest recorded in the temporary recording ring buffer. The frame 3 seconds before is recorded in the second stage ring buffer in order of time.
[0093]
In other words, a frame 3 seconds before detecting a moving object is recorded first, and a frame at the moment when the moving object is detected is recorded later. In the list display of the frame display means 800, a frame when the frame recording timing determination means H102 detects a moving object and makes a determination is adopted, and when playback from that frame is specified, the playback control means 104 is specified. Playback starts from the frame 3 seconds before the frame.
[0094]
As a result, not only can the fast-forward / rewind to the frame in which the moving object is detected be possible, but also the fast-forward / rewind to the frame 3 seconds before the moving object is detected.
[0095]
【The invention's effect】
As described above, according to the moving image playback / display apparatus of the present invention, it is possible to perform fast rewind playback or fast forward playback even for encoded moving image data in which a P-frame continues for a long time. Further, by combining the moving object detection means, it is possible to quickly reproduce the image of the scene where the moving object is detected.
[Brief description of the drawings]
FIG. 1 is a block configuration diagram of a moving image reproduction display device according to a first embodiment of the present invention.
FIG. 2 is a block configuration diagram of decoding means in the first embodiment of the present invention.
FIG. 3 is a block configuration diagram of display means in the first embodiment of the present invention.
FIG. 4 is an explanatory diagram of a record recorded in the frame recording unit in the first embodiment of the invention.
FIG. 5 is a flowchart showing the processing contents of a frame search function of the frame recording means in the first embodiment of the invention.
FIG. 6 is an explanatory diagram when a ring buffer is employed for the frame recording means in the first embodiment of the invention.
FIG. 7 is an explanatory diagram in the case where a double ring buffer is adopted as the frame recording means in the first embodiment of the invention.
FIG. 8 is a block configuration diagram of a moving image reproduction display device according to a second embodiment of the present invention.
FIG. 9 is a block configuration diagram of a moving image reproduction display device according to a third embodiment of the present invention.
FIG. 10 is an explanatory diagram of an example of frame display by frame display means in the third embodiment of the invention.
FIG. 11 is a block configuration diagram when moving object detection means and frame recording timing determination means are added to the second decoding means of FIG. 8 or FIG. 9;
FIG. 12 is a flowchart showing the processing contents of the moving object detection means in the embodiment of the present invention.
FIG. 13 is a flowchart showing the processing contents of macroblock static motion determination processing in the moving object detection means in the embodiment of the present invention.
FIG. 14 is a flowchart showing the processing contents of a moving object determination process in the moving object detection unit according to the embodiment of the present invention.
FIG. 15 is a flowchart showing the processing content of noise removal processing in moving object determination processing in the embodiment of the present invention;
FIG. 16 is a flowchart showing the processing contents of a moving object inclusion process in the moving object determination process according to the embodiment of the present invention.
FIG. 17 is a flowchart (part 1) showing the processing contents of a function Rectangular in the moving object inclusion processing in the embodiment of the present invention;
FIG. 18 is a flowchart (No. 2) showing the processing contents of the function Rectangular in the moving object inclusion processing in the embodiment of the present invention;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... File 101 ... Input control means 102 ... Decoding means 103 ... Frame recording means 104 ... Reproduction control means 105 ... Display means 700 ... Second decoding means 800 ... Frame display means

Claims (3)

In a moving image reproduction display device for reproducing and displaying encoded moving image data in which an I-frame including information on the entire frame is followed by a P-frame consisting of difference information between the frames,
Input control means for controlling the input of encoded moving image data;
Decoding means for decoding each frame from the encoded moving image data sent from the input control means;
A frame recording means for adding a frame number and a time code to the frame decoded by the decoding means and recording a position in the moving image data as information of the entire frame;
Based on the frame number and time code of the frame to be reproduced, the reproduction control for taking out the information of the entire corresponding frame from the frame recording means and setting it in the decoding means and setting the input position of the moving image data in the input control means Means,
Display means for displaying the decoded frame of the decoding means ,
The frame recording means has a two-stage configuration having two ring buffers for recording a new frame after the number of frames to be recorded reaches a predetermined number. A moving image reproducing / displaying device, wherein a frame recording period of the ring buffer in the stage is longer than a frame recording period of the ring buffer in the first stage. Second decoding for decoding moving image data encoded prior to the decoding means and transmitting the decoded frame, frame number, time code, and position in the moving image data to the frame recording means Providing means,
The frame recording means records the frame received from the second decoding means in the second stage ring buffer, and records the frame from the decoding means in the first stage ring buffer. The moving image reproduction display device according to claim 1 , characterized in that: Frame display means for displaying a list of frames recorded in the frame recording means;
Moving object detection means for determining whether the image in the block is stationary or moving for each block of the frame decoded by the second decoding means, and detecting the presence of the moving object from the static determination result;
A frame recording timing determination unit that records information of the entire frame in the frame recording unit at a predetermined timing when the moving object is detected by the moving object detection unit;
The frame recording means has a temporary recording ring buffer that periodically records the frames decoded by the second decoding means separately from the second stage ring buffer, and the frame recording timing determination A frame at a timing determined by the means and a frame that is a predetermined time before the frame recorded in the temporary recording ring buffer are recorded in the second stage ring buffer, and the frame recording means stores the frame recording. Display the frame of the timing when the timing judgment means made the judgment,
3. The moving image reproduction display apparatus according to claim 2 , wherein the reproduction control unit sets reproduction from a frame a predetermined time before the timing determined by the frame recording timing determination unit.

Images