JP5423166B2 - Video playback device, video playback program, and video playback method - Google Patents

Description

  The technology disclosed in this application relates to a video playback device, a video playback program, and a video playback method for playing back compressed video data.

  In the video surveillance system, a surveillance camera is installed in the vicinity of the surveillance object such as an office entrance, elevator hall, server room, office room, bank or convenience store ATM. The video surveillance system is mainly intended for crime prevention. In such a system, video data obtained by the surveillance camera is stored in a storage or the like, and can be reproduced as video at a later date for confirmation of an incident.

  Depending on the content of the incident, the time and date of the incident may not be limited. This is the case, for example, when there is an accident of leakage of confidential customer information and the trace of the information source being taken out is investigated. Such a survey operator narrows down the time range by, for example, surveying an entrance / exit log or video analysis. This time is, for example, the time when a person has come to a specific position in the video. When the time range cannot be narrowed down, the worker needs to visually check images of several days to several months. Appropriate time is required for the operator to check the video for a long time, and the physical and mental burden on the operator is large.

  As a related technique, there is a technique for transferring only frames extracted from a plurality of frames.

JP 2003-309811 A

  However, in order for a computer that reproduces video data to recognize a motion in the video data and perform reproduction according to the recognition result, a large amount of computation such as difference computation between frames is required. In this case, the calculation occupies a central processing unit (CPU) of the computer, or the video data cannot be reproduced in time due to the calculation, so that it is necessary to thin out the video data.

  The present invention has been made to solve the above-described problems, and is a video playback device, video playback program, and video playback that control the playback speed of compressed video data in accordance with the magnitude of movement in the compressed video data. It aims to provide a method.

  In order to solve the above-described problem, an aspect of the present invention is to analyze compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression, thereby obtaining a frame of each image in the compressed video data. An analysis unit for acquiring the data size, a determination unit for determining a playback speed of the compressed video data based on the data size, a decoding unit for decoding the compressed video data into the plurality of images, and the plurality of the plurality of images A display control unit that sequentially displays images on a display device in accordance with the reproduction speed.

  According to the technique disclosed in this application, it is possible to control the playback speed of compressed video data in accordance with the magnitude of motion in the compressed video data.

It is a block diagram which shows the structure of a video surveillance system. It is a block diagram which shows the structure of a video reproduction apparatus. It is a flowchart which shows operation | movement of a video reproduction apparatus. It is a flowchart which shows a reproduction speed determination process. It is a figure which shows an example of the relationship between m and r. It is a figure which shows the data size of the compression video data of each frame. It is a figure which shows the data size of the compression video data of each frame when a camera pans. It is a block diagram which shows the structure of the video reproduction apparatus of a modification. It is a figure which shows an example of the computer system to which this invention is applied.

  Embodiments of the present invention will be described below with reference to the drawings.

  Hereinafter, a video surveillance system 1 as an application example of the present invention will be described.

  FIG. 1 is a block diagram showing the configuration of the video monitoring system 1. The video monitoring system 1 includes a plurality of cameras 2, a plurality of recorders 3, a server 4, a storage device 5, a video playback device 6, a display device 7, and an input device 8.

  The camera 2 is installed toward the monitoring target, photographs the monitoring target, and transmits the obtained video (moving image) data to the recorder 3. The recorder 3 encodes the video data received from the camera 2 to compress it into compressed video data, temporarily stores the compressed video data, and transmits the compressed video data to the server 4 at predetermined time intervals. The server 4 stores the compressed video data received from the recorder 3 in the storage device 5. The storage device 5 has a storage such as a hard disk or a magnetic tape, and stores compressed video data. The video playback device 6 reads the compressed video data from the storage device 5 according to an instruction input from the operator to the input device 8, decodes it, and causes the display device 7 to display the decoding result.

  In order to collect long-term video data, the camera 2 performs shooting at a shooting frame rate of 1 fps (frame / second), for example. The recorder 3 encodes video data by a predetermined video encoding method to generate compressed video data. The predetermined video encoding method performs inter-frame compression. Video encoding systems that perform inter-frame compression are, for example, MPEG (Moving Picture Experts Group) 1, MPEG2, MPEG4, H.264, and the like. H.264 etc. The video encoding method in the video monitoring system 1 includes one I picture and a plurality of P pictures for each GOP (Group Of Pictures). An I picture is a frame compressed in a frame. A P picture is an inter-frame compressed frame that is encoded using information of the previous frame.

  Hereinafter, the video playback device 6 will be described.

  FIG. 2 is a block diagram showing the configuration of the video playback device 6. The video playback device 6 includes a communication unit 11, a memory 12, an analysis unit 14, a playback speed determination unit 18, a decoding unit 15, a timing control unit 16, and a display device control unit 17. The analysis unit 14, the playback speed determination unit 18, the decoding unit 15, the timing control unit 16, and the display device control unit 17 are stored in the storage unit in the video playback device 6 as a video playback program and are stored in the CPU in the video playback device 6. It may be executed by (Central Processing Unit) or may be realized as a circuit.

  In accordance with an instruction from the input device 8, the communication unit 11 reads the compressed video data from the storage device 5 and expands it in the memory 12. The analysis unit 14 reads the compressed video data from the memory 12, analyzes the header in the compressed video data, sends the analysis result to the timing control unit 16, and sends the compressed video data of each frame to the decoding unit 15. The decoding unit 15 decodes the compressed video data to generate an image of each frame. The timing control unit 16 includes a timer serving as a reference for video reproduction, and generates a signal indicating timing based on the analysis result of the analysis unit 14. The display device control unit 17 causes the display device 6 to display the images of each frame decoded by the decoding unit 15 in order according to the timing from the timing control unit 16.

  FIG. 3 is a flowchart showing the operation of the video reproduction device 6. The analysis unit 14 executes this flow every time a GOP is acquired. Here, it is assumed that the frame number of the I picture that is the first frame of the GOP is t.

  First, the timing control unit 16 sets a timer (S31). Next, the decoding unit 15 decodes the compressed video data of the I picture (t) (S32). Next, the analysis unit 14 acquires the data size a of the compressed video data of the I picture (t) from the header of the I picture (t) and stores it in the memory 12 (S33). Next, the timing control unit 16 waits until the timer value reaches the reference playback frame time interval (S34). Next, the display device control unit 17 causes the display device 7 to display the decoded I picture (t) image (S35).

  Next, the analysis unit 14 increases t by 1 (S39), and determines whether or not the next frame (t) is an I picture, that is, whether or not the GOP has ended (S40). When the next frame is an I picture, that is, when the GOP is completed (S40, Y), this flow ends.

  If the next frame is not an I picture, that is, if the GOP has not ended (S40, N), the timing control unit 16 sets a timer (S41). Next, the decoding unit 15 decodes the P picture (t) (S42). Next, the analysis unit 14 acquires the data size b of the compressed video data of the P picture (t) from the header of the P picture (t) and stores it in the memory 12 (S43). Next, the playback speed determination unit 18 determines the playback speed r by playback speed determination processing based on a and b stored in the memory 12 (S44). Next, the timing control unit 16 waits until the timer reaches the playback frame time interval (S45). Here, the playback frame time interval is the reference playback frame time interval / r. Next, the display device control unit 17 causes the display device 7 to display the decoded image of the I picture (t) (S46). Next, this flow moves to S39.

  Here, the reference playback frame rate is higher than the shooting frame rate, for example, 30 fps. Thereby, the operator can view long-time video data in a short time. In this case, the above-mentioned reference reproduction frame time interval is 1/30 second.

  The order of S32 and S33 may be reversed. Similarly, the order of S42 and S43 may be reversed.

  FIG. 4 is a flowchart showing the playback speed determination process. The analysis unit 14 executes this flow for each P picture. First, the reproduction speed determination unit 18 calculates m = b / a (S20).

  Next, the playback speed determination unit 18 determines whether or not 1/2 ≦ m (S21). When 1/2 ≦ m is satisfied (S21, Y), the playback speed determination unit 18 sets the playback speed r to 1 (1 × speed) (S22), and ends this flow.

  If 1/2 ≦ m is not satisfied (S21, N), the playback speed determination unit 18 determines whether 1/5 ≦ m <1/2 is satisfied (S23). If 1/5 ≦ m <1/2 (S23, Y), the playback speed determination unit 18 sets r to 8 (8 times speed) (S24), and ends this flow.

  When 1/5 ≦ m <1/2 is not satisfied (S23, N), that is, when m <1/5 (S25, Y), the playback speed determination unit 18 sets r to 32 (32 times speed) ( S26), this flow is finished.

  FIG. 5 is a diagram illustrating an example of the relationship between m and r. The playback speed determination unit 18 determines the value of r based on the range of m values. As described above, when 1/2 ≦ m, r is set to 1, when 1/5 ≦ m <1/2, r is set to 8, and when m <1/5. r is set to 32. That is, the video playback device 6 increases the playback speed of the P picture as the data size of the P picture is larger.

  In the reproduction speed determination process, the reproduction speed determination unit 18 may calculate r from m using a preset relationship between m and r.

  Hereinafter, effects of the video monitoring system 1 will be described.

  FIG. 6 is a diagram showing the data size of the compressed video data of each frame. In this figure, the horizontal axis indicates time t, and the vertical axis indicates the data size of the compressed video data of each frame. Further, below the horizontal axis, from the top, the frame number, the picture type (I or P) of the frame, and the image of the frame are shown. Here, the camera 2 images the door to be monitored. There is almost no temporal change of the frame image at t = 0-5. Since the person opened the door at t = 6 to 8, the time change of the frame image is larger than t = 0 to 5. The data size of the compressed video data of the I picture is larger than the data size of the compressed video data of the P picture regardless of the temporal change of the frame image. The data size of the compressed video data of the P picture is larger when the temporal change of the frame image is larger than when the temporal change of the frame image is small.

  Therefore, the video playback device 6 can increase the playback speed of a scene where the temporal change of the frame image is small compared to a scene where the temporal change of the frame image is large. Thereby, the worker who views the image displayed on the display device 6 can view the scene where the time change of the frame image is larger than that of the scene where the time change of the frame image is small. Thereby, the worker can efficiently view the video imaged by the surveillance camera.

  Furthermore, the video playback device 6 does not need to perform a large amount of calculations such as calculating a difference between frames in order to control the playback speed. Thereby, the video reproduction device 6 can operate other processes simultaneously. Further, since the load of controlling the playback speed in the video playback device 6 is small, the playback speed does not reach a peak. This eliminates the need to thin out and reproduce the video, increasing the reliability as evidence. Further, since the load of controlling the playback speed in the video playback device 6 is small, the power consumption of the CPU in the video playback device 6 is small.

  Hereinafter, the threshold value change in the reproduction speed determination process will be described.

  The threshold value in the playback speed determination process described above may be changed according to the video encoding method and the monitoring target. Hereinafter, three patterns A, B, and C are given as examples of changing the threshold value in the reproduction speed determination process.

  Pattern A is a case where the threshold value of m in the reproduction speed determination process is changed to be small. Pattern B is a case where the threshold value of m in the reproduction speed determination process is greatly changed. Pattern C is a case where the value of m is always large and the effect of the video monitoring system 1 is small.

  Pattern A includes the case where the frame rate of the video data is higher than the set value. This is pattern A because the higher the frame rate, the smaller the movement of the monitoring target between frames and the smaller the data size b of the P picture. Pattern A includes a case where the distance from the camera 2 to the monitoring target is farther than the set value. This includes the case where a wide space is a monitoring target. This is pattern A because the movement of the monitoring target between frames decreases as the distance from the camera 2 to the monitoring target increases, and the data size b of the P picture decreases.

  Pattern B includes the case where the frame rate of the video data is lower than the set value. This is pattern B because the lower the frame rate, the larger the movement of the monitoring target between frames and the larger the data size b of the P picture. Pattern A includes the case where the distance from the camera 2 to the monitoring target is closer than the set value. This includes a case where a moving part such as the upper body and the face is reflected in a large part of the image, such as when the camera 2 is built in an ATM, or a case where sports are photographed. This is pattern B because the closer the distance from the camera 2 to the monitoring target, the larger the movement of the monitoring target between frames and the larger the data size b of the P picture.

  Pattern C includes a case where the background is a hustle and bustle. This is pattern C because there are many objects to be monitored and the directions of movement of the objects to be monitored are different. Pattern C includes a case where the background fluctuates irregularly. This includes cases where the background is a tree branch or grass swaying in the wind, or a wave of water. This is a pattern C because there are many background movements and the movement directions are different.

  The advantages of the present invention over the prior art when the camera 2 pans (rotates in the horizontal direction with respect to the image) will be described below.

  When the camera 2 is moved or rotated, a video encoding method that performs motion compensation processing is used. Video encoding systems that perform this motion compensation processing are, for example, MPEG1, MPEG2, MPEG4, H.264, and the like. H.264 etc.

  FIG. 7 is a diagram illustrating the data size of the compressed video data of each frame when the camera 2 performs panning. In this figure, the vertical axis indicates time t, and the horizontal axis indicates the data size of the compressed video data of each frame. Further, on the left of the vertical axis, from the right, the frame number, the picture type (I or P) of the frame, and the image of the frame are shown. Here, since the camera 2 rotates to the left, the background in the frame image moves to the right as time passes.

  Therefore, it cannot be determined that there is a movement of a person in the image simply because the difference between the pixels of the previous and subsequent frames is large.

  On the other hand, the present invention can control the reproduction speed based on the data size of the frame. That is, since only the background is captured in the frame image at t = 1 to 3 and 7 to 8, the data size of the P picture is small due to motion compensation. Since a moving person is photographed in the frame image at t = 4-6, the data size of the P picture is larger than t = 1-3.

  Therefore, even when the camera 2 is moved or rotated, the playback speed can be appropriately controlled based on the frame data size.

  Hereinafter, modified examples of the video monitoring system 1 will be described.

  The camera 2 may be single. The recorder 3 may be single. The recorder 3 may store the compressed video data directly in the storage device 5 without using the server 4. The recorder 3, the storage device 5, and the video playback device 6 may be a single device.

  The video playback device 6 may determine a playback speed for each group having a plurality of frames, and display a plurality of consecutive decoded images on the display device in order according to the playback speed for each group.

  When the time from the start time of the display of the first frame to the start time of the display of the second frame next to the first frame is defined as a playback frame time interval, the video playback device 6 is based on the data size of the first frame. Thus, the playback frame time interval may be determined, or the playback frame time interval may be determined based on the data size of the second frame. When the video playback device 6 determines the playback frame time interval based on the data size of the first frame, the video playback device 6 decodes the first frame while waiting for the playback frame time interval. When the video playback device 6 determines the playback frame time interval based on the data size of the second frame, the video playback device 6 decodes the second frame while waiting for the playback frame time interval.

  The playback speed determination unit 18 may determine the playback frame time interval based on the data size of some frames. For example, the playback speed determination unit 18 may determine the playback frame time interval based on the data size of the frame at a predetermined frame interval. The playback speed determination unit 18 may determine the frame rate based on the data size of each frame.

  Hereinafter, a video playback device 6a, which is one of modifications of the video playback device 6, will be described.

  FIG. 8 is a block diagram showing a configuration of a video reproduction device 6a according to a modification.

  The video playback device 6a analyzes the compressed video data obtained by compressing a plurality of images by video encoding that performs inter-frame compression, and thereby obtains the data size of the frame of each image in the compressed video data; The determination unit 28 that determines the playback speed of the compressed video data based on the data size, the decoding unit 25 that decodes the compressed video data into a plurality of images, and the playback speed of the plurality of images decoded by the decoding unit 25 And a display control unit 27 that causes the display device 7 to display in order.

The analysis unit 24 includes, for example, the analysis unit 14.
The determination unit 28 includes, for example, a playback speed determination unit 18.
The decoding unit 25 includes, for example, the decoding unit 15.
The display control unit 27 includes, for example, a timing control unit 16 and a display device control unit 17.

  The present invention can be applied to the following computer system. FIG. 9 is a diagram illustrating an example of a computer system to which the present invention is applied. A computer system 900 shown in this figure includes a main body 901 incorporating a CPU, a disk drive, and the like, a display 902 that displays an image according to an instruction from the main body 901, a keyboard 903 for inputting various information to the computer system 900, A mouse 904 for designating an arbitrary position on the display screen 902a of the display 902 and a communication device 905 for accessing an external database or the like and downloading a program or the like stored in another computer system are provided. The communication device 905 may be a network communication card, a modem, or the like.

  A program for executing the above-described steps in the computer system constituting the video playback apparatus as described above can be provided as a video playback program. By storing this program in a recording medium that can be read by the computer system, the program can be executed by the computer system constituting the video reproduction apparatus. A program for executing the above steps is stored in a portable recording medium such as a disk 910 or downloaded from a recording medium 906 of another computer system by the communication device 905. In addition, a video playback program that causes the computer system 900 to have at least a video playback function is input to the computer system 900 and compiled. This program causes the computer system 900 to operate as a video playback system having a video playback function. Further, this program may be stored in a computer-readable recording medium such as a disk 910, for example. Here, examples of the recording medium readable by the computer system 900 include an internal storage device such as a ROM and a RAM, a portable storage such as a disk 910, a flexible disk, a DVD disk, a magneto-optical disk, and an IC card. It includes a medium, a database holding a computer program, or other computer systems and the database, and various recording media accessible by a computer system connected via communication means such as a communication device 905.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
An analysis unit that obtains the data size of a frame of each image in the compressed video data by analyzing compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression;
A determining unit that determines a playback speed of the compressed video data based on the data size;
A decoding unit for decoding the compressed video data into the plurality of images;
A display control unit for displaying the plurality of images on a display device in order according to the reproduction speed;
A video playback device comprising:
(Appendix 2)
The display control unit starts displaying an image of a second frame next to the first frame at a time interval determined based on the playback speed from a time when display of the first frame is started. ,
The video reproduction apparatus according to appendix 1.
(Appendix 3)
The determining unit determines the time interval based on a data size of the first frame or a data size of the second frame;
The video reproduction device according to attachment 2.
(Appendix 4)
The video encoding performs intra-frame compression and inter-frame compression,
The compressed video data includes an I picture that is a frame compressed by the intra-frame compression and a P picture that is a frame compressed by the inter-frame compression,
The determining unit determines a reproduction timing of the P picture based on a ratio of a data size of the P picture to a data size of the I picture;
The video reproduction apparatus according to appendix 1.
(Appendix 5)
The determination unit increases the reproduction speed of the P picture as the ratio between the data size of the I picture and the data size of the P picture increases.
The video reproduction device according to attachment 4.
(Appendix 6)
The determination unit determines the playback speed for each frame,
The video playback device according to claim 1, wherein the display control unit causes the display device to sequentially display a plurality of consecutive images decoded by the decoding unit according to the playback speed for each frame.
(Appendix 7)
The video encoding further performs motion compensation in the inter-frame compression.
The video reproduction apparatus according to appendix 1.
(Appendix 9)
By analyzing compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression, the data size of each image frame in the compressed video data is obtained,
Determining a playback speed of the compressed video data based on the data size;
Decoding the compressed video data into the plurality of images;
Displaying the plurality of images on a display device in order according to the reproduction speed;
A computer-readable recording medium storing a video playback program that causes a computer to execute the above.
(Appendix 10)
By analyzing compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression, the data size of each image frame in the compressed video data is obtained,
Determining the playback speed of the compressed video data based on the data size;
Decoding the compressed video data into the plurality of images;
Displaying the plurality of images on a display device in order according to the reproduction speed;
A video playback method comprising the above.

DESCRIPTION OF SYMBOLS 1 Video surveillance system 2 Camera 3 Recorder 4 Server 5 Storage apparatus 6 Video | video reproduction apparatus 7 Display apparatus 8 Input apparatus 11 Communication part 12 Memory 14 Analysis part 15 Decoding part 16 Timing control part 17 Display apparatus control part 18 Playback speed determination part 24 Analysis Unit 25 decoding unit 27 display control unit 28 determining unit

Claims (4)

An analysis unit that obtains the data size of a frame of each image in the compressed video data by analyzing compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression;
A determining unit that determines a playback speed of the compressed video data based on the data size;
A decoding unit for decoding the compressed video data into the plurality of images;
A display control unit for displaying the plurality of images on a display device in order according to the reproduction speed;
The video encoding performs intra-frame compression and inter-frame compression,
The compressed video data includes an I picture that is a frame compressed by the intra-frame compression and a P picture that is a frame compressed by the inter-frame compression,
The video playback apparatus according to claim 1, wherein the determination unit determines a playback timing of the P picture based on a ratio of a data size of the P picture to a data size of the I picture. When the second frame next to the first frame is not the I picture , the display control unit has a time interval determined based on the playback speed from the time when the display of the first frame starts. to start the display of the previous SL the second frame of the image,
The video playback apparatus according to claim 1. By analyzing compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression, the data size of each image frame in the compressed video data is obtained,
Determining a playback speed of the compressed video data based on the data size;
Decoding the compressed video data into the plurality of images;
Causing the computer to display the plurality of images on a display device in order according to the playback speed;
The video encoding performs intra-frame compression and inter-frame compression,
The compressed video data includes an I picture that is a frame compressed by the intra-frame compression and a P picture that is a frame compressed by the inter-frame compression,
A video playback program characterized in that, in determining the playback speed of the compressed video data, the playback timing of the P picture is determined based on the ratio of the data size of the P picture to the data size of the I picture. By analyzing compressed video data in which a plurality of images are compressed by video encoding that performs inter-frame compression, the data size of each image frame in the compressed video data is obtained,
Determining the playback speed of the compressed video data based on the data size;
Decoding the compressed video data into the plurality of images;
Displaying the plurality of images on a display device in order according to the reproduction speed;
The video encoding performs intra-frame compression and inter-frame compression,
The compressed video data includes an I picture that is a frame compressed by the intra-frame compression and a P picture that is a frame compressed by the inter-frame compression,
A video playback method, wherein, in determining the playback speed of the compressed video data, the playback timing of the P picture is determined based on the ratio of the data size of the P picture to the data size of the I picture.

Images