JP4689341B2 - Image display device and image display system - Google Patents

Description

  The present invention relates to an image display device and an image display system.

  2. Description of the Related Art Conventionally, there has been a moving image still image display device that displays a moving image and a still image related to contents on a single screen while synchronizing (see, for example, Patent Document 1).

FIG. 7 is a diagram showing a schematic configuration of a moving image still image display device described in Patent Document 1. In FIG. This associates contents between a moving picture and a still picture taken by a digital camera or a still camera, and performs simultaneous display while synchronizing. The reproduction control unit 705 controls the reproduction of the moving image recorded on the moving image recording medium 701 and the still image recorded on the still image recording medium 703 to synchronize the moving image reproducing unit 702 and the still image reproducing unit 704 with each other. A moving image and a still image are reproduced, and each is displayed by the multi-screen simultaneous display means 706.
JP 2003-259272 A

  As described above, the conventional moving image still image display device simply displays the moving image and the still image side by side in synchronization. By the way, the surveillance camera may have a function of not only recording a moving image but also issuing an alarm when an abnormal situation occurs and recording a high-definition still image. When analyzing images such as surveillance cameras that can shoot such related videos and still images, it is not only displayed side by side in synchronization, but also abnormal situations depending on the occurrence of abnormal situations etc. Advanced and various display controls for the purpose of confirming the above are required.

  The present invention has been made in view of the above-described conventional circumstances, and provides an image display device and an image display system capable of unified display control while synchronizing a moving image and a still image with an easy operation. For the purpose.

The image display apparatus according to the present invention reads still image information from a still image storage unit in which still image information including still image video information and still image synchronization information associated therewith is recorded, and decodes the still image decoding Means, moving picture decoding means for reading out and decoding the moving picture information from moving picture storage means in which moving picture information including moving picture video information and moving picture synchronization information associated therewith is recorded, and a control instruction for the input display video Display image control means for generating display image control information from the display image, and a display format instruction for defining a spatial layout and a temporal display pattern of the image display based on the display image control information, Display image generation control means for instructing the still picture decoding means and the moving picture decoding means to decode necessary still picture information and moving picture information, and the still picture decoding means Displayed video information included in the generated display video information, and display video information for generating display video information by processing the encoded still image information and the video information decoded by the video decoding means according to the display format instruction A display means for displaying information and outputting still image synchronization information and moving picture synchronization information included in the display video information to the display video control means , wherein the display video control means Generating the display video control information according to the input control instruction based on the still picture synchronization information and the video synchronization information output from the display means, and the display video generation control means displays only the video In the case of performing an error, the abnormal area is determined based on the abnormality detection state information input from the outside, and the still image of the abnormal area is decoded and displayed. When the abnormal region moves with time, as the tracking abnormal region based on said abnormal state information, continue to display the enlarged image of the still image continuously.

With this configuration, it is possible to unify the still image and the moving image and to control the display pattern flexibly with an easy operation while synchronizing the position and time of the moving image and the still image. Moreover, display control according to the abnormality detection state can be performed.

The image display system of the present invention, a still image and recording motion pictures, photographed to detect the movement of any object to be shot in the video was, imaging of both output and the still picture information and the video information as abnormality detection state information Means and the image display device described above.

  With this system, using image capturing means capable of reporting abnormality detection information, the position and time of moving images and still images are synchronized, and the display pattern can be changed flexibly by unifying the still images and moving images with easy operation. It becomes possible to control.

  According to the present invention, it is possible to provide an image display apparatus and an image display system capable of performing unified display control while synchronizing a moving image and a still image with an easy operation.

  Next, embodiments of the image display device and the image transmission display system of the present invention will be specifically shown.

(First embodiment)
FIG. 1 is a block diagram of an image display apparatus according to the first embodiment of the present invention. The image display apparatus according to the first embodiment of the present invention includes a display video control unit 101, a still image storage unit 102, a moving image storage unit 103, a display video generation control unit 104, a still image decoder 105, a moving image decoder 106, and a display video generation. A display unit 107, a display device 108, and a display video control information storage unit 109.

  The display video control means 101 is linked with a display on a personal computer, for example, and the user inputs an instruction to control the display video as a user instruction using an input device such as a mouse or a keyboard while looking at the display. The display image is controlled according to the control instruction.

  The still image storage unit 102 records, for example, a still image video in JPEG (Joint Photographic Experts Group) format. Still image synchronization information is set as a parameter corresponding to each still image information. This set is called still image information and is recorded in the still image storage unit 102.

  FIG. 2A illustrates still image information recorded in the still image storage unit. As shown in FIG. 2A, the still image information format 301 of the still image information 112 recorded in the still image storage unit 102 is an example of still image video information 113 and still image identification information for identifying a still image. A still image identification ID, a still image time stamp that is an example of shooting time information that indicates the time at which the still image was shot, and a still image shooting camera ID that is an example of the still image shooting camera identification information that identifies the camera that shot the still image And still picture synchronization information 114 having subordinate items such as Note that the items of the still image information format 301 may exist in addition to the illustrated contents, and the still image synchronization information 114 includes the still image identification ID, the still image time stamp, and the still image shooting camera ID. At least one should be included.

  The moving image storage unit 103 records at least one moving image in, for example, MPEG (Moving Picture Experts Group) format. Each moving picture video information is set in correspondence with each moving picture synchronization information as a parameter. This set is called moving image information and is recorded in the moving image storage unit 103.

  FIG. 2B is a diagram for explaining moving image information recorded in the moving image storage unit. As shown in FIG. 2B, the moving image information format 302 of the moving image information 115 recorded in the moving image storage unit 103 includes moving image information 116, a moving image identification ID that is an example of moving image identification information for identifying a moving image, and a moving image. Video synchronization information 117 having subordinate items such as a video time stamp that is an example of shooting time information indicating the time when the video was shot, and a video shooting camera ID that is an example of video shooting camera identification information that identifies the camera that shot the video. Including. Note that the item of the moving image information format 302 may exist in addition to the illustrated content, and the moving image synchronization information 117 includes at least one of the moving image identification ID, the moving image time stamp, and the moving image shooting camera ID. That's fine.

  The display video control information storage unit 109 generates and records display video control information describing how to reproduce a still image and a moving image based on a display video control instruction input from the display video control means 101. The display video control information storage unit 109 may record one or more patterns for displaying moving images and still images. In this case, the display video generation control unit 104 selects an arbitrary display pattern to display the display. Read from the video control instruction storage means.

  FIG. 2C is a diagram for explaining display video control information recorded in the display video control information storage unit. As shown in FIG. 2C, the display video control information format 303 of the display video control information 111 is a still picture playback information 331 including a still picture playback sequence and a playback usage area, and a moving picture including a video playback sequence and a playback usage area. Reproduction information 334, display mode, still image display position, display layout information 337 including a moving image display position, abnormality detection alarm state and abnormality detection state information 210 including an abnormality detection position, and the like. Regarding items, items of the display video control information format 303 may exist in addition to the illustrated contents.

  The still image reproduction sequence of the still image reproduction information 331 determines the order and time for reproducing each still image, and the reproduction use area defines an area when only a part of the still image is used for display. The moving image reproduction sequence of the moving image reproduction information 334 determines the order and time for reproducing each moving image, and the reproduction use area defines an area when only a part of the moving image is used for display. The display mode of the display layout information 337 represents a state indicating a reproduction format, the still image display position determines at which position and in which size the still image is arranged, and the moving image display position is arranged at which position and in which size of the moving image. Determine what. These are described in such a format that a plurality of still images and moving images can be arranged.

  The abnormality detection alarm state of the abnormality detection state information 210 describes whether or not there is an abnormality in any recorded video, and the abnormality detection position indicates a position where an abnormality is detected in a frame having an abnormality. The display video control information 111 including these is created based on a display video control instruction or abnormality detection state information input from the display video control unit 101. The abnormality detection state information will be described in a second embodiment described later.

  The display video generation control unit 104 reads the display video control information recorded in the display video control information storage unit 109, and determines the spatial layout and temporal display pattern of video display based on the display video control information. A display format instruction is generated and output to the display video generation unit 107. Further, it instructs the still picture decoder 105 and the moving picture decoder 106 to decode still picture information and moving picture information necessary for the display format. The display image generation control unit 104 combines the still image synchronization information and the movie synchronization information included in the display image control information when creating the display format instruction, thereby combining the movie image information and the still image image information. Make temporal and spatial associations.

  The still image decoder 105 reads out still image information from the still image storage unit 102 and performs decoding. The still image decoder 105 reads out still image information according to a still image sequence instructed from the display image generation control unit 104 and generates a display image. Output to the unit 107. The moving picture decoder 106 reads out the moving picture information from the moving picture storage unit 103 and performs decoding. The moving picture decoder 106 reads out and decodes the moving picture information according to the moving picture sequence instructed from the display video generation control unit 104, and sends it to the display video generation unit 107. Output.

  Based on the display format instruction from the display video generation control unit 104, the display video generation unit 107 is designated with the still image information and / or moving image information from the still image decoder 105 and / or the video decoder 106 as the display format instruction. Processing is performed according to the display mode, display layout, and display timing, and the result is output to the display device 108 as display video information. The display video information includes, in addition to the video information, synchronization information corresponding to the video information (moving image synchronization information for moving images, and still image synchronization information for generation).

  The display device 108 displays the video information included in the display video information from the display video generation unit 107 on a display such as a monitor. Further, the still image synchronization information and the movie synchronization information included in the displayed movie and still image are output to the display image control means 101.

  As a result, the display video control unit 101 can grasp the frame and still image of the moving image displayed on the display device 108, so that the display video corresponding to the control instruction input by the user in accordance with the progress of the display screen. Generate control instructions. Therefore, the display video control information storage unit 109 can generate display video control information corresponding to such an input control instruction.

  FIG. 3A is a diagram showing an example of a display screen for displaying a moving image according to the embodiment of the present invention. The display video control means 101 controls the display video by giving a user instruction using an input device such as a mouse or a keyboard while watching the display in conjunction with a display on a personal computer, for example. For example, a display menu is prepared in the form of a button such as the display screen 401 during playback of a moving image, and the user presses at least one button as indicated by a thick arrow to display the display layout information of the display video control information format 303. A display mode which is a subordinate item 337 can be designated. The currently displayed moving image 411 displayed on the display screen 401 is displayed in synchronization based on the still image synchronization information 114 and the moving image synchronization information 117 sent from the display device 108.

  Next, the viewer reproduces the video, recognizes the suspicious person 403 in the video 411 being displayed, and enlarges and displays only the region 413 that the person wants to enlarge as indicated by the thick-lined arrow as a high-definition still image. An example of the method will be described. First, the enlargement or moving image button 412 is pressed to reproduce only the moving image 411. In this case, when the enlargement or moving image button 412 is pressed, the display image control unit 101 records the display mode, which is a subordinate item of the display layout information 337, as the moving image mode in the display image control information storage unit 109 on the assumption that the user instruction is received. To do. The display video generation control unit 104 determines the video mode and gives a video decoding instruction to the video decoder 106. The video decoder 106 decodes the corresponding video and sends the video synchronization information to the display video generation unit 107 together with the decoded video. The display video generation unit 107 arranges a moving image at a position indicated by a display format instruction from the display video generation control unit 104, and transmits it to the display device 108 as display video information together with the moving image synchronization information. The display device 108 displays the received display video information on a display or the like. By sending the moving image synchronization information of the displayed moving image 411 to the display image control means 101, the user can issue a user instruction in real time based on the display timing of the moving image 411.

  Next, an operation in the case where an enlarged region 413 is displayed from the moving image 411 being displayed by the viewer and an enlarged still image 414 is displayed will be described. FIG. 3B is a diagram showing an example of enlarging and displaying the still image portion of the display screen according to the embodiment of the present invention.

  By pressing an enlargement or moving image button 412, the user changes the display mode, which is a subordinate item of the display layout information 337, to the enlargement mode, and specifies the coordinates of the area to be enlarged. This is realized by dragging a range to be specified with an input device such as a mouse. The display video control means 101 converts the video synchronization information of the video 411 being displayed on the display device 108, the user instruction by the area 413 to be enlarged and the enlargement or video button 412 into a display video control instruction signal, and displays the video control information. Recording as display video control information 111 of the storage unit 109.

  The display video generation control unit 104 gives a still picture decoding instruction to the still picture decoder 105 so as to decode a still picture used for enlargement reproduction based on the recorded display video control information 111, and further at a predetermined position. Then, a display format instruction is given to the display video generation unit 107 so as to display an area to be used for display determined in the still image to be reproduced. The display video generation unit 107 combines the still image and the moving image decoded by each decoder into a layout according to the display format instruction from the display video generation control unit 104, and displays the display video, the still image synchronization information, and the moving image synchronization information on the display device 108. Transmit to. As a result, a desired image is displayed on the display screen 402 during display of the enlarged still image.

  Note that the viewer may recognize the suspicious part in the moving image 411 being displayed, and perform an operation of performing replay by enlarging only the region of the suspicious part with a high-definition still image. In this case, the replay button 415 is prepared instead of the enlargement or moving image button 412 in the same manner as the above-described enlarged display procedure, and the still image time of the still image information format 301 recorded in the still image storage unit 102 is prepared. The display video control means 101 gives an instruction to perform continuous display for a designated portion from a still image that goes back a predetermined time by using a moving image time stamp of the moving image information format 302 recorded in the stamp and the moving image storage unit 103. You may output.

  FIG. 4 is a diagram illustrating thumbnail display according to the embodiment of the present invention. In this example, a case will be described in which a still image thumbnail 513 temporally related to the moving image 511 being displayed is displayed. When the user wants to display the thumbnail 513 while displaying the moving image, the user presses the thumbnail button 512 on the display. Based on the input still image synchronization information and moving image synchronization information, the display video control means 101 has the moving image information format 302 recorded in the moving image storage unit 103 of the frame of the moving image 511 being displayed when the thumbnail button 512 is pressed. An arbitrary number of still images near the time are determined based on the moving image time stamp, and the mode change is also sent to the display video control information storage unit 109 as a display video control instruction.

  Then, the display video generation control unit 104 uses the still image time stamp of the still image information format 301 recorded in the still image storage unit 102 so that the thumbnail 513 is displayed on the display screen 501 during the thumbnail display. An instruction is given to the still picture decoder 105 and the display video generation unit 107. In this way, the still image thumbnail 513 can be displayed at a desired timing while viewing the moving image 511 being displayed.

  Alternatively, the thumbnail 513 may be used as an option to display the entire still image view 515. Note that, unlike the creation of the thumbnail 513 from the moving image 511 being displayed, as shown by the thick-lined arrow, by specifying one still image 514 of the displayed thumbnail, it is recorded in the still image storage unit 102. The time-related moving image may be called and reproduced using the still image time stamp in the still image information format 301 and the moving image time stamp in the moving image information format 302 recorded in the moving image storage unit 103.

  According to the first embodiment of the present invention as described above, the moving image captured by the imaging unit such as the surveillance camera and the still image are temporally and spatially linked to each other, and the still corresponding to the time is synchronized with the moving image. Any area of the image can be recalled. As a result, for example, when the display resolution is low and the monitor size is exceeded when a still image is called, the user does not need to perform any operations such as cropping or reduction processing, and replay only the necessary area can be edited. This can be done easily without any work. Furthermore, thumbnails can be called in accordance with the timing of the moving image, and the efficiency of viewing the monitoring video can be improved.

(Second Embodiment)
FIG. 5 is a block diagram of an image transmission display system according to the second embodiment of the present invention. In the second embodiment, a receiving unit 201 is provided, and information sent from the image input unit 202 via the transmission path 203 is accumulated. In FIG. 5, the same components as those in FIGS.

  As shown in FIG. 5, the video transmission display system of the second embodiment is an image that captures still images and moving images via the transmission path 203 in addition to the video transmission display system of the first embodiment shown in FIG. 1. A receiving unit 201 that receives information from an image input unit 202 such as a photographing unit is provided.

  The still image storage unit 102, the moving image storage unit 103, and the display video control information storage unit 109 are connected to a reception unit 201 that receives information from the transmission path 203. The image input unit 202 outputs the still image information from the receiving unit 201 to the still image storage unit 102, the moving image information to the moving image storage unit 103, and the abnormality detection state information to the display video control information storage unit 109 via the transmission path 203. To do. The abnormality detection state information transmitted from the transmission path 203 side includes information such as a moving image frame determined to be abnormal by the image input unit 202 and an abnormal part region in the frame. A plurality of image input means 202 may be connected. Other configurations, operation details, handling information, processing details, and the like are the same as those in the first embodiment.

  Next, a description will be given of a method in which when the image input unit 202 detects an abnormal area in the video, the enlarged display is traced around the abnormal area. FIG. 6 is a diagram illustrating a method for tracking and enlarging an abnormal part in the embodiment of the present invention. The image input unit 202 records moving image information simultaneously with still image information.

  As an image input unit 202 of the present embodiment, when an image is analyzed by an apparatus (not shown) that detects the movement of an arbitrary shooting target in the shot image and an abnormality is detected, A case will be described in which a network camera that can output abnormality detection information such as an abnormality detection position state that indicates an abnormality portion using a detection alarm and an image to the transmission path 203 is applied.

  The receiving unit 201 receives the information of the still image information 112, the moving image information 115, and the abnormality detection information 210 via the transmission path 203, and stores the still image storage unit 102, the moving image storage unit 103, and the display video control information storage, respectively. It sorts and accumulates in part 109.

  As a premise for explaining the outline of the operation of the tracking enlargement display of the second embodiment, it is assumed that only the moving image is displayed as in the first embodiment. Further, as shown in FIG. 6, it is assumed that the suspicious person 403 appears in the moving image 611 being displayed, and the abnormal area 613 moves to the tracking abnormal area 615 in the moving direction 614 of the abnormal area. At this time, the image display device of the present invention performs an operation of continuously displaying enlarged still images centered on the moving suspicious person 403.

  First, the user presses the abnormal part tracking button 612. The display video generation control unit 104 detects that the display mode of the display video control information in the display video control information storage unit 109 has been changed using the display video control unit 101 as indicated by a thick arrow, and detects an abnormality. Based on the information, the abnormal region 613 and the tracking abnormal region 615 are determined. Next, the still image of the part is decoded and a display format instruction signal is sent to the still image decoder 105 and the display video generation unit 107 so as to be displayed as the tracking still image 616. When the abnormal area 613 moves with time, the indication of the display area may be changed based on the change of the abnormal area 613 in the abnormality detection state information. That is, the display video generation control unit 107 changes the display pattern in time series based on the received abnormality detection state information.

  In addition, as a display method when there are a plurality of abnormal areas 613 indicated by the abnormality detection state information, a display that includes all the portions determined to be abnormal, or a display that has the largest area determined to be abnormal is centered. May be displayed.

  In addition, when there is no abnormal part on the way, a still image may not be particularly displayed, and the part finally determined to be abnormal may be continuously displayed. In addition, a display device may be devised to clearly indicate a location being tracked on the displayed moving image side with a red frame or the like.

  According to the second embodiment of the present invention as described above, a high-definition still image is obtained by capturing only the abnormal part while grasping the whole by using the abnormality detection state information output from the image input unit 202 such as a surveillance camera. Since the user can perform the enlargement display operation without bothering the editing operation and the operation, the monitoring video can be viewed efficiently.

  As described above, the image display system of the present invention, when displaying a moving image and a still image on the display, allows the display pattern to be changed flexibly while synchronizing the position and time of the moving image and the still image, and still images. Easily perform operations that unify movies, recall still images corresponding to the specified area from the movie frame that the user is watching while watching the movie, display it enlarged, and replay display in that enlarged region, such as network camera abnormalities By tracking and displaying an enlarged still image of the area that moves according to the detection state information, it is possible to provide an image display system capable of various advanced expression methods for analyzing a monitoring image. The image display system of the present invention combines the still image synchronization information recorded in the still image information storage unit with the moving image synchronization information recorded in the movie information storage unit, so that enlarged display, replay display, and enlarged still image tracking display are performed. There is an advantage that the display video generation control unit such as, etc. associates a moving image and a still image temporally and spatially.

  The image display device and the image display system of the present invention have an effect that enables unified display control while synchronizing a moving image and a still image with an easy operation, and are useful for a monitoring system and the like.

1 is a block diagram of an image display device according to a first embodiment of the present invention. (A) A diagram for explaining information recorded in the still image storage unit, (B) a diagram for explaining information recorded in the moving image storage unit, and (C) for explaining information recorded in the display video control information storage unit. Figure (A) The figure which shows an example of the display screen of the moving image display of embodiment of this invention, (B) The figure which shows an example which expands and displays the still image part of the display screen of embodiment of this invention The figure explaining the thumbnail display of the embodiment of the present invention The block diagram of the image transmission display system in the 2nd Embodiment of this invention The figure explaining the method of tracking and enlarging the abnormal part in embodiment of this invention The figure which shows the display method of the conventional moving image still image display apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Display video control means 102 Still image storage part 103 Movie storage part 104 Display video generation control part 105 Still picture decoder 106 Movie decoder 107 Display video generation part 108 Display apparatus 109 Display video control information storage part 201 Reception part 202 Image input means

Claims (2)

Still image decoding means for reading and decoding the still image information from still image storage means in which still image information including still image video information and still image synchronization information associated therewith is recorded;
A moving picture decoding means for reading out and decoding the moving picture information from a moving picture storage means in which moving picture information including moving picture video information and moving picture synchronization information associated therewith is recorded;
Display video control means for generating display video control information from an input display video control instruction;
Based on the display video control information, generates a display format instruction that defines a spatial layout and a temporal display pattern of the video display, and further decodes still image information and moving image information necessary for the display format. Display image generation control means for instructing the still image decoding means and the moving image decoding means;
Display video generation means for processing the still picture information decoded by the still picture decoding means and the moving picture information decoded by the moving picture decoding means in accordance with the display format instruction, and generating display video information;
And displays the video information included in the display image information said generated with a display means for outputting still image synchronization information and the video synchronization information to the display image control unit included in the display image information image A display device,
The display video control unit generates the display video control information according to the input control instruction based on the still image synchronization information and the video synchronization information output from the display unit ,
The display video generation control means determines an abnormal area based on abnormality detection state information input from the outside, and decodes and displays a still image of the abnormal area when displaying only a moving image. In addition, when the abnormal area moves with time, an image display device that continuously displays enlarged images of the still image based on the abnormal state information as a tracking abnormal area . Still image and photographing a moving image, an image capturing means for detecting a movement of any object to be shot in the captured video, and outputs together with the still picture information and the video information as an abnormal detection state information,
The image display device according to claim 1 ,
An image transmission display system comprising:

Images