JP2003324718A - Monitoring system and method, and program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a monitoring system for photographing an image of a wide range, recording photographed image data to a memory and enhancing the operability of image analysis. <P>SOLUTION: A memory 30 stores a compressed image comprising (M×N) still pictures photographed by a camera unit 3. A display apparatus 2 displays a panoramic entire image produced from the compressed image stored in the memory 30. Image data corresponding to an area pointed out by a mouse 36 are read from the memory 30 among the entire images displayed on the display apparatus 2 and the image of the area pointed out by the mouse 36 is updated. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a monitoring system and method, a program and a recording medium applied to a surveillance camera or the like.

[0002]

2. Description of the Related Art Conventionally, a monitoring system for monitoring a wide range of conditions has been used. For example, monitoring systems are used for maritime surveillance, river surveillance, on-site surveillance, wildlife behavior observation, etc. Since it is necessary to capture a wide range of images, a video camera with a large number of pixels is used. Therefore, the system price becomes high, and there is a problem in terms of cost. On the other hand, it has been proposed that a still image is captured while sequentially shifting the capturing range of the camera, and a large number of still images are connected to generate an image in the range to be monitored.

By the way, in the above-mentioned monitoring system, not only can the photographed image be monitored in real time, but also the photographed image can be recorded in a memory and the images photographed in the past can be monitored. By monitoring images taken in the past and analyzing the images, the cause of a specific event such as an accident or disaster can be clarified.

[0004]

However, the conventional monitoring system has a problem that it takes time to read out and display the image stored in the memory and the operability is poor. This is because the image used for image analysis is required to have a high resolution and a clear image, and therefore the data per image becomes very large.

Therefore, an object of the present invention is to provide a monitoring system and method, a program, and a recording medium, which can shorten the time for reading the image data stored in the memory.

[0006]

In order to solve the above-mentioned problems, a first invention of the present application is to provide an image capturing section for capturing a video image, and an original image composed of a plurality of still images captured by the image capturing section.
Alternatively, a storage unit that stores a plurality of compressed images formed by compressing a plurality of still images, an image display unit that displays the images, and an instruction unit that indicates an area on the image displayed on the image display unit. And the panoramic whole image generated from the original image or the compressed image accumulated in the accumulating unit is displayed on the image display unit, and only the image of the indication area indicated by the instruction unit on the whole image is the image other than the indication area And a control unit for updating with images having different photographing times.

A second invention of the present application is a photographing step for photographing a video, an original image composed of a plurality of still images photographed in the photographing step, or a compressed image composed of a compressed still image obtained by compressing a plurality of still images. A storage step for storing a plurality of images in a storage section, a display step for displaying a panoramic whole image generated from the original image or the compressed image stored by the storage step, and specified on the whole image displayed by the display step A monitoring method comprising: an instructing step for instructing an area; and an updating step for updating only an image of the instructed area instructed by the instructing step with an image having a different shooting time from an image other than the instructed area.

A third aspect of the invention of the present application is a photographing step for photographing a video image, an original image composed of a plurality of still images photographed in the photographing step, or a compressed image composed of a compressed still image obtained by compressing a plurality of still images. A storage step for storing a plurality of images in a storage section, a display step for displaying a panoramic whole image generated from the original image or the compressed image stored by the storage step, and specified on the whole image displayed by the display step A program characterized by causing a computer to execute an instruction step of instructing an area, and an updating step of updating only an image of the instructed area instructed by the instructing step with an image having a different shooting time from an image other than the instructed area Is.

A fourth invention of the present application is a photographing step for photographing a video image, an original image consisting of a plurality of still images photographed in the photographing step, or a compressed image consisting of a compressed still image obtained by compressing a plurality of still images. A storage step for storing a plurality of images in a storage section, a display step for displaying a panoramic whole image generated from the original image or the compressed image stored by the storage step, and specified on the whole image displayed by the display step A program for causing a computer to execute an instruction step of instructing an area and an updating step of updating only an image of the instruction area instructed by the instruction step with an image having a different shooting time from an image other than the instruction area is recorded. It is a characteristic recording medium.

In the monitoring system of the present invention configured as described above, an original image composed of a plurality of still images photographed by the imaging unit or a compressed image composed of a compressed still image obtained by compressing a plurality of still images is accumulated. Accumulated in the department. The panoramic whole image generated from the original image or the compressed image stored in the storage unit is displayed on the display unit. Since the still image or the compressed still image corresponding to the designated area in the displayed whole image is read from the storage unit and displayed, only the image in the specific area of the whole image can be updated.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a schematic configuration of an embodiment of the present invention. The computer 1 to which the display 2 is connected controls the camera unit 3. The example of FIG. 1 is an example of a system in which one computer 1 controls two camera units 3 and another computer 1 ′ having a display 2 ′ controls another camera unit 3 ′. One computer can control multiple camera units.

The camera unit 3 comprises a pan tilter section 4 and a camera section 5 which are integrally formed. The camera unit 3 is installed so that a distant target area can be photographed. As an example, the camera unit 5 has a telephoto lens having a magnification of 10 times, 70 times, etc., and is capable of photographing a place away from several tens of meters to several kilometers.

The camera unit 5 is, for example, a digital still camera capable of turning on a shutter in synchronization with an external trigger.
deoGraphics Array, 640 x 480 pixels, XGA (eXtende)
d Graphics Array, 1024x768 pixels, SXGA (Super
eXtended Graphics Array, 1280 x 1024 pixels). 30 fps for VGA image sensor
Image data is output at a rate of (frames / second) and X
In the case of a GA image sensor, 15 fps (frame /
The image data is output at a rate of 2 seconds), and in the case of the SXGA image sensor, the image data is output at a rate of 7.5 fps (frames / second).

Video data is transmitted from the camera unit 3 to the computer 1 via the bus 6. Bus 6
Transmits the transmission path of the video data and the control signal of the camera unit 3. The configuration described above is the same for the computer 1'and the camera unit 3 '.

In the computers 1 and 1 ', image data from the camera units 3 and 3'is stored in a memory, and a GUI (Graphical User Interface) for operation is used as described later.
The camera unit 3, so that the user can take an image of a desired target area with the camera unit 3, 3 '.
3'can be controlled. Compression encoding For example, JPEG (Joint
Photographed images are compressed by Photographic Experts Group).

The computers 1 and 1'are connected to each other via a LAN 7. Another computer 8 is further connected to the LAN 7. Reference numeral 9 is a display of the computer 8. Computer 8 is L
It receives image data and the like from the computers 1, 1'via the AN 7, stores video data in the archive 10, and further processes the image data. For example, processing such as face recognition, baggage recognition, environment recognition, and vehicle recognition is performed using video data. The archive 10 is capable of accumulating a large amount of data like a tape streamer.

FIG. 2 shows a more detailed structure of the computer 1 and the camera unit 3 in the above-mentioned monitoring system. In the example of FIG. 2, components of the camera unit and the computer are connected to a common controller bus 21 indicated by reference numeral 21.

The pan tilter section includes a pan section 4a and a tilt section 4
It consists of b. Each of the pan unit 4a and the tilt unit 4b has, for example, a stepping motor as a drive source, and pans or tilts the camera unit according to a control signal supplied from the controller CPU 33 via the controller bus 21. The camera unit is mounted on the pan tilter unit. Here, pan means to rotate the camera in the horizontal direction, and tilt means to rotate the camera in the vertical direction. As an example, the maximum pan angle is 180
And the maximum value of the tilt angle is 50 °.

As will be described later, within the maximum movement range of the camera section, the camera section is moved within a range of tilt angle = ± 15 ° and pan angle = ± 50 °. Every time the center of imaging is moved by the angle of view, the shutter is turned on and a still image (hereinafter, appropriately referred to as a frame) is captured. The total of M (eg 8) in the vertical direction and N (eg 16) in the horizontal direction (M × N
= 8 × 16 = 128 frames) are taken in order,
These are compressed and connected to form one whole image. Each frame is, for example, XGA (1024x768 pixels)
It is an image. Therefore, the 128 frames form an image of about 100 million pixels (1024 × 16 = 16,384 pixels in the horizontal direction and 768 × 8 = 6,144 pixels in the vertical direction), ignoring the overlapping portion. It takes about 5 seconds to shoot 128 frames. The overlapping portion is, for example, 16 pixels vertically and horizontally.

The camera section has a structure of a digital still camera and comprises a lens section 22, a focus / zoom / iris control section 23 and an image pickup section 24. The focus / zoom / iris control unit 23 is controlled by a control signal supplied from the controller CPU 33 via the controller bus 21. The image pickup unit 24 includes a solid-state image pickup device such as a CCD and a camera signal processing circuit. The digital video signal from the image pickup unit 24 is an IEEE (Institute of Elec
Trical and Electronics Engineers) 1394 interface 25 is written to the buffer memory 26.

The output data of the buffer memory 26 is JPE.
The image data is supplied to the G encoder / metadata adding unit 27 and the image data is converted into JPEG data. JPEG is one of the compression methods, and other compression methods may be used or may not be compressed.

The camera unit 3 is provided with a GPS (Global Positioning System) 28 for detecting its position. By including the GPS 28, it is possible to record the data of the installation location of the camera, detect the orientation of the camera, and control the orientations of the plurality of cameras in conjunction with each other. The GPS 28 is the controller bus 21
It is controlled by a control signal supplied from the controller CPU 33 via the.

The output signal of the GPS 28 is supplied to the metadata generation unit 29, and the position information (information such as latitude / longitude, azimuth and altitude) based on the positioning result of the GPS 28 and metadata (time, parameters of the camera unit (magnification). , Focus value, iris value, etc.) are generated. The position information and the metadata are supplied to the JPEG encoder / metadata adding unit 27, and the position information and the metadata are added to the JPEG data.

J with metadata and position information added
The PEG data is accumulated in the main memory 30 such as a hard disk and is supplied to the graphic controller 31 and the image compression unit 32. In this specification, accumulation in the main memory 30 is called recording, and reading data from the main memory 30 is called reproduction. Further, displaying an image that is currently being photographed without passing through the main memory 30 is called a live mode, and playing back and displaying data recorded in the past from the main memory 30 is called a view mode.

The main memory 30 has a function as a server. For example, as a result of compressing an XGA image by JPEG, the data amount of one frame is 100 kbytes, and the data amount of 128 images is 12.8 Mbytes. If the main memory 30 has a capacity of about 80 Gbytes, it is possible to store one day's worth of JPEG data. In view mode, main memory 30
However, it is possible to reproduce older data stored in a storage device such as an archive.

JPE read from the main memory 30
The G data is supplied to the graphic controller 31. The image compression unit 32 generates a compressed image or thumbnail from the JPEG data from the JPEG encoder / metadata addition unit 27 or the JPEG data read from the main memory 30. For example, by thinning out each of the vertical direction and the horizontal direction, a panoramic whole image is formed. The image compression unit 32 also performs a compression process for forming a movable range image, which will be described later.
As described above, in the case of XGA, data of about 100 million pixels is processed by the JPEG compression and the image compression unit 32.
A panoramic whole image such as (400 × 1000 pixels) is formed. Although the movable range image is also a thumbnail, it is a coarser image than the entire image.

The graphic controller 31 is a JPE
The G data is converted into bitmap data, and graphics processing is performed so that a desired image is displayed on the screen of the display 2. That is, the movable range image display, the whole image display, the selection image display, and the GUI display such as buttons are displayed on the screen of the display 2. The details of the display will be described later.

Further, the graphic controller 31 is
Image processing is performed to detect image changes. An image change is a change that has occurred with respect to a reference image. For example, in view mode, a comparison is made with a previously stored reference image to detect image changes. As the reference image, the image at the predetermined time on the previous day is set, the difference between the pixels of the image accumulated after that and the reference image is detected, and the change occurs when the absolute value of the pixel difference is equal to or larger than the predetermined value. Detect it as something. For the detection of the difference, a method of calculating the difference value of the pixel at the same position for each frame at the spatially same position between the image to be compared and the reference image can be used. Instead of detecting the difference for all pixels, the difference may be calculated for the representative pixel or the thinned pixels. Further, by limiting the predetermined color, it is possible to perform change detection focusing on an object of a predetermined color.

When a change is detected, an alarm is displayed on the display 2, for example, a frame in which a change is detected can be distinguished from other frames. Specifically, the alarm can be displayed by a method such as brightness change, color change, blinking, or the like. As the reference image, a predetermined one can be arbitrarily selected from the accumulated images.

As described above, the controller CPU 33 connected to the controller bus 21 controls the camera lens (for example, focus) and the exposure (for example, focus).
(Aperture, gain, electronic shutter speed, etc.), white balance control, image quality control, and the like, and also controls the pan section 4a and the tilt section 4b.

Reference numeral 34 is an I / O port. I
For the / O port 34, the key 35 and mouse 36
, And a memory card 37 and a clock 38 are connected to the I / O port 34. The JPEG data to which the position information and the metadata accumulated in the main memory 30 are added can be written in the memory card 37. Further, time data is obtained from the clock 38.

In FIG. 2, each component is connected to the controller bus 21. However, the camera unit and the computer are installed in separate places, and both are connected to the IEEE.
You may make it connect by E1394, USB, etc. In this case, an optical fiber is used as the physical transmission line. If an optical fiber is used, the camera unit and the control computer can be arranged apart from each other by several hundred meters to several kilometers. Furthermore, both may be connected by a wireless LAN.

FIG. 3 shows a GU according to an embodiment of the present invention.
The screen example of I is shown. Hereinafter, with reference to FIG. 3, a display unit, operation buttons, a display area, and the like provided on the GUI screen according to the embodiment of the present invention will be described. A movable range image display unit 101, an entire image display unit 102, and a selected image display unit 103 are arranged on one screen.

A movable range image is displayed on the movable range image display section 101. The movable range image is an image showing the maximum range that can be captured by the camera unit, and is composed of a plurality of frames. As described above, the maximum value of the pan angle is 180 ° and the maximum value of the tilt angle is 50 °, and the movable range image is generated from a plurality of frames captured in this maximum movable range. For example, a camera unit is installed, and at the start of shooting, the camera is moved over the maximum movable range, and a thumbnail obtained by thinning out pixels in the vertical and horizontal directions with respect to the resulting image composed of multiple frames is used as the movable range image. .

In the movable range image display section 101, the position where the optical axis of the lens of the camera unit is currently facing (Came
The ra live position) is indicated by the intersection of the line segment 101a and the line segment 101b. By moving these line segments 101a and 101b, a desired position in the movable range image can be designated,
The imaging direction can be controlled in the direction of the designated position. Then, with the direction of the designated position as the center or home position, (M
× N) frames (still images) are captured, stored, or displayed. Not only the line segments 101a and 101b, but when an arbitrary position on the display screen displayed on the movable range image display unit 101 is pointed by a pointer, the optical axis of the lens of the camera unit is directed to the position corresponding to this point. The camera unit may be controlled.

The panoramic whole image is displayed on the whole image display section 102. The entire image is an image obtained by compressing the JPEG data corresponding to the captured original image by the image compression unit 32. Monitoring can be performed by looking at the displayed whole image. further,
As described above, when an image change is detected, an alarm occurs that the frame in which the change is detected in the entire image displayed on the entire image display unit 102 is displayed differently from other frames.

The selected image display section 103 displays the selected image. The selected image is an image obtained by enlarging a part of the entire image. It can be enlarged by displaying an uncompressed 1-frame original image. Further, the image can be enlarged by digital signal processing.

The EXIT button 104 is a button for turning off the power of the monitoring system. Camera s
The ystem OFF button 105 is a button for turning off the power of the camera unit.

The VIEW MODE button 106 is a button for switching the mode of the monitoring system to the view mode. The view mode is a mode in which the whole image and the partial image are displayed based on the image data stored in the memory 30 or another server.

The LIVE MODE button 107 is a button for switching the mode of the monitoring system to the live mode. The live mode is a mode in which the entire image and the partial image are displayed based on the frame currently taken by the camera unit.

The Compas display area 108 is an area for displaying a compass indicating the direction in which the optical axis of the lens of the camera is oriented. GPS Data display area 109
Is an area for displaying the latitude, lightness and altitude of the place where the camera is installed, and the date and time of shooting. In addition,
The data displayed in the areas 108 and 109 is the data measured by the GPS 28 provided in the camera unit.

The View offset button 110 is
It is a button for adjusting the position of the selected frame. View offset button 110A, 110
B, 110C and 110D respectively indicate one frame selected by the pointer in the whole image displayed on the whole image display unit 102 in the upward direction, the downward direction, the left direction,
It is for moving to the right. A plurality of frames forming the entire image are connected to adjacent frames by overlapping a predetermined number of pixels, for example, 16 pixels. By moving the frame within the range of this overlapping portion, the consistency with the adjacent frames can be obtained, and the connected state can be made smooth.

The mode display area 129 is an area for displaying mode information, alarm information, error information and the like. The mode information is information for informing the user of the mode of the monitoring system, and specifically, information such as live mode and view mode. The alarm information is information for urging the user to warn, and is displayed, for example, when the frame movement limit is reached by the above-mentioned View offset button 110. The error information is information for informing the user of an error occurring in the monitoring system.

Camera Control Unit 111
Is a ZOOM button 112, a FOCUS button 113,
IRIS button 114, Camera Configuration button 1
15 and a White Balance button 116. The ZOOM button 112 is a button for adjusting the zoom of the camera unit. The FOCUS button 113 is a button for adjusting the focus of the camera unit. The IRIS button 114 is a button for adjusting the iris of the camera unit. Camera
The Configuration button 115 is a button for adjusting the γ characteristic, shutter speed, gain characteristic and the like of the camera unit. The White Balance button 116 is a button for adjusting the white balance of the camera unit. When the camera system is in the view mode, the CameraControl section 1
The display of 11 may be omitted.

The SELECT button 117 is a button for displaying the select screen in the view mode. The select screen is an image for specifying a region desired to be reproduced and recorded by the frames forming the entire image.

FIG. 4 shows an example of the selection screen. Figure 4
As shown in, on the selection screen, a grid-like display showing frame delimiters is superimposed on the entire image. In the select screen, for example, when a frame at a desired position is pointed by a pointer, that frame is selected, and in order to display the selection,
The brightness, resolution, contrast, etc. of the designated frame change.

The REC MODE selection menu 118 is
It is a pull-down menu for selecting a recording mode. In this pull-down menu, a recording mode in which the image size to be recorded and the recording method (RUN or SINGLE) are combined is displayed. The image size is (8 x 1
6) The whole image composed of the frames, the partial image composed of the selected (4 × 8) frames of the entire image, and the selected (2 × 4) of the entire images Any of partial images composed of one frame is possible. The partial image is at the position selected from the selection screen. The recording method RUN is a method of recording a captured image generated in a predetermined cycle (for example, a 5-second cycle), and SINGLE is a method of recording only once. A combination of these can be selected as the recording mode.

The Stage Config (Stage Configuration) button 119 is a button for finely adjusting the precision of moving the stage. The message area 120 is an area for displaying the connection status between the control computer and the camera unit and the control status of the stage of the camera unit. When the computer for control and the camera unit are connected, FIG.
As shown in, the message area displays “IMAGE SER
“VER CONNECT” is displayed. If the stage of the camera unit is in a controllable state, “STAGE CONTRO” is displayed in the message area.
L ACTIVE ”is displayed.

The REC button 121 is a button for starting image recording, and when this button is pressed, the R button is pressed.
Recording is started according to the recording mode selected in the EC mode menu. Specifically, RUN (8 × 1
6), RUN (4x8), RUN (2x4), SELE
CT SINGLE RUN (8x16), SELECT
TSINGLE RUN (4x8), SELECT S
Recording according to a mode selected from the modes such as INGLE RUN (2 × 4) is started.

The PLAY button 122 is a button for reproducing the image data stored in the server (main memory 30). Specifically, when the PLAY button 122 is pressed, the accumulated data display screen is displayed. Information for identifying the accumulated image data is displayed on the accumulated data display screen. This information is based on the information described in the direction file described later.

FIG. 5 shows an example of the accumulated data display screen. As shown in FIG. 5, the accumulated data display screen includes
Minimize button A101, maximize button A102, close button A103, date designation field A104, time designation field A1
05, accumulated data display column A106, latest accumulated data display column A107, OK button A108, cancel button A
109 and a storage section change check button A110 are displayed.

The minimize button A101 is a button that is clicked when the accumulated data display screen is minimized to, for example, an icon. The maximize button A102 is a button that is clicked when the accumulated data display screen is maximized and the entire display range of the monitor is used for display. The close button A103 is a button clicked when closing the accumulated data display screen.

In the date designation field A104, the date of the accumulated data to be displayed on the whole display portion is designated. For example, a button A104a provided at the right end of the date designation field A104
By clicking, the date of the accumulated data that can be displayed is displayed in a pull-down menu format, and the date may be selected from the displayed dates.

In the time designation column A105, the time of accumulated data to be displayed on the entire display section is designated. For example, a button A105a provided at the right end of the time designation field A105
By clicking, the time of the displayable accumulated data is displayed in a pull-down menu format, and the displayed time may be selected.

In the accumulated data display column A106, accumulated data of the date and time designated in the date designation column A104 and the time designation column A105 from the accumulation unit is displayed. In the latest accumulated data display field A107, the latest accumulated data among the accumulated data accumulated in the accumulator is displayed.
Further, the latest accumulated data may be displayed from the accumulated data within the date and time designated in the date designation column A104 and the time designation column A105.

The OK button A108 is a button that is clicked when the desired accumulated data is designated.
The cancel button A109 is a button clicked when closing the accumulated data display screen. The storage section change check button A110 is a check button for inputting a check when changing the storage data reading destination from the storage section to, for example, a removable semiconductor memory (memory card).

Returning to FIG. 3, the STOP button 123 is a button for stopping the recording or reproducing operation. The STOP button 123 may be displayed by pressing the REC button 121 or the PLAY button 122.

Set Camera Center POS (Set Camera Cente
The r POSITION) button 124 is a button for designating the direction in which the camera is currently facing as the center of the (8 × 16) image.

The HOME button 125 is a button for controlling the camera unit and directing the optical axis of the lens of the camera unit to the home position. The home position is the position where the camera faces the leftmost position.
LIVE / VIEW POSITION button 126
Is a button for panning or tilting the camera.

ZOOM buttons 127A and 127B
Is a button for enlarging and reducing the selected image displayed on the selected image display unit 103. MAX VIEW
The button 128 is a button for enlarging and displaying the selected image on another screen, for example, the entire image display unit 102.

Next, with reference to FIG. 6, an example of a method of creating a whole image according to an embodiment of the present invention will be described.
As shown in FIG. 6, in the camera unit 3, the camera unit 5 is installed on the pan head of the pan tilter unit 4, and the imaging direction is variable from the home position. In FIG. 6, when the photographed (M × N) frames are viewed from the camera side, the numbers 1, 2, ... Numbers are attached in order from 1, 2, ..., N.
The home position is, for example, a position where a frame of coordinates (1, 1) is captured.

When the frame at the coordinate position (1, 1) is photographed, the camera unit 3 is tilted downward,
The frame at the coordinate position of 1) is photographed, then the frames at the coordinate positions of (3, 1) ..., (M, 1) are photographed in order, and then the top coordinate position of the second column is photographed. (1,2)
Frame is taken. Hereafter, each frame is (M,
Shoot up to the frame at the coordinate position of N). As described above, each frame has a 16-pixel overlapping portion with another frame. Each captured frame undergoes processing such as JPEG compression and storage in the main memory 30.

As described above, each frame is, for example, XG.
In the case of A (1024 x 768 pixels) image, the image consisting of 128 frames is (1
(024 x 16 = 16,384 pixels, vertical direction is 768 x 8 = 6,144 pixels)
Is an image of about 100 million pixels. Overall image display unit 1 described above
In 02, a compressed image or thumbnail image formed from this image is displayed, and in the selected image display unit 103, for example, one frame of XGA image is displayed. Therefore, an image with extremely high resolution can be displayed on the selected image display unit 103, and an unclear image in the entire image can be displayed as a clear image in the selected image.

FIG. 7 shows the range that can be photographed in one frame when the camera unit is equipped with a 75 × telephoto lens. When shooting an object at a distance of 100 m from the camera unit, one frame, vertical 8.
It is possible to shoot an area of 7 m × width 1.17 m. For example, when XGA is used as the image pickup device of the camera unit, the range of vertical 0.87 cm × horizontal 1.17 cm of the subject can be represented by about 1 pixel.

When a subject at a distance of 200 m from the camera unit is photographed, one frame has a length of 1.
It is possible to shoot an area of 74 m × 2.34 m in width.
For example, when XGA is used as the image pickup device of the camera unit, the range of 1.74 cm in height and 2.34 cm in width of the subject can be represented by one pixel.

When a subject at a distance of 500 m from the camera unit is photographed, one frame is used, and the vertical length is 4.
It is possible to shoot a range of 36 m x 5.84 m in width.
For example, when XGA is used as the image sensor of the camera unit, the range of vertical 4.36 cm × horizontal 5.84 cm of the subject can be represented by one pixel.

A data management method for saving the acquired image data in the archive 10, the main memory 30, etc. will be described with reference to FIG. As described above, the images of (M × N) frames are captured, compressed, and stored at predetermined time intervals. As shown in FIG. 8A, the position of each frame is defined by M rows and N columns. For example, the position address (1,1) specifies the uppermost frame at the right end.
Each frame has this position address and recording time information as a file name. The time information is composed of year / month / day / hour / minute / second. Therefore, the file name of each frame is (year, month, day, hour, minute, second, position address).

Further, as shown in FIG. 8B, (M × N)
The direction file is defined corresponding to the formation of an overall image by one frame.
The direction file has the same data as the file name (that is, year, month, day, hour, minute, second, position address) of the frame having the position address of (1, 1), so that the set of (M × N) frames is set. Is defined. In addition, the direction file has location information and metadata for this set of frames. The position information and the metadata are generated by the metadata generation unit 29. That is, the direction file has position information such as latitude / longitude, azimuth, and altitude, and metadata information such as camera unit parameters (magnification, focus value, iris value, etc.).

FIG. 9 is a flow chart for explaining the operation of frame capturing of the whole image on the whole image display unit 102 according to the embodiment of the present invention. LIVE M
The ODE button 107 is pointed by the pointer and the REC
When the button 121 is designated by the pointer, the control algorithm of this flowchart is called.

First, in step S101, when the capture position is indicated by the pointer in the movable range image displayed on the movable range image display unit 101, it is determined which position of the movable range image the whole image is. Then, the capture coordinates of the entire image are confirmed.

Next, in step S102, the start position of capturing the entire image is calculated, and the pan section 4a and the tilt section 4b provided in the camera unit 3 are controlled based on the calculation result, and the lens of the camera unit 3 is controlled. Move the optical axis to the capture start position. Here, the capture start position is the center position of the frame captured first.

Next, in step S103, the camera unit 3, specifically, the lens unit 22, the focus / zoom / iris control unit 23, the image pickup unit 24, and the like are controlled, frames are captured, and the computer 1 is used as image data. Is supplied to.

Next, in step S104, the image data supplied from the camera unit 3 is converted into a predetermined image format, for example, JPEG.

Next, in step S105, metadata and position information are added to the image data converted into a predetermined image format.

Next, in step S106, the image data to which the metadata and the position information are added is recorded in the main memory 30.

Next, in step S107, the image data converted into the predetermined image format is displayed at the designated address of the whole image display unit, for example, (0,0).

Next, in step S108, the moving distance of the optical axis of the lens of the camera unit to the next image data is calculated.

Next, in step S109, the pan unit 4 is operated based on the moving distance calculated in step S108.
a and the tilt portion 4b are controlled to direct the optical axis of the lens of the camera unit to the center of the next frame.

Next, in step S110, the number of captured frames is calculated. For example, the number of frames may be counted by incrementing each time a frame is captured.

Next, in step S111, it is determined whether the calculated number of captured frames has reached the specified number of frames. When it is determined that the designated number of frames has been reached, the control proceeds to step S112, and when it is determined that the designated number of frames has not yet been reached, the control returns to step S103. The designated number of frames is calculated in advance according to the mode selected in the REC MODE selection menu 118. Specifically, when the RUN (8x16) mode is selected, the number of frames is 128, and R
When the UN (4x8) mode is selected, the number of frames is 32, and when the RUN (2x4) mode is selected, the number of frames is 8.

Next, in step S112, the distance from the position corresponding to the optical axis of the lens of the current camera unit to the capture start position of the whole image display section 102 is calculated.

Next, in step S113, the pan portion 4a and the tilt portion 4b are controlled based on the distance calculated in step S112, and the optical axis of the lens of the camera unit is directed to the center of the frame which is the start position for capturing. Make it

Next, in step S114, it is determined whether or not the number of updates of the whole image display unit 102 has reached a predetermined number of updates. Specifically, REC
In the MODE selection menu 118, it is determined whether the select mode or the run mode has been selected. Therefore, when it is determined that the select mode is selected in the REC MODE selection menu 118, the control proceeds to step S115, and when it is determined that the run mode is selected, the control proceeds to step S117.

That is, when the select mode is selected in the REC MODE selection menu 118, the number of updates is set to "1", and therefore all the frames displayed on the whole image display section 102 are displayed only once. Since it is captured, recorded, and displayed, the process of capturing, recording, and displaying again is not performed. On the other hand, REC
When the run mode is selected in the MODE selection menu 118, the number of updates is set to “infinite”. Therefore, until the end of import, that is, until the STOP button 123 is pressed, import and record are repeated. Then
The process of displaying is repeated.

Next, in step S115, the distance from the capture start position of the whole image display section 102 to the center of the whole image display section 102 is calculated, and based on the calculation result, the pan section 4a provided in the camera unit is calculated. The tilt unit 4b is controlled to move the optical axis of the lens of the camera unit to the center of the whole image display unit 102.
Here, the center portion of the entire image is, for example, the center position of 8 × 16 frames.

Next, in step S116, the operation of the pan portion 4a and the tilt portion 4b, for example, the stepping motor is stopped. Then, the control algorithm of this flowchart ends.

Next, in step S117, it is determined whether or not the end of capture is instructed. In particular,
It is determined whether or not the STOP button 123 is designated by the pointer. If it is determined that the STOP button 123 is designated by the pointer, the control proceeds to step S115, and if it is determined that the STOP button 123 is not yet designated by the pointer, the control returns to step S103.

FIG. 10 is a flow chart for explaining the operation of reproducing the accumulated image data according to the embodiment of the present invention. When the play button is clicked, the control algorithm of this flowchart starts.

First, when the play button is clicked in step S201, the accumulated data display screen shown in FIG. 9 is displayed, for example, in a pop-up window format.

Next, in step S202, the date is designated in the date designation column A104 and the time designation column A10 is set.
In 5, it is determined whether the time is designated. If it is determined that the date is specified in the date specification field A104 and the time is specified in the time specification field A105, the control proceeds to step S203, where the date is specified in the date specification field A104 and the time is specified in the time specification field A105. If it is judged that both designations or one of the designations has not yet been made,
Until the date is designated in the date designation column A104 and the time is designated in the time designation column A105, this step S20 is performed.
The control of 2 is repeated.

Next, in step S203, the movable range image and / or the entire image is displayed from the accumulated data of the designated date and time.

FIG. 11 is a flow chart for explaining the operation of the monitoring system according to the embodiment of the present invention in the multipoint view mode. The multipoint view mode is a mode in which only the image at the specified position in the entire image is played back in time.

First, in step S301, SEL
It is determined whether the ECT button 117 has been pressed.
If it is determined that the SELECT button 117 has been pressed, the process proceeds to S302 and the SELECT button 117 is pressed.
If it is determined that is not pressed, the process proceeds to S301.

Next, in step S302, the select screen as described with reference to FIG. 4 is displayed. Then, in step S303, it is determined whether or not the frame in the selected image is selected by the pointer. If it is determined that the frame in the selected image is selected by the pointer, the process proceeds to step S304,
If it is determined that the frame in the selected image is not selected by the pointer, the process proceeds to step S306.

Next, in step S304, the display of the selected frame is switched. For example, the brightness of the selected frame is switched.

Next, in step S305, the coordinate position of the selected frame is confirmed.

Next, in step S306, it is determined whether or not the CLOSE button 153 on the select screen has been pressed. If it is determined that the CLOSE button 153 is pressed, the process proceeds to step S307 and CLOS
If it is determined that the E button 153 is not pressed, the process proceeds to step S303.

Next, in step S307, the PLA
It is determined whether the Y button 122 has been pressed. PL
If it is determined that the AY button 122 is pressed, the process proceeds to step S308, and if it is determined that the PLAY button 122 is not pressed, the process proceeds to step S307.
Move on to.

Next, in step S308, the accumulated data display screen as described with reference to FIG. 5 is displayed. Then, in step S309, the date and time range desired to be reproduced is designated on the accumulated data display screen,
It is determined whether or not the OK button 168 has been pressed. If it is determined that the date / time range is designated and the OK button 168 is pressed, the process proceeds to step S310. If it is determined that the date / time range is designated and the OK button 168 is not pressed, the process proceeds to step S309. Move.

Next, in step S310, the image data corresponding to the coordinate position confirmed in step S305 is read from the memory 30 among the image data within the date / time range specified in step S309. For example, of the image data within the date / time range designated in step S309, the image data corresponding to the coordinate position confirmed in step S305 is read out from the memory 30 in the order of newest photographing date / time.

Next, in step S311, the coordinate position confirmed in step S305 is set to step S31.
An image is displayed based on the image data read at 0.

In step S312, it is determined whether all the image data designated on the accumulated data display screen have been read. If it is determined that all of the image data specified on the accumulated data display screen has not been read, the process proceeds to step S310, and it is determined that all of the image data specified on the accumulated data display screen has been read. , The processing ends.

The image other than the designated area in which the area is designated by the pointer is an image whose photographing date is newer than the image updated in the designated area, for example, an image which is currently photographed by the camera unit 3. In this case, for example, only the image of the designated area designated by the pointer is
Updated to go back in time.

Further, the image other than the designated area may have a photographing date older than the image updated in the designated area. In this case, for example, the designated area image in which the area is designated by the pointer is updated from the past to the present.

According to the embodiment of the present invention, the following advantages can be obtained. (M × N) frames photographed by the camera unit 3 are compressed and stored in the memory 3
Stored at 0. The panoramic whole image generated from the (M × N) frames accumulated in the memory 30 is displayed on the display 2, and the image data corresponding to the area designated by the mouse 36 in the displayed whole image is stored in the memory. It is read from 30 and displayed in the above area. Therefore, only the frame of the specific area in the entire image can be updated and displayed. Therefore, the memory 30
It is possible to shorten the time required to read the image data stored in. Therefore, the operability of image analysis in the monitoring system can be improved.

Further, the image corresponding to the coordinate position of the frame designated by the pointer in the entire image is stored in the memory 30.
Can be read from and played back. Therefore, the frame designated by the pointer in the entire image can be reproduced backward in time.

Further, by browsing the movable range image showing the range in which the camera unit 3 can photograph, the user can
It is possible to easily understand how much range the camera unit is installed in.

Although one embodiment of the present invention has been specifically described above, the present invention is not limited to the above-mentioned embodiment, and various modifications can be made based on the technical idea of the present invention. is there.

For example, the numerical values given in the above-mentioned one embodiment are merely examples, and different numerical values may be used if necessary.

Further, in the above-described embodiment, the monitoring system has been described as an example including the JPEG encoder / metadata adding section 27.
The G encoder / metadata adding unit 27 may be omitted. That is, it is not compressed (N ×
The panoramic entire image may be displayed on the display 2 using M) original images.

In the above-described embodiment, the image of the area designated by the pointer is updated (reproduced) in ascending order of shooting date and time. However, the image of the area designated by the pointer is The shooting date and time may be updated (played) in the order of oldness.

Further, in the above-described embodiment, the mode of the monitoring system may be switched to the multipoint view mode by the following operation.

First, the VIEW MODE button 106 is pressed by the pointer. Next, SELECT button 1
17 is pressed by the pointer to display the selection screen as described with reference to FIG. Then, in this selection screen, a frame is designated by a pointer to select a region in which a view is desired.

Next, the PLAY 122 is pressed by the pointer to display the accumulated data display screen as shown in FIG. Then, the range of reproduction date and time is designated by a pointer or the like. Finally, the OK button 168 is pressed. As a result, the mode of the monitoring system shifts to the multipoint view mode.

[0115]

As described above, according to the present invention, it is possible to update and display only the image of the specific area of the entire image. Therefore, the operability of image analysis in the monitoring system can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a configuration of a monitoring system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an example of a detailed configuration of a computer and a camera unit in the monitoring system according to the embodiment of the present invention.

FIG. 3 is a schematic diagram showing an example of a screen of a GUI configuration according to an embodiment of the present invention.

FIG. 4 is a schematic diagram showing an example of a select screen according to an embodiment of the present invention.

FIG. 5 is a schematic diagram showing an example of a stored data list display screen according to an embodiment of the present invention.

FIG. 6 is a schematic diagram showing an example of an entire screen creating method according to an embodiment of the present invention.

FIG. 7 is a schematic diagram showing a range that can be imaged in one frame in the monitoring system according to the embodiment of the present invention.

FIG. 8 is a schematic diagram for explaining a data management method according to an embodiment of the present invention.

FIG. 9 is a whole image display unit 1 according to an embodiment of the present invention.
12 is a flowchart for explaining the operation of capturing the frame of the entire image 02.

FIG. 10 is a flowchart illustrating an operation of reproducing accumulated image data according to the embodiment of the present invention.

FIG. 11 is a flowchart illustrating an operation of the monitoring system according to the exemplary embodiment of the present invention in a multipoint view mode.

[Explanation of symbols]

1 ... Computer, 2 ... Display, 3 ...
-Camera unit, 4 ... Pantilter section, 5 ... Camera section, 21 ... Controller bus, 24 ... Imaging section, 27 ... JPEG encoder / metadata addition section, 30 ... Main memory , 31 ... Graphic controller, 32 ... Image compression section, 33 ... Controller CPU, 101 ... Movable range image display section, 1
02 ... Whole image display section, 103 ... Selected image display section

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/907 H04N 5/907 B 5/91 5/91 J (72) Inventor Nezu Dai Minato-ku, Tokyo Takanawa 4-1018 Sony Marketing Co., Ltd. (72) Inventor Takeyoshi Kurotani 2-17-1 Higashi-Gotanda, Shinagawa-ku, Tokyo 2-17-1 Sony EMCS Co., Ltd. (72) Inventor Masaaki Kurebayashi East, Shinagawa-ku, Tokyo 2-17-1, Gotanda Sony EMCS Co., Ltd. F-term (reference) 5B057 BA02 CA12 CA16 CB12 CB16 CC01 CE08 CE09 CH08 5C022 AA01 AB62 AB65 AC27 AC42 AC69 5C052 GA01 GA03 GA08 GA09 GB06 GB10 GC05 GE08 5C053 FA05 FA09 FA09 FA09 FA06 FA09 FA09 KA24 LA01 LA11 LA14 5C054 AA02 CA04 CE15 CF06 DA08 EA01 EA03 EA05 EA07 ED07 EG01 EH07 FD02 FE18 FE19 GB11 GB16 HA19

Claims (20)

[Claims] 1. A photographing unit for photographing a video, an original image composed of a plurality of still images photographed by the imaging unit, or a plurality of compressed images composed of a compressed still image obtained by compressing the plurality of still images. A storage unit for storing, an image display unit for displaying an image, an instruction unit for designating an area on the image displayed on the image display unit, and a generation from the original image or the compressed image stored in the storage unit. The panoramic entire image is displayed on the image display unit, and only the image of the designated area designated by the pointing unit on the entire image is updated with an image having a different shooting time from the image other than the designated area. And a monitoring system. 2. The control unit updates an image of a designated area designated by the designating unit on the entire image with a plurality of images having different photographing times. Monitoring system. 3. The monitoring system according to claim 1, wherein an image other than the designated area has a newer shooting date and time than an image updated in the designated area. 4. The monitoring system according to claim 3, wherein the image other than the designated area is an image photographed by a photographing unit. 5. The monitoring system according to claim 1, wherein an image other than the designated area has a photographing date older than an image updated in the designated area. 6. A photographing step for photographing a video, an original image composed of a plurality of still images photographed in the photographing step, or a plurality of compressed images composed of a compressed still image obtained by compressing the plurality of still images. An accumulation step of accumulating in an accumulation section, a display step of displaying a panoramic entire image generated from the original image or the compressed image accumulated in the accumulation step, and an on-screen image displayed by the display step. In the above, the monitoring is characterized by comprising an instructing step of instructing a specific area, and an updating step of updating only an image of the instructing area instructed by the instructing step with an image having a different shooting time from an image other than the instructing area. Method. 7. The method according to claim 6, wherein the updating step updates the image of the designated area designated by the pointing step on the entire image with a plurality of images having different photographing times. Monitoring method. 8. The monitoring method according to claim 6, wherein an image other than the designated area has a newer shooting date and time than an image updated in the designated area. 9. The monitoring method according to claim 8, wherein the image other than the designated area is an image currently captured. 10. The monitoring method according to claim 6, wherein an image other than the designated area has a shooting date and time older than an image updated in the designated area. 11. A shooting step of shooting a video image, an original image composed of a plurality of still images taken by the shooting step, or a plurality of compressed images composed of a compressed still image obtained by compressing the plurality of still images. An accumulation step of accumulating in an accumulation section, a display step of displaying a panoramic entire image generated from the original image or the compressed image accumulated in the accumulation step, and an on-screen image displayed by the display step. In the computer, an instruction step of instructing a specific area, and an updating step of updating only the image of the instruction area instructed by the instruction step with an image other than the instruction area and an image capturing time different from each other A program characterized by. 12. The updating step updates the image of the designated area designated by the pointing step on the whole image with a plurality of images having different photographing times. program. 13. The program according to claim 11, wherein the image other than the designated area has a newer shooting date and time than the image updated in the designated area. 14. The program according to claim 13, wherein the image other than the designated area is an image currently captured. 15. The program according to claim 11, wherein an image other than the designated area has a shooting date and time older than an image updated in the designated area. 16. A photographing step for photographing a video, an original image composed of a plurality of still images photographed in the photographing step, or a plurality of compressed images composed of a compressed still image obtained by compressing the plurality of still images. An accumulation step of accumulating in the accumulation section, a display step of displaying a panoramic whole image generated from the original image or the compressed image accumulated in the accumulation step, and an entire image displayed in the display step A program for causing a computer to execute an instructing step of instructing a specific area in step 1, and an updating step of updating only an image of the instructing area instructed by the instructing step with an image other than the instructing area and an image having a different shooting time A recording medium on which is recorded. 17. The method according to claim 16, wherein the updating step updates the image of the designated area designated by the pointing step on the entire image with a plurality of images having different photographing times. recoding media. 18. The recording medium according to claim 16, wherein an image other than the designated area has a newer shooting date and time than an image updated in the designated area. 19. The recording medium according to claim 18, wherein the image other than the designated area is an image captured at the present time. 20. The recording medium according to claim 16, wherein an image other than the designated area has a shooting date and time older than an image updated in the designated area.