JP4462647B2 - Image information recording apparatus and image information processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image information recording apparatus and an image information processing method capable of effectively securing a recording area of a recording medium and easily searching for recorded image information and various image processing.
[0002]
[Prior art]
Conventionally, recording of a photographed image by a video camera or the like has been mainly recorded in time series using a tape-shaped recording medium. In addition, information other than images, such as date and time, titles, and the like, is also converted into image information and added to the captured image and recorded.
[0003]
However, in recording using a tape-shaped recording medium, it is necessary to feed and rewind the tape while reproducing and checking the image in order to reproduce the target image, which is cumbersome and time consuming. there were. On the other hand, while having such problems, it is widely used in various fields due to the huge recording capacity of the tape-shaped recording medium. For example, in monitoring applications, it is necessary to continue recording, and it is essential that the capacity of the recording medium is large.
[0004]
Searching for the required image from this enormous recording capacity requires complicated operations and time to reproduce the location even if the date, title, etc. are recorded, as described above. In addition, retrieving these images via a communication line and transmitting the desired image information requires a great deal of time and cost.
[0005]
On the other hand, a disk-shaped recording medium is extremely effective in the speed of access, but has a small storage capacity. Therefore, in order to obtain a large recording capacity, it is necessary to configure the apparatus using a plurality of disks. There was a problem that it became large and expensive.
[0006]
[Problems to be solved by the invention]
Accordingly, an object of the present invention is to provide an image information recording apparatus that can record for a long time, can easily search for recorded images, has good operability, and is suitable for transmission via a communication medium. To do.
[0007]
[Means for Solving the Problems]
The present invention has been proposed in view of the above-described problems, and is an image in which a subject is photographed via a communication line and a remote operation of an imaging means for converting the image into a video signal is performed, and the photographed image is recorded on a recording medium. in the information recording apparatus, an image information recording means for recording the received recording medium via a communication line features extracted from images taken with captured images, actually from the captured image and previously captured images recorded an image selection means for selecting an image based on the extracted features, wherein the image erasing means for erasing an image recorded on a recording medium, the recording medium on the recorded image again compressing and recording image comprising a recompression means, said extracted features, the movement of the object includes any one of the color or brightness, and records the extracted features on the recording medium together with the image, wherein Erasing the image selected by the image selecting means, or, characterized in that compressed again recorded.
[0009]
In addition, the present invention further includes various detection means for detecting a shooting environment at the time of shooting, and the extracted features include an elapsed time since recording, a high space detected by the detection means. It further includes any one of frequency component energy, saturation dispersion, moving object detection result, subject recognition result, pyroelectric sensor change, sunshine sensor value, and detection result based on microphone volume. To do.
[0010]
Furthermore, the present invention is characterized in that, in the above-described image information recording apparatus, an image recorded on the recording medium together with the extracted features is searched and image processing is performed.
[0014]
The image information recording apparatus according to the present invention compresses and deletes an image based on the viewpoint of the image that the user pays attention to, so that search is easy and work efficiency is improved, and an image recording area is secured. Is possible.
[0015]
Furthermore, the image information recording apparatus according to the present invention extracts features from an image captured based on a viewpoint that the user pays attention to, for example, a specific movement, a specific subject, and the like, and records it together with the image so that the search can be performed. Easy and efficient work.
[0016]
Furthermore, the image information recording apparatus according to the present invention records various environmental conditions at the time of shooting, such as date and time, weather, detection results of various sensors, and the like together with the image, so that search is easy and work efficiency is improved. improves.
[0017]
Furthermore, the image information recording apparatus according to the present invention compresses and erases an image based on the viewpoint that the user pays attention to, and further, based on the viewpoint that the user pays attention, such as a specific movement, a specific subject, and the like. Since the features are extracted from the captured image and recorded together with the image, the search is easy, the work efficiency is improved, and the recording area of the image can be secured.
[0018]
Furthermore, the image information recording apparatus according to the present invention compresses, erases, and further shoots an image based on the viewpoint that the user pays attention to, such as date and time, weather, detection results of various sensors, and the like. Is recorded together with the image, search is easy, work efficiency is improved, and an image recording area can be secured.
[0019]
Furthermore, the image information recording apparatus according to the present invention is extracted by the feature extraction means, by the retrieved image processing the image recorded on the basis of the reported characteristics, a variety of images recorded Processing such as search and editing can be performed efficiently. In addition, by searching for and processing the recorded image based on the detection result of the photographing environment detected and recorded by the detection unit, various recorded images can be obtained based on the detection result of the detection unit. Processing such as search and editing can be performed efficiently.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
The present invention provides an image information recording apparatus and an image that can efficiently perform various processing of an image based on a viewpoint noted by a user and that can effectively use a limited recording capacity to record a large number of required images. An information processing method is provided, and further, the image data and control information communication time when the image information recording apparatus is remotely operated via a communication line is shortened to realize immediacy of control. Is.
[0021]
As the configuration of the image information recording apparatus, an imaging means for photographing a subject and converting it into an electrical signal, for example, a video camera or an electronic still camera using a CCD, and an image information recording means for recording the photographed image on a recording medium, for example, VTR, image selection means for selecting a specific image from the actually captured image or recorded image, for example, a CPU having a predetermined recognition program, an image deletion means for deleting the selected image, and the selection Image compression means for compressing the recorded image, for example, a CPU having a JPEG compression algorithm, etc., and selecting and erasing the image in accordance with a predetermined standard, or compressing and recording again An image recording area such as a medium such as a magnetic tape is secured.
[0022]
Also, means for extracting features desired by the user, such as movement, color, brightness, etc. of the user from the recorded image, for example, a CPU having a program for extracting predetermined features, and detection for detecting the environment at the time of shooting An apparatus, for example, a temperature sensor, a door sensor, a microphone, and the like are included, and the extracted image characteristics and the detection result of the detection apparatus are recorded together with the image on a recording medium. In addition, information on date, time, or angle of view and direction of imaging may be recorded.
[0023]
Images are recorded with the above-described configuration, and when the recording area runs short, priorities are assigned to recorded images, and the recorded images are erased in order from the least important ones, or compressed and recorded again. This priority order is weighted according to the type of image features and each detection device from the viewpoint intended by the user, and an image to be erased or compressed is determined based on their comprehensive statistics. .
[0024]
For example, when focusing on the movement of the target object, if the movement of the object is recorded in 10 out of 100 frames, the remaining 90 frames are unnecessary and are erased, and the area is used for recording a new image. Used. Also, for example, leave only an image at a predetermined temperature range, or leave only an image for a predetermined time, or set various conditions, or set multiple conditions at the same time, weight between them, and count The image is erased, compressed, recorded, and the like.
[0025]
When the recording area of the recording medium is further reduced, the level of the above-described condition is made stricter, and further erasure or compression is performed in the order of necessity to secure an area for recording a new important image.
[0026]
Moreover, it is also possible to use for the desired search and classification process based on the feature extracted from the image and the output of each detection device recorded simultaneously with the image.
Note that the above-described image information recording apparatus requires operation input means for designating an image to be reproduced, decompression means for decompressing a compressed image, image display means, and the like in order to reproduce the recorded image. Needless to say.
[0027]
Next, an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a system configuration of an image information recording apparatus according to the present invention, FIG. 2 is a diagram showing a flow of processing of photographed image information, and FIG. 3 is a network of the image information recording apparatus according to the present invention. It is a figure which shows the structure of the used remote control system.
[0028]
First, the image information recording apparatus of the present invention is configured as shown in FIG. The image pickup apparatus 10 is, for example, a video camera, and forms an image on the CCD 12 that is a solid-state image pickup device through the lens and diaphragm of the lens block 11 to form image pickup light arriving from a subject. Perform automatic gain adjustment. For signal separation and automatic gain adjustment, the SH / AGC 13 samples and holds the video signal, and gain control is performed so as to obtain a predetermined gain by an auto iris control signal.
[0029]
The video signal thus obtained is converted into a digital signal by an A / D (analog / digital converter) 14 and supplied to a signal processor 15. The signal processing device 15 converts the supplied signal into a signal such as a luminance, a color difference, and a video signal, and supplies the signal to a video capture device 21 built in the monitor 18 or the computer 20. The output of the signal processing device 15 is input to the image processing device 19 and performs image processing such as feature extraction in accordance with instructions.
[0030]
The lens block 11 of the image pickup apparatus 10 is a zoom lens that can change the angle of view to be picked up by driving the zoom lens 16. For example, a stepping motor or the like is used according to a drive command of the camera controller 2. The zoom motor 17 is rotated to drive the zoom lens 16. The camera controller 2 is a controller that constantly performs lens control such as focus and zoom of the image pickup device 10, exposure control such as aperture, gain, and electronic shutter speed, white balance control, image quality control, and the like, and the mode controller 1 and the image pickup device. The interface with 10 is taken.
[0031]
As an interface control of the camera controller 2 related to the control of the zooming lens 16, the zooming lens 16 is driven at the requested position in response to the driving request of the zooming lens 16 sent from the mode controller 1. Thus, the control signal is output to the motor driver to control the position of the zoom motor 17 and the current position of the zoom motor 17 is always transmitted to the mode controller 1.
[0032]
Further, the imaging device 10 is installed on a device having a degree of freedom in two-axis rotation directions such as pan and tilt, for example, on the pan tilter 4, and a pan motor 5 using a stepping motor or the like according to a driving command from the pan tilter controller 3. And the tilt motor 6 are rotated, and the pan tilter 4 is driven in two axial directions. The pantilter controller 3 sends a control signal to the driver of the motor so that the pantilter 4 is driven to the requested position in response to the driving request in the pan and tilt directions of the pantilter 4 sent from the mode controller 1. And the current position of the pan tilter 4 is always transmitted to the mode controller 1.
[0033]
The mode controller 1 controls the system in accordance with the setting state of the pantilter 4 and the imaging device 10 and other information input from the outside. The mode controller 1 is connected to the computer 20, and in response to an absolute position drive request from the computer 20, the camera controller 2 and the pantilter controller 3 are driven to drive the pantilter 4 and the zoom lens 16 of the lens block 11. On the other hand, the current position information sent from the camera controller 2 and the pantilter controller 3 is transmitted to the computer 20.
[0034]
In the present invention, the function of the computer 20 is used as the control device 23, and the operation input uses a graphical user interface of a graphic display of the monitor 24 connected to the computer 20. Each control data to be transmitted to the mode controller 1 is determined based on the operation command input in the operation area 24c. For the operation input, an external input device such as a pointing device 25 may be used.
[0035]
Further, a video capture device 21 is built in the computer 20 for displaying and storing the video obtained by the imaging device 10. The video capture device 21 can display the video signal input from the imaging device 10 on the image display area 24a of the monitor 24 with an arbitrary image quality, and can also display a predetermined image format (for example, bit) by the capture signal. Map image, JPEG format still image, JPEG format moving image, etc.) with arbitrary image quality and stored in the storage device 22 constituted by a hard disk or the like in the computer 20. The storage device 22 uses a magnetic tape or a magnetic disk as a recording medium. When a magnetic tape is used, random access is good. For example, the position and index of the image on the magnetic tape are recorded in a semiconductor memory or the like, thereby facilitating access to the magnetic tape. The index is a feature extracted from an image, outputs from various sensors, and the like, which will be described in detail later.
[0036]
Further, various sensors such as a sunshine sensor 26, a pyroelectric sensor 27 for detecting the approach of a person, a microphone 28, and the like are provided in the vicinity of the image pickup apparatus 10, and outputs thereof are connected to the computer 20 so that environmental conditions at the time of shooting are taken. And the information is recorded together with the image.
[0037]
The screen of the monitor 24 is divided into, for example, an image display area 24a, an index display area 24b, and an operation area 24c. The image display area 24a has a captured image, an image read from the storage device 22, or an index display. A reduced list of images and the like retrieved by the index displayed in the area 24b and the retrieval instruction input from the operation area 24c is displayed.
[0038]
Next, the flow of data processing such as image information and sensor output according to the present embodiment will be described with reference to FIG.
[0039]
The image captured by the camera (S101) is first subjected to moving object detection, subject recognition, etc. by the image processing device 19, and is used to evaluate the priority of the computer 20 together with information such as shooting conditions, such as angle of view, focus, exposure, and white balance. It is sent (S105). Here, the priority of the image to be deleted or compressed is evaluated based on the weighted condition described above. Information from the sensor (S102) is also sent to the priority evaluation, and is used for the evaluation of the priority of the image based on the evaluation criteria set by the operator together with the result of spatial frequency component analysis (S104) described later. .
[0040]
The video captured by the camera is sent to video capture (S103), and is used for the spatial frequency component analysis (S104) and image compression (S110) described above. For example, the spatial frequency component analysis is calculated for one of four frames. In this case, other feature analysis means such as saturation may also be introduced.
[0041]
Also, in order to determine the priority order of image recording, the importance of each image is scored from a plurality of viewpoints, for example, 1 byte is scored every 4 frames and stored in the memory. Examples of viewpoints include elapsed time since recording, high spatial frequency component energy, saturation dispersion, moving object detection results, subject recognition results, pyroelectric sensor changes, sunshine sensor values, microphone volume, The degree of compression of the image so far. Furthermore, viewpoints that the operator pays attention may be added, and each viewpoint may be weighted.
[0042]
The score of each viewpoint mentioned above is totaled, a priority table is created (S106), a frequency distribution is obtained from this priority table (S107), and a threshold value is set so that an area necessary for further image recording can be secured (S107). S108), an image to be erased and recompressed is determined (S109). Based on this determination, the corresponding image position of the recording apparatus is instructed (S111), and deletion or recompression is performed (S112). When an image to be displayed is selected by the pointing device during reproduction (S113), the corresponding position is read from the recording position (S114), the compressed image data is expanded (S115), and the result is displayed on the monitor (S115). S116).
[0043]
As a compression algorithm, for example, there is a progressive method by spectrum selection, which is one of the extended methods of JPEG. This method has the feature that when the image is recompressed with a reduced resolution, it is only necessary to delete the coefficient of the high frequency component, and there is no need to repeat the compression / decompression process. Since it can be expanded by the same algorithm, it is suitable for use in the image information recording apparatus of the present invention.
[0044]
FIG. 3 shows a system configuration example of an image information recording apparatus that performs remote operation via a communication line. In this system, a computer 20 is provided as a server computer, and a computer 30 is provided as a client computer, and these are connected via the Internet 40. The computers 20 and 30 are provided with network interfaces 29 and 39, respectively, for connection between the computers.
[0045]
In addition to the network interface 39, the computer 30 includes a storage device 32 for storing transmitted image information, a control device 33 for remote control, and a monitor 34, which is connected to display images and provide instructions. An area for a graphical user interface for input is provided. The instruction input may be input from the pointing device 35 or the like.
[0046]
The other components and the operation of the system configuration are the same as those of the above-described embodiment, and the description thereof is omitted here.
[0047]
As described above in detail, the present invention uses a video camera or an electronic still camera, and is particularly suitable for uses such as time shift of a camera image by remote operation recording and reproduction, construction and search of an image database, monitoring, and introduction. .
Of course, the present invention is not limited to a magnetic tape recording medium, but may be applied to a magnetic disk recording medium and further to an optical recording / reproducing medium.
[0048]
【The invention's effect】
As is apparent from the above description, according to the invention of claim 1, the image is compressed or deleted based on the viewpoint of the image that the user pays attention to, so that the search is easy and the work efficiency is improved. It is possible to secure an image recording area.
[0049]
According to the second aspect of the present invention, since a feature is extracted from an image photographed based on a viewpoint focused by the user, for example, a specific movement, a specific subject, etc., and recorded together with the image, the search is easy. This improves work efficiency.
[0050]
According to the invention of claim 3, since various environmental conditions at the time of shooting, such as date and time, weather, detection results of various sensors, etc., are recorded together with the images, search is easy and work efficiency is improved. .
[0051]
According to the invention of claim 4, according to the image information recording apparatus, the image is compressed and erased based on the viewpoint that the user pays attention to, and the viewpoint that the user pays attention to, for example, the specific movement, Since features are extracted from an image shot based on a specific subject and recorded together with the image, search is easy, work efficiency is improved, and an image recording area can be secured.
[0052]
In addition, according to the invention of claim 5, various environmental states such as date and time, weather, detection results of various sensors, etc. when the image is compressed, erased, and further photographed based on the viewpoint that the user pays attention to, Since the image is recorded together with the image, searching is easy and work efficiency is improved, and an image recording area can be secured.
[0053]
Furthermore, according to the sixth aspect of the present invention, it is possible to efficiently perform various processes such as search and editing of recorded images based on the extracted features. According to the seventh aspect of the present invention, it is possible to efficiently perform various processes such as search and editing of recorded images based on the detection result by the detection means.
[Brief description of the drawings]
FIG. 1 is a system configuration of an image information recording apparatus according to the present invention.
FIG. 2 is a diagram showing a flow of processing of photographed image information.
FIG. 3 is a diagram showing a configuration of a remote operation system using a network of the image information recording apparatus according to the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Mode controller, 2 ... Camera controller, 3 ... Pantilter controller, 4 ... Pantilter, 5 ... Pan motor, 6 ... Tilt motor, 10 ... Imaging device, 11 ... Lens block, 12 ... CCD, 13 ... SH / AGC, 14 ... A / D, 15 ... signal processing device, 16 ... magnification lens, 17 ... zoom motor, 18, 24,34 ... monitor, 19 ... image processing device, 20,30 ... computer, 21 ... video capture device, 22 , 32 ... storage device, 23, 33 ... control device, 24a, 34a ... image display area, 24b, 34b ... index display area, 24c, 34c ... operation area, 25, 35 ... pointing device, 26 ... sunshine sensor, 27 ... Pyroelectric sensor, 28 ... Microphone, 29, 39 ... Network interface, 40 ... Internet

Claims (3)

In an image information recording apparatus for performing remote operation of an imaging means for photographing a subject and converting it to a video signal via a communication line, and recording the photographed image on a recording medium,
An image information recording means for recording the features extracted from the photographed photographing image image received recording medium via the communication line,
An image selection means for selecting an image based on the extracted features from the actually captured image and the already captured and recorded image;
Image erasing means for erasing the image recorded on the recording medium ;
Image re- compression means for compressing and re-recording the image recorded on the recording medium ,
The extracted features, the movement of the object includes any one of the color or brightness, and records the extracted features on the recording medium together with the image,
An image information recording apparatus, wherein the image selected by the image selection means is erased or compressed and recorded again.   Furthermore, it has various detection means for detecting the shooting environment when shooting,
  The extracted features include the time elapsed since recording, the energy of high spatial frequency components, the dispersion of saturation, the result of moving object detection, the result of subject recognition, the result of subject recognition, The image information recording apparatus according to claim 1, further comprising any one of a detection result based on a change, a value of a sunshine sensor, and a volume of a microphone. 3. The image information recording apparatus according to claim 1, wherein an image recorded on the recording medium together with the extracted features is searched and image processing is performed.

Images