JPH05191699A - Multi-information camera - Google Patents

Abstract

PURPOSE:To obtain an index and information required for processing by automatically recording information, which is not electronically recorded in a conventional system, as attendant information together with pictures and sounds. CONSTITUTION:The absolute position of a camera is detected by a camera position detecting mechanism 47, and the downward distance of the camera is measured by the camera position detecting mechanism 47 also. The change of the camera position and the absolute direction and the change of the lens optical axis are detected with an angular velocity detecting mechanism (sensor) 48, etc., and ambient circumstances are detected by, for example, temperature and humidity measuring sensors 49 and 50. Brightness information is generated by a secondary processing part 38 in accordance with the stop value of a lens 30 and an output average value detected on an image pickup element 34. These various attendant information are constituted in a certain format by a format constituting part 39 and are synchronously recorded together with time information generated by a time stamp generating part 42 by an attendant information recording mechanism 41.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera for recording video and audio, which is used for searching, editing, analyzing, registering in a database, etc., so that a computer can process the video / audio data after recording. It is related to technology for collecting necessary data.

[0002]

2. Description of the Related Art Conventionally, in a camera for recording video / audio, a time code is recorded on a special track on a recording tape as identification information corresponding to a frame according to time,
It has been used as a standard for synchronizing video and audio during editing work. The time code is added at the time of shooting or added to the tape after shooting by a time code generator.

FIG. 5 shows a structure in which, when video and audio are respectively recorded on different media, time codes are recorded on the respective media at the same time and the time codes are associated with each other at the time of editing. Section, 11 is a video recording section, 12 is a time code generator, 13 is an audio recording section, 1
4 is a microphone. Although the date and time are also superimposed in the video, other information necessary for processing such as editing is not digitized by people, mainly on paper. It is recorded in the form of. On the other hand, at the beginning of the editing process, a person may input information for editing and digitize it, but in that case as well, information about a cut point, which is a break in the image, is mainly added.

[0004]

When a video is processed by a computer, for example, when retrieval, editing, analysis, processing, database registration, etc. are performed, in addition to the video data itself, various data relating to the shooting of the video are provided. Additional information is required. For example, when searching or analyzing video,
Information on the shooting conditions of the image is required, but in the above-mentioned conventional technology, the time code used as the reference of time is digitized at the time of editing, and other various information is not digitized. There is a problem that a person must intervene to digitize necessary information from the record written in, or the editor must process it according to his / her memory and feeling. There are also cases where the date and time are superimposed on the image, but these are not enough as information, and there is the problem that special recognition processing is required for automatic processing. It was

Further, the cut point of the image generated by the start / stop / pause of the photographing is the basis for editing and the like. However, since electronic recording of the photographing operation has not been made conventionally, for example, There was a problem that the cut point would not be known unless people saw it after recording.

Further, conventionally, for a camera, images are sequentially recorded on a recording / reproducing medium such as a tape. However, no matter how much information is added to the recording / reproducing medium, the information at an arbitrary time can be recorded at high speed. There was a problem that it could not be read.

The present invention has been made to solve the above problems, and an object of the present invention is to provide information as an index when an image is automatically processed by a computer after recording.
It is to provide a technology capable of obtaining information necessary for more processing.

Another object of the present invention is to provide a technique capable of high-speed processing such as retrieval by processing only incidental information without necessarily processing all the video during computer processing after recording.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[0010]

In order to achieve the above object, the means (1) of the present invention is a camera for recording video and audio, which generates absolute time at the time of photographing and converts it into data. Means, means for detecting the position of the camera and converting it into data, means for detecting the direction of the camera and converting it into data, means for generating imaging related information from the imaging control system and converting it into data, and camera movement data. Means for detecting and converting to data,
It is characterized by comprising means for detecting ambient environment information and converting it into data, and means for automatically recording such data as incidental information in correspondence with the photographed video every moment.

The means (2) of the present invention is, in the multi-information camera of the above-mentioned means (1), means for performing a predetermined pre-processing on the data at the time of data recording, and evaluation of the processing result, and setting in advance. When a certain condition is met, the marking information indicating that condition is automatically recorded as new supplementary information in association with the captured video every moment.

According to a third aspect of the present invention, in the camera according to the second aspect, the frame image at the time when the condition is met is automatically recorded as new supplementary information in association with the momentarily photographed image. It is characterized by including.

According to a fourth aspect of the present invention, in the camera according to any one of the above-mentioned first to third aspects, the data described above can be stored in a random access recording medium independently of a video recording system. It is characterized in that it is provided with a means for recording.

[0014]

According to the above means (1) to (4), the sensor detects the absolute time at the time of photographing, the image-related information, the camera position information, the camera direction information, the camera motion information, and the ambient environment information or the internal processing. Generated by the above, and it is converted into data, and these are automatically recorded as incidental information in association with the captured video every moment, so that the captured video can be automatically processed by the computer.

By recording the supplementary information on the random access recording medium separately from the image itself, even if the long-time image is recorded on the sequential recording / reproducing medium such as a video tape, the supplementary information is recorded. Random access is possible.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing a photographer, a camera, a subject, a surrounding environment, and the like concerning photographing in one embodiment of the present invention. 20 is the sun, 21 is a camera, 22 is a photographer, and 23 is a photographer. A subject, 24 is a camera operation, and 25 is a surrounding environment. The camera 21 for photographing is held by the photographer 22 with respect to the subject 23 at a position at a certain time. First, with respect to the imaging system, the focus is automatically or manually adjusted to the main subject 23, and the aperture of the lens, the shutter speed, etc. are also automatically or manually set.

The camera operation 24 is performed as a lens operation such as zoom-in / out. Also, as the camera operation 24, recording on / off by start / stop / pause, panning to shake the camera left and right, tilting to shake up and down, tracking to move the camera position in parallel, and doling to move back and forth in parallel are performed. Be seen. On the other hand, with respect to the surrounding environment 25, surrounding brightness, temperature, humidity and the like are shooting conditions.

FIG. 2 is a block diagram showing the construction of an embodiment of the information recording mechanism of the multi-information camera according to the present invention, in which 30 is a lens, 31 is a diaphragm, 32 is a zoom mechanism, and 3 is a zoom mechanism.
3 is an electronic shutter, 34 is an image sensor, 35 is an electronic drive system, 36 is a data conversion unit, 37 is a video / audio processing unit, 3
8 is a secondary processing system, 39 is a format composing unit (conversion unit), 40 is a video / audio recording mechanism, 41 is an auxiliary information recording mechanism, 42 is a time stamp generating unit, 43 is a recording start / pause / stop switch, 44 Is a microphone, 45 is a distance sensor, 46 is a camera height sensor, 47 is a camera position detection mechanism, 48 is an angular velocity detection mechanism, 49 is a temperature sensor,
50 is a humidity sensor, 51 is another ambient environment sensor, 5
2 is an external input interface.

FIG. 3 is a diagram for explaining the relationship of information in the multi-information camera of this embodiment, 30A is original information, 30B is additional information to be recorded, 301 is imaging system information, and 302 is camera position information. 303 is camera movement information, 304 is subject-related information, 305 is ambient environment information, 306 is external input information, 307 is various event marks, and 308 is absolute time information.

FIG. 4 is a diagram for explaining the state of the recorded video / audio information and the supplementary information and the use thereof in this embodiment. 40A and 40B are recording media, 401 is an image sequence, and 401A is a cut. Points, 402 is information on the cut point 401A, 403 is a frame image at the cut point 401A, 4
Reference numeral 04 is a frame video sequence.

Next, the operation of this embodiment will be described with reference to FIGS. First, when the photographer 22 holds the camera 21, the absolute position and orientation of the camera 21 are determined. Camera 21
The absolute position of is detected by the camera position detecting mechanism 47, for example, by a positioning method using a radio wave from a satellite or a positioning method using a reference of a ground reference wave. In terms of accuracy, macro position detection is performed.
Further, the downward distance of the camera 21 is set as the height of the camera 21, and the height is measured by the camera position detection mechanism 47 by the ultrasonic reflection measurement method or the like. The change in the position of the camera 21, the absolute direction of the optical axis of the lens, and the change can be detected by using a gyro with high accuracy by an angular velocity detection mechanism (sensor) 48 or the like.

In the image pickup system, the result obtained by the automatic or manual determination mechanism of the diaphragm and the focus, which has been conventionally used in the video camera, is used as the diaphragm 31 and the zoom mechanism 32.
It is taken out from the encoder output of the electronic drive system 35 of. In addition, electronic shutter 3 for shutter speed
The set value of 3 is taken out.

Regarding the ambient environment, for example, temperature and humidity are detected by sensors 49 and 50 for measurement, respectively.

The secondary processing unit 38 generates brightness information from the aperture value of the lens and the average output value detected on the image pickup device 34.

These various kinds of supplementary information are formed into a fixed format by the format constructing unit 39 and are recorded by the supplementary information recording mechanism 41 in synchronization with the time information generated by the time stamp generating unit 42.
The video and audio are recorded by the video / audio recording mechanism 40 together with the time stamp by the recording mechanism of an ordinary video camera. Regarding the start / stop / pause of shooting, not only the timing of recording mechanism control to the recording mechanisms 40 and 41 is given, but also mark information indicating a cut point is recorded as a trigger of the cut point at the start timing. The time stamp at this cut point defines the video (shot) between the cut points in absolute time. When the frame video recorded at this timing is compressed and recorded as a part of the supplementary information, it can be used as a graphical index as a representative video of a shot starting at the cut point.

In addition, real-time processing is performed at the time of recording by using the information detected by various sensors and the generated information.
Information for recording as supplementary information is generated by the secondary processing unit 38. As a method of generating secondary information, when the processing result is always recorded at the time of recording, or when the processing itself is filtering and the value of the result satisfies a certain condition,
Mark information indicating that may be recorded. The processing / evaluation algorithm in the secondary processing unit 38 is set in advance, but by making them programmable, general-purpose processing becomes possible. For example, when the amount of change in the camera position or the optical axis of the camera exceeds a certain value, mark information indicating that an event requiring a rapid camera operation has occurred is recorded. For example, using the camera optical axis displacement obtained from the angular velocity detection value, mark a range exceeding a certain threshold as panning or tilting, or add zoom-in / out mark information based on changes in focal length. You can

Further, information relating to photographing from the outside by data such as voice, image and text, for example, voice narration of the photographer, texts and drawings to be explained, are inputted from the external input interface 52 and used as supplementary information. Add.

On the other hand, when the photographed video is recorded on a sequential recording medium / playback medium such as a video tape, the above-mentioned supplementary information is a randomly accessible recording medium different from the video,
For example, if the image is recorded on a floppy disk, it is possible to perform a search or an image analysis primary process based on the information recorded as the additional information without processing the entire image itself after the recording. For example, as shown in FIG. 4, by using the information 402 of the cut points 401A recorded as mark information, it is possible to immediately list the time information of each cut and the frame at the cut points 401A. Immediately list video 403 (frame video sequence 404)
By displaying, the contents list of the photographed video can be visually displayed, and based on this, rough summary video editing as a primary process can be realized in a short time without using actual video information.

Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Absent.

[0031]

As described above, according to the multi-information camera of the present invention, at the time of photographing, information which has not been electronically recorded in the past is automatically recorded as supplementary information simultaneously with video and audio. Since it is possible to do so, the information as an index when the video is automatically processed by the computer after recording,
The information necessary for more processing can be obtained. That is, the supplementary information after recording gives information necessary for image recognition as well as temporary processing such as retrieval and editing as described above when the computer processes the image during image processing.
For example, the imaging system condition information, the camera position and the camera optical axis information give important information for estimating the three-dimensional model of the object from the image.

In addition to the information temporarily detected / generated by the camera, the secondary information is added as supplementary information by performing real-time processing at the time of photographing based on the information, and the supplementary information can be randomly accessed. By recording on the recording medium, it is possible to perform a process such as a search at high speed by processing only the incidental information without necessarily processing all the video during computer processing after recording.

[Brief description of drawings]

FIG. 1 is a diagram showing a situation of a photographer, a camera, a subject, a surrounding environment, and the like regarding photography in one embodiment of the present invention,

FIG. 2 is a block diagram showing a configuration of an embodiment of an information recording mechanism of a multi-information camera according to the present invention,

FIG. 3 is a diagram for explaining a relation of information in the multi-information camera of the present embodiment,

FIG. 4 is a diagram for explaining a state and use of recorded video / audio information and incidental information in the present embodiment;

FIG. 5 is a diagram showing a configuration of a conventional video / audio recording system using a time code.

[Explanation of symbols]

10 ... Imaging unit, 11 ... Video recording unit, 12 ... Time code generator, 13 ... Audio recording unit, 14 ... Microphone, 20 ... Sun, 21 ... Camera, 22 ... Photographer, 23 ... Subject, 24
… Camera operation, 25… Surrounding environment, 30… Lens, 31…
Aperture, 32 ... Zoom mechanism, 33 ... Electronic shutter, 34
... image pickup device, 35 ... electronic drive system, 36 ... data conversion unit,
37 ... Video / audio processing unit, 38 ... Secondary processing system, 39 ... Format conversion unit, 40 ... Video / audio recording mechanism, 41 ...
Additional information recording mechanism, 42 ... Time stamp generating section, 43
... Recording start / pause / stop switch, 44 ... Microphone, 45 ... Distance sensor, 46 ... Camera height sensor, 47
... Camera position detection mechanism, 48 ... Angular velocity detection mechanism, 49 ...
Temperature sensor, 50 ... Humidity sensor, 51 ... Other ambient environment sensor, 52 ... External input interface.

Claims (4)

[Claims] 1. A camera for recording video and audio, comprising:
Means for generating absolute data at the time of shooting and converting it into data, means for detecting the position of the camera and converting it into data, means for detecting the direction of the camera and converting it into data, and generating imaging related information from the imaging control system To convert the data into data, to detect the camera movement data to convert it into data, to detect the surrounding environment information to convert it into data, and to use the data as supplementary information to correspond to the captured image every moment. A multi-information camera comprising means for automatically recording. 2. The multi-information camera according to claim 1, wherein a means for performing a predetermined pre-processing on the data at the time of recording the data and a result of the processing are evaluated, and when the preset conditions are met. A multi-information camera comprising means for automatically recording marking information indicating this as new supplementary information in association with captured images at every moment. 3. The camera according to claim 2, further comprising means for automatically recording the frame image when the condition is met as new supplementary information in association with the captured image every moment. Information camera. 4. The camera according to claim 1, further comprising means for recording the data described above on a random access recording medium independently of a video recording system. Multi information camera.