JP3362362B2 - Multi information camera - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera for recording video and audio, which is used for searching, editing, analyzing, registering in a database, and the like so that a computer can process the video / audio data after recording. It is related to technology for collecting necessary data.

[0002]

2. Description of the Related Art Conventionally, in a camera for recording video / audio, a time code is recorded on a special track on a recording tape as identification information corresponding to a frame according to time,
It has been used as a standard for synchronizing video and audio during editing work. The time code is added at the time of shooting or added to the tape after shooting by a time code generator.

FIG. 5 shows a configuration in which, when video and audio are respectively recorded on different media, time codes are recorded on the respective media at the same time and the time codes are associated with each other at the time of editing. Section, 11 is a video recording section, 12 is a time code generator, 13 is an audio recording section, 1
4 is a microphone. Although the date and time are also superimposed in the video, other information necessary for processing such as editing is not digitized by people, mainly on paper. It is recorded in the form of. On the other hand, at the beginning of the editing process, a person may input information for editing and digitize it, but even in that case, the information about the cut point, which is a break in the image, is mainly added.

[0004]

When a video is processed by a computer, for example, when retrieval, editing, analysis, processing, database registration, etc. are performed, in addition to the video data itself, various data relating to the shooting of the video are provided. Additional information is required. For example, when searching or analyzing video,
Information about the shooting conditions of the video is required, but in the above-mentioned conventional technology, the time code used as the reference of time is digitized at the time of editing, and other various information is not digitized. There is a problem that a person must intervene to digitize necessary information from the record written in, or the editor must process it according to his / her memory and feeling. There are also cases where the date and time are superimposed on the image, but not only that they are not enough information, but there is also the problem that special recognition processing is required for automatic processing. It was

Further, the cut point of the image generated by the start / stop / pause of photographing is the basis for editing and the like. However, since electronic recording of the photographing operation has not been made conventionally, for example, There was a problem that the cut point would not be known unless people saw it after recording.

Further, conventionally, for a camera, images are sequentially recorded on a recording / reproducing medium such as a tape. However, no matter how much information is added to the recording / reproducing medium, the information at an arbitrary time can be recorded at high speed. There was a problem that it could not be read.

The present invention has been made to solve the above problems, and an object of the present invention is to provide information as an index when an image is automatically processed by a computer after recording.
It is to provide a technique capable of obtaining information necessary for more processing.

Another object of the present invention is to provide a technique capable of high-speed processing such as retrieval by processing only the incidental information without necessarily processing all the video during computer processing after recording.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[0010]

DISCLOSURE OF THE INVENTION Disclosed herein
Of the inventions, a brief description of typical ones will be made as follows.
It is as follows. That is, the present invention relates to a video / audio recording means for recording video and audio , a time data generating means for generating an absolute time at the time of photographing and converting it into data, and a camera position data for detecting the position of the camera and converting it into data. means a, and camera orientation data of the hand <br/> stage for data of detecting the orientation of the camera, and the imaging-related information data means that the data of generated from the imaging control system for imaging-related information, the camera Camera motion data conversion means for detecting and converting motion data into data, and ambient environment information data for detecting and converting ambient environment information into data
From the time data generating means, the data converting means and the respective data converted by the respective data converting means are incidental information.
And an additional information recording means for recording in synchronization with the time data of
A multi-information camera equipped with each of the data
Data converted by at least one data conversion means
A processing means for performing a predetermined pre-processing on the data
The additional information recording means has the processing by the processing means.
When the result meets the preset conditions, the mark
The time information as the supplementary information.
It is characterized in that it is recorded in synchronization with the time data .

In a preferred embodiment of the present invention , the above
The band information recording means is preset with the processing result of the processing means.
The frame video when the specified conditions are met is attached.
As a report, the time data from the time data generating means is
It is characterized by recording in synchronization .

In a preferred embodiment of the present invention , the above
The supplementary information recording means records the supplementary information in the form of video and audio.
Another random access recording independent of the recording medium to be recorded
It is characterized by recording on a medium .

[0013]

[0014]

According to the above-mentioned means, the absolute time at the time of photographing, the image-related information, the camera position information, the camera direction information, the camera motion information, and the ambient environment information are detected by the sensor or generated by the internal processing, and the data are generated. The captured images can be automatically processed by a computer because they are automatically recorded as incidental information in association with the captured images.

Further, by recording the additional information on the random access recording medium separately from the image itself, even if the long-time image is recorded on the sequential recording / reproducing medium such as a video tape, the additional information of the additional information is recorded. Random access is possible.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing the conditions of a photographer, a camera, a subject, an ambient environment, etc., relating to photographing in one embodiment of the present invention. 20 is the sun, 21 is a camera, 22 is a photographer, and 23 is a photographer. A subject, 24 is a camera operation, and 25 is a surrounding environment. The camera 21 for photographing is held by the photographer 22 at a position at a certain time with respect to the subject 23. First, regarding the image pickup system, the focus is automatically or manually adjusted to the main subject 23, and the aperture of the lens, the shutter speed, etc. are also automatically or manually set.

In the camera operation 24, zoom-in / out operations are performed as lens operations. Also, as the camera operation 24, recording on / off by start / stop / pause, panning to shake the camera left and right, tilting to shake up and down, tracking to move the camera position in parallel, and tracking to move back and forth in parallel are performed. Be seen. On the other hand, with respect to the surrounding environment 25, the surrounding brightness, temperature, humidity and the like are shooting conditions.

FIG. 2 is a block diagram showing the construction of an embodiment of an information recording mechanism of a multi-information camera according to the present invention, in which 30 is a lens, 31 is a diaphragm, 32 is a zoom mechanism, and 3 is a zoom mechanism.
3 is an electronic shutter, 34 is an image sensor, 35 is an electronic drive system, 36 is a data conversion unit, 37 is a video / audio processing unit, 3
8 is a secondary processing system, 39 is a format configuration unit (conversion unit), 40 is a video / audio recording mechanism, 41 is an auxiliary information recording mechanism, 42 is a time stamp generating unit, 43 is a recording start / pause / stop switch, 44 Is a microphone, 45 is a distance sensor, 46 is a camera height sensor, 47 is a camera position detection mechanism, 48 is an angular velocity detection mechanism, 49 is a temperature sensor,
50 is a humidity sensor, 51 is another ambient environment sensor, 5
2 is an external input interface.

FIG. 3 is a diagram for explaining the relationship of information in the multi-information camera of this embodiment. 30A is original information, 30B is additional information to be recorded, 301 is imaging system information, and 302 is camera position information. , 303 is camera movement information, 304 is subject-related information, 305 is surrounding environment information, 306 is external input information, 307 is various event marks, and 308 is absolute time information.

FIG. 4 is a diagram for explaining the state of the recorded video / audio information and the supplementary information and the use thereof in this embodiment. 40A and 40B are recording media, 401 is an image sequence, and 401A is a cut. Points, 402 is information on the cut point 401A, 403 is a frame image at the cut point 401A, 4
Reference numeral 04 is a frame video sequence.

Next, the operation of this embodiment will be described with reference to FIGS. First, when the photographer 22 holds the camera 21, the absolute position and orientation of the camera 21 are determined. Camera 21
The absolute position of is detected by the camera position detection mechanism 47, for example, by a positioning method using radio waves from a satellite or a positioning method using reference of a ground reference wave. In terms of accuracy, macro position detection is performed.
Further, the downward distance of the camera 21 is set as the height of the camera 21, and the height is measured by the camera position detection mechanism 47 by the ultrasonic reflection measurement method or the like. The change of the position of the camera 21, the absolute direction of the optical axis of the lens and the change thereof can be detected by using a gyro with high accuracy by the angular velocity detection mechanism (sensor) 48 and the like.

In the image pickup system, the result obtained by the automatic or manual determination mechanism of the diaphragm and the focus, which is conventionally used in the video camera, is used as the diaphragm 31 and the zoom mechanism 32.
It is taken out from the encoder output of the electronic drive system 35. In addition, electronic shutter 3 for shutter speed
The set value of 3 is taken out.

Regarding the surrounding environment, for example, temperature and humidity are detected by sensors 49 and 50 for measurement, respectively.

The secondary processing unit 38 generates brightness information from the aperture value of the lens and the average output value detected on the image pickup device 34.

These various kinds of supplementary information are formed into a fixed format by the format constructing unit 39 and are recorded by the supplementary information recording mechanism 41 in synchronization with the time information generated by the time stamp generating unit 42.
The video and audio are recorded by the video / audio recording mechanism 40 together with the time stamp by the recording mechanism of a normal video camera. Regarding the start / stop / pause of shooting, not only the timing of recording mechanism control to the recording mechanisms 40 and 41 is given, but also mark information indicating a cut point is recorded as a trigger of the cut point at the start timing. The time stamp at this cut point defines the video (shot) between the cut points in absolute time. If the frame video recorded at this timing is compressed and recorded as a part of the supplementary information, it can be used as a graphical index as a representative video of the shot starting at the cut point.

In addition, by using information detected by various sensors and generated information, real-time processing is performed at the time of recording,
Information for recording as supplementary information is generated in the secondary processing unit 38. As a method of generating secondary information, when the processing result is always recorded at the time of recording, or when the processing itself is filtering and the value of the result satisfies a certain condition,
Mark information indicating that may be recorded. The processing / evaluation algorithm in the secondary processing unit 38 is set in advance, but by making them programmable, general-purpose processing becomes possible. For example, when the camera position or the amount of change in the camera optical axis exceeds a certain value, mark information indicating that an event requiring a sudden camera operation has occurred is recorded. For example, using the camera optical axis displacement obtained from the angular velocity detection value, mark a range exceeding a certain threshold as panning or tilting, or add zoom-in / out mark information based on the change in focal length. You can

Further, information relating to photographing from the outside by data such as voice, image and text, for example, voice narration of the photographer, texts and drawings to be explained, etc. are inputted from the external input interface 52 and used as supplementary information. Add.

On the other hand, in the case of recording captured images on a sequential recording medium / reproducing medium such as a video tape, the above-mentioned additional information is a randomly accessible recording medium different from the image,
For example, if the image is recorded on a floppy disk, it is possible to perform a search and an image analysis primary process based on the information recorded as additional information without processing the entire image itself after the recording. For example, as shown in FIG. 4, by using the information 402 of the cut points 401A recorded as the mark information, it is possible to immediately list the time information of each cut and the frame at the cut points 401A. Immediately list video 403 (frame video sequence 404)
By displaying the contents of the captured image visually, a rough summary image editing etc. as a primary process can be realized in a short time without using the actual image information based on this.

Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above embodiments, and it is needless to say that various modifications can be made without departing from the scope of the invention. Absent.

[0031]

As described above, according to the multi-information camera of the present invention, at the time of photographing, information which has not been electronically recorded in the past is automatically recorded as supplementary information at the same time as video and audio. Since it is possible to do so, the information as an index when the video is automatically processed by the computer after recording,
The information necessary for more processing can be obtained. That is, the supplementary information after recording gives information necessary for image recognition as well as temporary processing such as searching and editing as described above when the computer processes the image during image processing.
For example, the imaging system condition information, the camera position information, and the camera optical axis information give important information for estimating the three-dimensional model of the object from the image.

In addition to the information temporarily detected / generated by the camera, the secondary information is added as supplementary information by performing real-time processing at the time of photographing based on the information, and the supplementary information can be randomly accessed. By recording on the recording medium, it is possible to perform a process such as a search at a high speed by processing only the incidental information without necessarily processing all the video during computer processing after recording.

[Brief description of drawings]

FIG. 1 is a diagram showing a situation of a photographer, a camera, a subject, a surrounding environment, and the like regarding photography in one embodiment of the present invention,

FIG. 2 is a block diagram showing the configuration of an embodiment of an information recording mechanism of a multi-information camera according to the present invention,

FIG. 3 is a diagram for explaining a relation of information in the multi-information camera of the present embodiment,

FIG. 4 is a diagram for explaining a state and use of recorded video / audio information and incidental information according to the present embodiment;

FIG. 5 is a diagram showing a configuration of a conventional video / audio recording system using a time code.

[Explanation of symbols]

10 ... Imaging unit, 11 ... Image recording unit, 12 ... Time code generator, 13 ... Audio recording unit, 14 ... Microphone, 20 ... Sun, 21 ... Camera, 22 ... Photographer, 23 ... Subject, 24
… Camera operation, 25… Surrounding environment, 30… Lens, 31…
Aperture, 32 ... Zoom mechanism, 33 ... Electronic shutter, 34
... image pickup device, 35 ... electronic drive system, 36 ... data conversion unit,
37 ... Video / audio processing unit, 38 ... Secondary processing system, 39 ... Format conversion unit, 40 ... Video / audio recording mechanism, 41 ...
Additional information recording mechanism, 42 ... Time stamp generating section, 43
... Recording start / pause / stop switch, 44 ... Microphone, 45 ... Distance sensor, 46 ... Camera height sensor, 47
... Camera position detection mechanism, 48 ... Angular velocity detection mechanism, 49 ...
Temperature sensor, 50 ... Humidity sensor, 51 ... Other ambient environment sensor, 52 ... External input interface.

Continuation of front page (56) Reference JP-A-5-252432 (JP, A) JP-A-5-37846 (JP, A) JP-A-5-14776 (JP, A) JP-A-4-347977 (JP , A) JP 4-306079 (JP, A) JP 4-71070 (JP, A) JP 4-70726 (JP, A) JP 4-52635 (JP, A) JP 2-301267 (JP, A) JP-A-2-98287 (JP, A) JP-A-1-25068 (JP, A) International Publication 92/9082 (WO, A1) (58) Fields investigated (Int.Cl) . ⁷ , DB name) H04N 5/225

Claims (3)

(57) [Claims] 1. A video / audio recorder for recording video and audio.
Time data student into data is generated and the stage, the absolute time at the time of shooting
Camera position data into data by detecting the formation unit, the position of the camera
It means a, camera orientation data into data by detecting the direction of the camera
And means, Taking into data to generate a captured related information from the imaging control system
Camera motion detecting and data of the image related information data means, the camera motion data
And data means, surrounding environment information data to data of detecting the ambient environment information
Data conversion means and each data converted into data by each data conversion means as additional information , the time data from the time data generation means.
And an auxiliary information recording means for recording in synchronization with the data.
It is a multi-information camera, and at least one data conversion means of each data conversion means
Pre-determined pre-processing for digitized data
And a processing result obtained by the processing means in the incidental information recording means.
Marking information is displayed when the preset conditions are met.
The time from the time data generating means as supplementary information.
A multi-information camera characterized by recording in synchronization with time data . 2. The additional information recording means is the processing means.
When the processing result in
The frame image is used as additional information to generate the time data.
Characterized by recording in synchronization with time data from the column
The multi-information camera according to claim 1, wherein 3. The incidental information recording means is configured to detect the incidental information.
Information is recorded separately from the recording medium for recording video and audio.
Characterized by recording on a random access recording medium
The multi-information camera according to claim 1 or 2.