JP2009159279A - Camera instrument and image reproducer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make clear, even after editing, where and how images are shot without recording shooting location data separately from image data. <P>SOLUTION: The camera instrument 1 includes an image pick-up means which outputs an image pick-up signal corresponding to an optical image, an image data generating means which generates image data corresponding to the optical image by using the image pick-up signal output from the image pick-up means, a position information obtaining means which obtains position information relative to the shooting location, a direction information obtaining means which obtains direction information for specifying the direction of the image pick-up means and an inserting means which inserts the position information obtained by the position information obtaining means and the direction information obtained by the direction information obtaining means into the image data generated by the image data generating means when the image data generating means generates image data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a camera device such as a digital camera or a video camera, and a video playback device that plays back a video signal of a video shot by the camera device.

  2. Description of the Related Art Conventionally, a camera device (also referred to as an electronic camera device) that captures an optical image of a subject, converts it into video data, and stores it electronically is known. Conventional camera devices include a digital camera mainly for taking still pictures and a video camera mainly for taking moving pictures.

Various digital cameras and video cameras have been widely used. For example, Patent Document 1 discloses a digital camera that displays a place name of a shooting location added to a still image.
JP 2002-262216 A

  In the case of the camera device described above, when a still image is reproduced, the place name of the shooting location can be added and displayed.

  However, in order to add and display the location name of the shooting location when the video is played back, record the shooting location data (hereinafter referred to as “shooting location data”) in addition to the video data obtained by shooting. There must be. In addition, unlike still images, moving images are often shot while the camera orientation and tilt angle (angle) changes during shooting, so shooting location data is recorded for each scene in the video data, and video data And shooting location data must be associated with each other.

  In addition, if editing work such as deleting unnecessary parts in video data or inserting another scene is performed after shooting, the video data will change accordingly. If the location data is not associated with each other, the location of the place name or the like cannot be clarified after editing.

  Therefore, the present invention has been made to solve the above-mentioned problem, and clearly shows where and how the video was shot after editing without recording the shooting location data separately from the video data. It is an object of the present invention to provide a camera device and a video playback device that can be used.

  In order to solve the above-described problems, the present invention provides an image pickup unit that outputs an image pickup signal corresponding to an optical image, and an image data generation that generates image data corresponding to an optical image using the image pickup signal output from the image pickup unit. Means, position information acquisition means for acquiring position information relating to the shooting location, orientation information acquisition means for acquiring orientation information for specifying the orientation of the imaging means, and when the video data generation means generates video data, The camera apparatus includes an insertion unit that inserts the position information acquired by the position information acquisition unit and the direction information acquired by the direction information acquisition unit into the video data generated by the video data generation unit.

  Further, the present invention provides a decoding means for decoding the compression-encoded video data, a map information storage means for storing map information for generating a position image, and when the decoding means decodes the video data, The additional information inserted in the user data area for adding the additional information in the video data is extracted and stored in the storage means together with the synchronization data, and from the additional information stored in the storage means, the latitude, Extraction means for extracting position information including longitude and altitude and azimuth information indicating the direction, and reading means for reading out corresponding map information corresponding to latitude and longitude from the position information extracted by the extraction means from the map information storage means And a display control means for displaying the corresponding position image generated from the corresponding map information read by the reading means.

  As described above in detail, according to the present invention, it is possible to clarify where and how the video was shot even after editing without recording the shooting location data separately from the video data. A camera device and a video playback device are obtained.

Embodiments of the present invention will be described below. In addition, the same code | symbol is used for the same element and the overlapping description is abbreviate | omitted.
(Configuration of camera device)

  FIG. 1 is a block diagram showing a main configuration of a camera apparatus 1 according to an embodiment of the present invention. A camera apparatus 1 shown in FIG. 1 is a digital video camera apparatus that mainly captures moving images and can also capture still images.

  The camera device 1 includes a lens 2, a CCD (Charge Coupled Device) 3, AD converters 4 and 8, a video encoder 5, a PS multiplexer 6, a microphone 7, and an audio encoder 9. The camera device 1 also includes a magnetic orientation sensor 10, a GPS (Global Positioning System) receiver 11, a CPU 12, a memory 13, a display microcomputer 14, an SD card controller 15, a color LCD panel 16, and an operation panel 17. 1, each component is connected to a bus 18.

  The lens 2 is an image capturing means for capturing an optical image, and captures an optical image of a subject and forms an image on the CCD 3. The CCD 3 is an imaging unit, and generates an imaging signal corresponding to the formed optical image and outputs it to the AD converter 4. The AD converter 4 converts the imaging signal output from the CCD 3 into digital video data and outputs it. The video encoder 5 compresses and encodes the input video data into an MPEG2 (Moving Picture Experts Group phase 2) video stream, generates a PES (Packetized Elementary Stream) packet from the MPEG2 video stream, and outputs it to the PS multiplexer 6. The AD converter 4 and the video encoder 5 constitute video data generating means.

  The microphone 7 takes in the sound and converts it into an electric signal and outputs it to the AD converter 8. The AD converter 8 converts the electric signal output from the microphone 7 into a digital format and generates sound data. The audio encoder 9 encodes input audio data into an audio stream, generates a PES packet, and outputs the PES packet to the PS multiplexer 6. The PS multiplexer 6 multiplexes input PES packets and outputs PS packets.

  The magnetic azimuth sensor 10 is azimuth detection means, detects the azimuth that the lens 2 and the CCD 3 are facing, and outputs azimuth information. The GPS receiver 11 receives a GPS signal transmitted from a GPS satellite and outputs it to the CPU 12. In the following description, azimuth information and position information are collectively referred to as azimuth position information.

  The CPU 12 operates according to a program stored in the memory 13 to control the camera device 1, while operating as a position information acquisition unit, a direction information acquisition unit, and a conversion unit, which will be described later. The memory 13 stores a program executed by the CPU 12. The display microcomputer 14 controls display of moving images and still images on the color LCD panel 16, and the SD card controller 15 controls reading and writing of PS packets with respect to the SD card 20. A moving image and a still image are displayed on the color LCD panel 16.

  In order to clarify where and how the shooting was performed without recording the shooting location data separately from the video data when the camera device 1 having the above configuration shot a landscape or the like, The position information is inserted into the MPEG2 video stream 31. The configuration for this will be described below.

  FIG. 2 is a diagram showing an operation procedure when the azimuth position information is inserted into the MPEG2 video stream 31 by the video encoder 5, the CPU 12, the magnetic azimuth sensor 10, and the GPS receiving unit 11.

  The video encoder 5 outputs an interrupt P1 to the CPU 12 when compressing and encoding the video data and outputting the MPEG2 video stream 31. When the interrupt P <b> 1 is output, the CPU 12 performs operations as a position information acquisition unit and a direction information acquisition unit, and performs acquisition P <b> 2 of the direction position information from the magnetic direction sensor 10 and the GPS receiver 11. At this time, the CPU 12 detects the position where the camera device 1 is placed, that is, the latitude, longitude, and altitude of the shooting location from the GPS signal output from the GPS receiving unit 11 and acquires position information.

  Subsequently, the CPU 12 performs an operation as a conversion unit, and performs conversion P3 of azimuth position information according to a conversion rule described later. Further, the CPU 12 performs writing P4 of the azimuth position information to the buffer 5a. Then, the video encoder 5 reads out the azimuth position information from the buffer 5 a and performs an operation as an inserting unit, and performs insertion P5 of the azimuth position information into the MPEG2 video stream 31 to generate the MPEG2 video stream 31.

  Here, the MPEG2 video stream 31 is composed of a set of a plurality of images called GOP (group of pictures) 32 as shown in FIG. There is a GOP header 33 at the head of the GOP 32, and after that, a user data area for storing additional information called extension_and_user_data 34 is defined, and thereafter, data of each scene is arranged. In each scene data, extension_and_user_data 37 is provided after picture_header 36. In addition, there are picture_cording_extension 38 and picture_data 39.

  In the camera device 1, based on the configuration of the MPEG2 video stream 31, when the video encoder 5 compresses and encodes video data, the azimuth position information is inserted into the extension_and_user_data 34 and the extension_and_user_data 37. Focusing on the fact that it is possible to easily determine which frame corresponds to playback, orientation position information is inserted into extension_and_user_data 34 and extension_and_user_data 37.

  By doing this, the MPEG2 video stream 31 has a configuration inseparable from the shooting location data by incorporating the azimuth position information. For this reason, the video playback apparatus 100 described later can display a shooting location map or a place name in synchronization with the corresponding frame by using only the MPEG2 video stream 31 without separately recording shooting location data. it can.

  H. In the case of H.264, azimuth position information can be inserted into user_data_unregistered of SEI (Supplemental Enhancement Information). In VC1, the azimuth position information can be inserted as User Data immediately after the Entry-Point Header.

  Further, it is also possible to insert azimuth position information into TS (Transport Stream), PS (Program Stream), ES (Elementary Stream), etc. defined in IEC13818-1. The current digital broadcast or DVD video data is generally recorded on a recording medium in a TS or PS format defined by a standard called IEC13818-1, or is superimposed on a broadcast wave and transmitted.

  In IEC13818-1, Private Data can be inserted in various places so that data other than data defined by standards such as video and audio can be inserted at this PS and TS, and ES level before PS and TS. A storage location is prepared.

  This is described in Annex-H of IEC13818-1. In this case, the azimuth position information can be inserted as a stream-type user private of the TS packet, and can also be inserted into the adaptation_field of the TS packet. Further, it can be inserted into private_stream_1 and private_stream_2 of the PES packet, or can be inserted into a descriptor in program_stream_map and the private section of TS. In the following description, a case where azimuth position information is inserted into extension_and_user_data 34 of the MPEG2 video stream 31 will be described as an example.

  Here, supplementary explanations for ES, TS, and PS are as follows. Each data such as video and audio compressed and encoded by MPEG2 is called ES (Elementary Stream). A packetized ES is called a PES (Packetized Elementary Stream). Data for reproducing video and audio is a collection of ESs composed of video ESs, audio ESs, and the like, and all these ESs must be decoded synchronously. MPEG2 defines a method for that purpose. A group of multiplexed ESs is called a program. When these ESs are multiplexed to form a stream as a group of programs, two types of streams, PS and TS, are defined.

  PS is assumed to be applied to data transmission and storage in an error-free environment, and is used in a recording medium using a powerful error correction code such as a DVD. PS can only have one program per stream. The packet length is variable.

  TS is assumed to be applied to environments where transmission errors occur, such as broadcasting and communication networks, and redundancy is higher than that of PS. The TS can put a plurality of programs in one stream. The packet length is fixed at 188 bytes.

  By the way, in the DVD standard, a method for inserting a closed caption into extension_and_user_data 34 immediately after the GOP header 33 is defined. Therefore, in this embodiment, the azimuth position information is inserted as a closed caption according to the DVD standard.

  According to the DVD standard, the closed caption information (line 21 data) 35 to be inserted into the line 21 is inserted into the extension_and_user_data 34 immediately after the GOP header 33 in the format shown in FIG. A closed caption, copy control information, and the like are inserted into the line 21 in accordance with the standard of EIA-608-B in US broadcasting. Closed captioning is a teletext in the United States that causes subtitles to appear on the screen.

  In the NTSC signal, one image is composed of 525 lines, but the first 21 lines are periods for moving the scanning lines of the television from the bottom to the top. This period is called VBI (Vertical Blanking Interval), and is a period during which video display is not performed. Therefore, additional information such as aspect ratio and copy control information is transmitted. Line 21 is used in the United States, and line 20 is used in Japan.

  In the camera device 1, since the azimuth position information is inserted into the MPEG2 video stream 31 in the format of the line 21 data 35, as described above, the CPU 12 converts all the azimuth position information into text data.

  In the present embodiment, latitude and longitude are set to a 0.1 degree unit in a range of 0.0 to 360.0 degrees, and a 4-digit decimal number, and for identification, a character string of “LO” for longitude and “LA” for latitude. Use the notation. Then, for example, when the longitude is 35 degrees and the latitude is 139 degrees, the position information is “LO0350LA1390”.

  The altitude is minus 1000 to plus 8999 m, the unit is meters (m), and is expressed as a 4-digit decimal number obtained by adding “1000” to the actual value, with the character string “HI” added for identification. . Then, for example, when the altitude is 600 m, the altitude information becomes “HI1600”.

  Furthermore, if the direction is expressed as 360 degrees in a counterclockwise direction with 0 degree as the north, the direction is expressed by adding a character string of “DR” for identification at the beginning with a 3-digit decimal number. Then, when the direction is true north, the direction information is “DR000”.

  In accordance with the above rules (hereinafter referred to as “conversion rules”), the CPU 12 converts the azimuth position information into text information. In this case, for example, when the shooting location is 35 degrees longitude, 139 degrees latitude, altitude 400 m, and the direction is true north, the azimuth position information is converted into text data such as “LO0350LA1390HI1400DR000”. The video encoder 5 inserts the text data generated in this way into the extension_and_user_data 34 as a closed caption for each GOP unit.

  The closed caption defined in the DVD standard is based on the standard for embedding data in the line 21 described in EIA-608, and only 2 characters per frame can be inserted.

  However, since one GOP is usually about 0.5 seconds, if the MPEG2 video stream 31 is generated at 60 frames per second (60 F / S), since 1 GOP is 30 frames, 60 characters are inserted with 30 × 2 characters. Is possible. If a mechanism for displaying the direction additional information is displayed for each head of the GOP, it is possible to display additional information such as the position on the map where the image was taken in synchronization with the video.

  Next, the operation content of the camera apparatus 1 having the above configuration will be described. In the camera device 1, an optical image is formed on the CCD 3 by the lens 2, and an imaging signal corresponding to the optical image is generated by the CCD 3. The generated imaging signal is converted into digital video data by the AD converter 4 and input to the video encoder 5.

  The video encoder 5 compresses and encodes the input video data into an MPEG2 video stream 31, further converts it into a PES packet, and outputs the PES packet to the PS multiplexer 6.

  Further, when the video encoder 5 compresses and encodes the video data into the MPEG2 video encoder 31, as described above, the CPU 12 acquires the orientation information from the magnetic orientation sensor 10 in real time and receives the GPS signal from the GPS receiving unit 11. The position information is acquired by detecting latitude, longitude, and altitude from the input GPS signal. Then, the CPU 12 converts the acquired azimuth information and position information into text data in accordance with the conversion rules described above. Further, the CPU 12 converts the obtained text data into the DVD standard format shown in FIG.

  Then, the video encoder 5 inserts the data generated by the CPU 12 into the extension_and_user_data 34 immediately after the GOP header 33 as a closed caption. Thus, the MPEG2 video encoder 31 in which the azimuth position information is incorporated and integrated is generated.

  On the other hand, the electric signal output from the microphone 7 is converted into a digital signal by the AD converter 8, encoded by the audio encoder 9 with MPEG1 Layer 3, and further a PES packet is generated and output to the PS multiplexer 6. .

  The PS multiplexer 6 multiplexes the input PES packet of the MPEG2 video stream 31 and the PES packet of the audio stream, and outputs the PS packet as one PS stream.

  Then, the CPU 12 performs an operation as a PS packet recording control means, instructs the SD card controller 15 to record the PS packet acquired from the PS multiplexer 6 on the SD card 20 as a recording medium.

  Next, the video playback device 100 that plays back video shot by the camera device 1 will be described. The video reproduction device 100 reproduces the video signal using the PS packet recorded on the SD card 20.

  As shown in FIG. 5, the video reproduction apparatus 100 includes an SD card controller 101, a CPU 102, a memory 103, a video decoder 104, an audio decoder 105, a graphic controller 106, a compositor 107, a D / A converter 108, 109. The video playback apparatus 100 also includes an HDD controller 110, a hard disk 111, an operation panel 112, a display sub-microcomputer 113, and a VFD display panel 114.

  In the video reproduction apparatus 100, the SD card controller 101 reads out the PS packet recorded on the SD card 20 in accordance with an instruction from the CPU 102. Then, the CPU 102 analyzes the PS packet, extracts the PES packet, and classifies it into a video stream and an audio stream. The former is output to the video decoder 104 and the latter is output to the audio decoder 105. The audio decoder 105 extracts the ES from the input PES packet, performs a decoding process, and outputs an audio signal to the D / A converter 109. The D / A converter 109 generates and outputs an analog audio signal.

  The video decoder 104 is a decoding means that extracts an ES from an input PES packet, performs MPEG2 decoding processing, and synchronizes a generated video frame with PTS (Presentation Time Stamp) as synchronization data. While outputting to the compositor 107. On the other hand, the video decoder 104 operates as storage control means for storing data (azimuth position information) as a closed caption extracted from the extension_and_user_data 34 immediately after the GOP header 33 in the memory 103 together with the PTS, and notifies the CPU 102 by an interrupt. .

  When the CPU 102 receives the notification by interruption, the CPU 102 reads out the data as the closed caption written in the memory 103 and performs the operation as the extracting unit, analyzes the read data, and acquires the position information (latitude, longitude, and altitude). Orientation information is extracted.

  On the other hand, the CPU 102 operates as a reading unit, instructs the HDD controller 110 to read corresponding map information corresponding to the position information among the map information recorded in the HDD 111 as the map information storage unit, and further An operation as an image generation unit is performed, and a corresponding map image is generated based on the read corresponding map information.

  In addition, the CPU 102 performs an operation as a display control unit, and outputs a corresponding map image display instruction to the graphic controller 106. At this time, the CPU 102 performs an operation as a changing unit, and outputs an instruction for changing the orientation by rotating the corresponding map image so that the display corresponds to the orientation of the camera device 1 based on the azimuth position information. .

  The graphic controller 106 performs scaling of the video data in accordance with an instruction from the CPU 102 and rotates the corresponding map image to draw it in a frame buffer (not shown). Then, the compositor 107 synthesizes the video map output from the video decoder 104 with the corresponding map image output from the graphic controller 106 by alpha blending, and outputs it to the D / A converter 108. The D / A converter 108 generates and outputs an analog video signal from the digital video data. When the video playback device 100 operates as described above, a video 200 as shown in FIG. 6 is displayed on a display device (not shown) by an output video signal.

  Since this video image 200 displays a map image 201 in addition to the video imaged by the camera device 1, the map image 201 can clarify where the video image 200 was shot. In addition, since the orientation of the displayed map image 201 changes according to the orientation information extracted from the PES packet stored in the camera device 1, it is possible to confirm how the orientation of the map image 201 changes. It is also possible to clarify how the shooting was performed.

  As described above, the azimuth position information is inserted into the MPEG2 video stream 31 in real time at the time of shooting so that the camera device 1 can reproduce the video in synchronization with the map image. Therefore, it is not necessary to record the shooting location data as data different from the video data, and it is possible to clarify where and how the shooting was performed only with the MPEG2 video stream 31.

  Further, when editing processing such as unnecessary part deletion, addition of another scene, and joining of a plurality of scenes is performed on the generated MPEG2 video stream 31, the corresponding azimuth position information is also edited. The correspondence with the position information is maintained as it is. Therefore, even after editing, only playback using the MPEG2 video stream 31 is performed, and it is possible to clarify where and how shooting was performed as before editing without preparing shooting location data separately. .

  In addition, in the video reproduction apparatus 100, the map image 201 is displayed, but in addition to the map image 201, images of characters and symbols (for example, 1-*-* Shibaura, Minato-ku, Tokyo) that clarify the place name and location. May be displayed. These images and the map image 201 constitute a position image indicating the position. In addition to such display of the position image, voice guidance may be output.

  The video playback apparatus 100 includes the HDD 111 as an external storage device that can read and write data by random access. In addition to the HDD 111, the video playback apparatus 100 includes a DVD drive that records data on a DVD (Digital Versatile Disk). It may be. Then, the moving image data may be stored in a rewritable storage medium such as DVD-RW or DVD-RAM by a DVD drive.

  The above description is the description of the embodiment of the present invention, and does not limit the apparatus and method of the present invention, and various modifications can be easily implemented. In addition, an apparatus or method configured by appropriately combining components, functions, features, or method steps in each embodiment is also included in the present invention.

It is a block diagram which shows the main structures of the camera apparatus 1 which concerns on embodiment of this invention. It is a figure which shows the operation | movement procedure when inserting azimuth | direction position information in a PS packet by a video encoder, CPU, a magnetic azimuth | direction sensor, and a GPS receiving part. It is a figure which shows the structure of an MPEG2 video stream. It is a figure which shows an example of the format of line 21 data. It is a block diagram which shows the main structures of the video reproduction apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the image | video reproduced | regenerated with a video reproduction | regeneration apparatus.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Camera device, 2 ... Lens, 3 ... CCD, 5 ... Video encoder, 10 ... Magnetic direction sensor, 11 ... GPS receiver, 12, 102 ... CPU, 13, 103 ... Memory, 15 ... SD card controller, 20 ... SD card 31... MPEG2 video stream 34. 37 extension_and_user_data 100... Video playback device 104.

Claims (10)

An imaging means for outputting an imaging signal corresponding to the optical image;
Video data generating means for generating video data corresponding to the optical video using the imaging signal output from the imaging means;
Position information acquisition means for acquiring position information regarding the shooting location;
Orientation information acquisition means for acquiring orientation information for specifying the orientation of the imaging means;
When the video data generation means generates the video data, the position information acquired by the position information acquisition means and the direction information acquired by the direction information acquisition means are generated by the video data generation means. A camera device comprising an insertion means for inserting into video data. The video data generation means includes encoding means for compressing and encoding the video data,
2. The camera apparatus according to claim 1, wherein the insertion unit inserts the position information and the azimuth information into the video data when the video data is compressed and encoded by the encoding unit. The encoding means converts the video data into MPEG2, VC1, or H.264. H.264 format video stream,
The insertion means is the MPEG2 or VC1 or H.264 encoded by the encoding means. 3. The camera apparatus according to claim 2, wherein the position information and the orientation information are inserted into a user data area for adding additional information in a H.264 format video stream. Converting the position information into text information indicating latitude, longitude and altitude, and further converting means for converting the azimuth information into text information indicating a direction;
The said insertion means inserts the said text information converted by the said conversion means as a closed caption in the user data area arrange | positioned immediately after the GOP header of the said user data area. Camera device. GPS receiving means for receiving GPS signals transmitted from GPS satellites;
Azimuth detecting means for detecting the azimuth of the imaging means,
The position information acquisition means acquires latitude, longitude and altitude as the position information from the GPS signal received by the GPS reception means when the video data is encoded by the encoding means,
5. The azimuth information acquisition means acquires the azimuth detected as the azimuth information from the azimuth detection means when the video data is encoded by the encoding means. The camera device according to one item.   5. The camera apparatus according to claim 4, further comprising PS packet generation means for generating an MPEG2 PS packet from the MPEG2 video stream in which the text information is inserted. 7. The camera apparatus according to claim 6, further comprising PS packet recording control means for recording the MPEG2 PS packet generated by the PS packet generating means on a recording medium. Decoding means for decoding the compression-encoded video data;
Map information storage means for storing map information for generating a position image;
Storage control means for extracting additional information inserted in a user data area for adding additional information in the video data when the decoding means decodes the video data, and storing it in the storage means together with synchronization data; ,
Extraction means for extracting position information including latitude, longitude and altitude, and azimuth information indicating a direction from the additional information stored in the storage means;
Reading means for reading out corresponding map information corresponding to the latitude and longitude of the position information extracted by the extraction means from the map information storage means;
And a display control means for displaying a corresponding position image generated from the corresponding map information read by the reading means.   9. The apparatus according to claim 8, further comprising a changing unit that changes the orientation of the corresponding position image according to the orientation information extracted by the extraction unit when the display control unit displays the corresponding position image. The video playback device described.   10. The video reproduction apparatus according to claim 8, further comprising a synthesizing unit that synthesizes the corresponding position image read by the reading unit with the video data generated by the decoding unit.