JP6248314B2 - Imaging apparatus, captured image processing method, and imaging system - Google Patents

Description

  The present invention relates to a system and method for recording / reproducing an image showing position information and a user's captured image as one moving image.

  In recent years, digitalization of still cameras and video cameras has progressed, and it has become possible for users to shoot and record very easily, and a large amount of video information can be recorded. For this reason, for example, when going on a long trip, it may be possible that when viewing a video shot after the trip is over, it is not possible to know which video was shot where. At this time, if the map information and the captured video are combined in one video data, it is very convenient for viewing and organizing later. In addition, even if the data is taken by a surveillance video camera installed at a plurality of locations, it is very convenient if the image showing the position information and the taken video are combined in one video data.

  As a technology close to this, there is a driving recorder technology that has recently been installed in taxis and the like. The driving recorder automatically determines shooting timing and records situation data and shooting data including position information. In order to automatically determine the photographing timing, a large amount of unnecessary data is recorded in some cases, and a technique for preventing this is invented. For example, in Patent Document 1, a learning function is used to select recording data, and when shooting, the user inputs necessary / unnecessary information later to learn whether shooting is necessary.

JP 2007-11907 A

  However, since it automatically shoots, there is a possibility that it may not match the necessity of the user completely, and it is not suitable for the user to freely shoot a favorite video together with map information.

  The present invention has been made in view of the above problems, and provides a system and method for recording / reproducing an image indicating position information and a user's captured image as one moving image without performing complicated processing. provide.

  The video camera is provided with means for inputting an image from another video input source in addition to the means for inputting a normal photographed image. Further, the video data from these two input means is arbitrated and fetched and stored in a memory. An image input arbitration unit is provided. Thus, by using the other video input source as the video output terminal of the car navigation, the car navigation screen (map information) and the image taken by the user can be taken in the memory together. Further, in a surveillance video camera or the like, a medium that stores map image data in advance, such as a USB flash memory or an SD card, and a reading means for reading map image data from the medium are used as separate video input sources. , Can be taken into memory as well. By compressing and encoding the data in the memory as a moving image, the image indicating the position information and the user's captured image can be processed as one moving image.

  According to the present invention, an image showing position information and a user's captured image can be recorded and reproduced as one moving image without performing complicated processing.

Configuration diagram of image recording / reproducing system of Embodiment 1 FIG. 1 is a flowchart of an image recording / reproducing system according to the first embodiment. Schematic diagram of stream data generated by the image recording / reproducing system of Embodiment 1 Configuration diagram of image recording / reproducing system of Embodiment 2 Flow chart of image recording / reproducing system of embodiment 2 Schematic diagram of stream data generated by the image recording / reproducing system of Embodiment 2 Configuration diagram of image recording / reproducing system of Embodiment 3 Schematic diagram of a system for transferring stream data at hot spots Thumbnail display example

  The best mode for carrying out the present invention will be described.

  Embodiments of an image recording / reproducing system and method according to the present invention will be described below with reference to the drawings.

  Example 1 will be described. In this embodiment, an operation from when a user gives a shooting instruction to when shooting is performed will be described with reference to FIGS. FIG. 1 illustrates an image recording / playback system according to a first embodiment of the present invention. A video camera 1 directly inputs various instructions including a shooting instruction by a user, such as buttons and a touch panel. Such an operation input unit 11, a control unit 12 that controls each unit related to photographing, a photographed image input unit 13 that inputs a target image to be photographed by a user, digitizes, and outputs photographed image data, a navigation system described later Image input arbitration that arbitrates the order of the captured image data input from the external image input unit 14, the captured image input unit 13 and the navigation image data input from the external image input unit 14 Unit 15, image memory unit 16 that temporarily stores data output from image input arbitration unit 15, and image data read out from image memory unit 16 , MPEG2 and H. An image encoding unit 17 that performs image compression / encoding by a method such as H.264 and outputs encoded image data, and an audio input unit that inputs audio to be photographed by a user and digitizes and outputs audio data 18, the audio data output from the audio input unit 18 is compressed by a method such as AC3 or DolbyDigital, and the encoded audio data is output from the audio encoding unit 19, the encoded image data, and the encoded audio data. A data generation unit 1A that multiplexes and outputs stream data, a data recording unit 1B that records the stream data on a recording medium to be described later, and a recording medium 1C such as a DVD, Blu-ray (registered trademark), hard disk, and semiconductor memory. Have. The video camera 1 is connected to the navigation 2 via the AV cable 3. The navigation 2 includes a navigation main body unit 21 and a navigation display unit 22. The navigation main body unit 21 includes a navigation image output unit 211 that outputs a navigation video. The navigation 211 outputs a signal to the AV cable 3 connected to an external video terminal (not shown) or the like in the same manner as the video signal is always output to the navigation display unit 22.

  FIG. 2 shows an operation flow of each part in the first embodiment.

  A photographing procedure in the image recording / reproducing system of Embodiment 1 will be described. The user issues a shooting instruction to the operation input unit 11 using a button or switch of the video camera 1. The operation input unit 11 issues a shooting start instruction to the control unit 12. The control unit 12 issues an activation instruction to the captured image input unit 13, the audio input unit 18, and the external image input unit 14. The external image input unit 14 takes in a video signal that is constantly output from the external video terminal of the navigation 2 through the AV cable 3 in response to the activation instruction, generates navigation image data, and outputs it to the image input arbitration unit 15 To do. The captured image input unit 13 inputs an image to be captured by the user, digitizes it, generates captured image data, and outputs it to the image input arbitration unit 15. The voice input unit 18 inputs voice to be photographed by the user, digitizes it, generates voice data, and outputs it to the voice encoding unit 19. The navigation image data and the captured image data input to the image input arbitration unit 15 are temporarily stored in an area of the image memory 16 that is determined to be encoded in this order. That is, the navigation image data is read into a certain area of the image memory unit 16 that is read when the subsequent image encoding unit 17 processes the first image data, and the image memory is read when the second and subsequent image data is processed. The captured image data is temporarily stored in another area of the unit 16. The navigation image data and the captured image data temporarily stored in the image memory unit 16 are read out as image data by the image encoding unit 17 in this order. The image encoding unit 17 performs image compression and encoding of the image data. Here, the image encoding unit 17 compresses and encodes the navigation image data, which is the first image, only with data in one screen. Compress and encode as a H.264 I picture. The captured image data subsequent to the navigation image data is compressed and encoded using the image data for the first screen of the captured image data as an I picture. Subsequent images are, for example, P pictures and B pictures that are predicted and compressed from forward and backward images, for example, P pictures and B pictures, or I pictures that are compressed and encoded only with data in one screen, or The mixed data is compressed and encoded to generate encoded image data, which is output to the data generation unit 1A. In addition, the image encoding unit 17 performs processing with coarser quantization or temporarily increases the clock frequency in order to reduce the amount of code and lighten the processing in order to make the compression / encoding processing close to real-time operation. To change the processing speed. On the other hand, the audio data input to the audio encoding unit 19 is compressed and encoded by the audio encoding unit 19 to generate encoded audio data, which is output to the data generation unit 1A. The data generator 1A multiplexes the encoded image data and the encoded audio data to generate moving image stream data. A schematic diagram (example) of the stream data is shown in FIG. In FIG. 3, the stream data is shown in order from the top in the order of images that have been subjected to decoding / decompression processing. The first image shows a navigation screen, is compressed and encoded as an I picture, multiplexed so that there is no audio data, the second, third, and fourth images show captured image data, The second image is compressed and encoded as an I picture, the third image is a B picture, the fourth image is a P picture, and audio data is multiplexed using the encoded audio data. The stream data generated by the data generation unit 1A is output to the data recording unit 1B, and the data recording unit 1B records the stream data on the recording medium 1C.

  In order to play back the recorded stream data with the video camera 1, a picture analysis unit (not shown) for analyzing the picture type of the stream data is provided, and the stream data may be stream data in which two or more I pictures are continuous. For example, after the first I picture is output to the display unit of the video camera 1 (not shown), the same image is displayed for a preset time, and then the next I picture is output to the display unit. If the following picture is not an I picture, it is output to the display section one after another as in normal video data playback. If the stream data includes two or more I pictures, the first I picture is output to the display unit of the video camera 1 (not shown), and then the same image is displayed for a preset time. However, for example, the next image may be displayed starting from a user operation such as displaying the same image until the user performs an operation such as pressing the play button.

  In addition, in a moving image compression / encoding method in which data for stopping stream reproduction is embedded in a stream, such as a DVD-Video standard Still command, for example, a picture on the navigation screen and a captured image on the second and subsequent screens. For example, a VOBU (Video Object Unit) may be divided from the picture and inserted into the data generation unit 1A by the VOBU Still command, so that playback on a device other than the video camera 1 may be supported.

  As described above, in addition to means for inputting a normal photographed image to a video camera, means for inputting an image from another video input source is provided, and further, video data from these two input means is arbitrated and captured. By providing an image input arbitration unit for storing in the memory and using the other video input source as a video output terminal for car navigation, the car navigation screen (map information) and the image taken by the user are combined and stored in the memory. Can be taken in. By compressing and encoding the data in the memory as a moving image, the map information and the user's captured image can be processed as one moving image.

  In this embodiment, the navigation 2 is connected to the outside of the video camera 1. However, the same effect can be obtained even if the video camera 1 and the navigation 2 are a single device.

  The stream data recorded on the recording medium 1C may be automatically transferred to a server on the network via a wireless LAN environment (hot spot) installed in the city. FIG. 8 shows a schematic diagram of a system for transferring the stream data to a server in a hot spot environment. The video camera 1 includes a data reading unit 1E that reads the stream data from the recording medium 1C, and a wireless communication unit 1F that wirelessly transmits the stream data to the network 4. The stream data output from the wireless communication unit 1F is transferred to the server 5 connected to the network 4.

  In the present embodiment, the data generation unit 1A merely generates stream data. However, the current position information is received from the navigation 2 separately from the navigation image data, and stream data is generated in association with the current position information. The data may be output to the recording unit 1B and recorded on the recording medium 1C. At this time, when the thumbnail screen of the stream data recorded on the recording medium 1C is generated, the thumbnail display screen may be configured in combination with the map screen as shown in FIG. 9, using the current position information as a clue. In the example of FIG. 9, the position on the map corresponding to the position information associated with the stream data is indicated by a circle, and the first image of the captured image of the stream data is displayed in association with the circle. doing. In addition, when the video camera 1 has a plurality of recording media, for example, when it has a hard disk and a DVD, it is recorded on the hard disk during normal use of the video camera 1, and the user selects and selects the stream data. A title screen such as that shown in FIG. 9 may be automatically created and recorded on a DVD, such as when editing and recording on a DVD.

  In this embodiment, the user operates the buttons and switches of the video camera 1 to issue a shooting instruction from the operation input unit 11. However, the navigation 2 and the video camera 1 are connected to a communication means such as USB, Bluetooth, or wireless LAN. The user sets a point to be photographed in advance with respect to the navigation 2, and outputs a photographing instruction from the navigation 2 to the operation input unit 11 of the video camera 1 when moving to the set point. The unit 11 may start shooting by outputting a shooting start instruction to the control unit 12.

  Further, instead of the navigation 2 and the AV cable 3, a medium that stores map image data in advance, such as a USB flash memory or an SD card, and a reading device that reads the map image data from the medium are replaced, and FIG. As shown, the data is transferred to a server on the network in a hot spot environment, and the compressed image data recorded on the recording medium is read, and the compressed image data is packetized at a predetermined timing set by the user. Thus, the same effect can be obtained in video cameras for surveillance applications installed at a plurality of locations by having sending means for sending to the network. At this time, the wireless communication unit 1F may be replaced with a function of communicating via a wired LAN.

  A second embodiment will be described. FIG. 4 shows an image recording / reproducing system according to the second embodiment of the present invention. In the present embodiment, the configuration differs from the first embodiment in that the video camera 1 has an external audio input unit 181 that inputs audio from the navigation system 2 in addition to the configuration of FIG. The order of the data and the navigation voice data input from the external voice input unit 181 is arbitrated and output to the voice input arbitration unit 182, the voice memory unit 183 temporarily holding the data output from the voice input arbitration unit 182, and the navigation 2 In addition to the configuration of FIG. 1, the video camera 1 includes a voice activation unit 1D that transmits a voice activation instruction to the navigation 2 by, for example, infrared communication so as to activate voice. Receives voice activation instructions from the voice activation unit 1D via infrared communication and outputs navigation voice Navigation voice activation unit 213 issues a starting instruction to, and by the start instruction is that it has a navigation voice output unit 212 for outputting the navigation voice.

  FIG. 5 shows an operation flow of each part in the second embodiment.

  A photographing procedure in the image recording / reproducing system of the second embodiment will be described. The user issues a shooting instruction to the operation input unit 11 of the video camera 1. This photographing instruction may be an operation for starting a sound indicating position information or the like from the navigation 2 such as an operation of pressing a “current location” button of the navigation. The operation input unit 11 issues a shooting start instruction to the control unit 12. In addition, the operation input unit 11 issues a voice activation instruction to the voice activation unit 1D. The control unit 12 issues an activation instruction to the captured image input unit 13, the audio input unit 18, the external image input unit 14, and the external audio input unit 181. The external image input unit 14 takes in a video signal that is always output from the external video terminal or the like from the navigation 2 according to the activation instruction, generates navigation image data, and outputs the navigation image data to the image input arbitration unit 15. The navigation image data in this case is data of one picture or more for a predetermined time set in advance by the user or the like. The captured image input unit 13 inputs an image to be captured by the user, digitizes it, generates captured image data, and outputs it to the image input arbitration unit 15. On the other hand, in response to a voice activation instruction issued to the voice activation unit 1D by the operation input unit 11, the voice activation unit 1D issues a voice activation instruction to the navigation voice activation unit 213 of the navigation 2 by infrared communication or the like. This voice activation instruction is a signal that is generally provided as a navigation function and is output to the navigation 2 from the remote control when a “current location” button or the like of a remote control (not shown) attached to the navigation is pressed. May be output from the voice activation unit. When the navigation voice activation unit 213 receives the voice activation instruction, it issues an activation instruction to the navigation voice output unit 212. In response to the activation instruction, the navigation voice output unit outputs a voice such as “Current location is near Yoshida-cho, Totsuka-ku, Yokohama-shi” and the like and outputs it from an external voice terminal. The external voice input unit 181 takes navigation voice output from the navigation 2 from an external voice terminal or the like according to the activation instruction from the control unit 12, generates navigation voice data, and outputs the navigation voice data to the voice input arbitration unit 15. The navigation voice data in this case is data for a predetermined time set in advance by a user or the like. This set time may be the same as the time when the external image input unit 14 captures. The voice input unit 18 inputs voice to be photographed by the user, digitizes it, generates voice data, and outputs the voice data to the voice encoding unit 19. The navigation image data and the captured image data input to the image input arbitration unit 15 are temporarily stored in an area of the image memory 16 that is determined to be encoded in this order. That is, the navigation image data is read into a certain area of the image memory unit 16 that is read when the subsequent image encoding unit 17 processes the first image data, and the image memory is read when the second and subsequent image data is processed. The captured image data is temporarily stored in another area of the unit 16. The navigation image data and the captured image data temporarily stored in the image memory unit 16 are read out as image data by the image encoding unit 17 in this order. The image encoding unit 17 performs image compression and encoding of the image data. Here, the image encoding unit 17 compresses and encodes the navigation image data, which is the first image, only with data in one screen. Compress and encode as a H.264 I picture. The captured image data subsequent to the navigation image data is compressed and encoded using the image data for the first screen of the captured image data as an I picture. Subsequent images are, for example, P pictures and B pictures that are predicted and compressed from forward and backward images, for example, P pictures and B pictures, or I pictures that are compressed and encoded only with data in one screen, or The mixed data is compressed and encoded to generate encoded image data, which is output to the data generation unit 1A. On the other hand, the navigation voice data and the voice data input to the voice input arbitration unit 182 are temporarily stored in the area of the voice memory 183 determined to be encoded in this order. The navigation voice data and the voice data temporarily stored in the voice memory unit 183 are read as mixed voice data by the voice encoding unit 19 in this order. The mixed audio data input to the audio encoding unit 19 is compressed and encoded by the audio encoding unit 19, generates encoded audio data, and outputs the encoded audio data to the data generation unit 1A. The data generator 1A multiplexes the encoded image data and the encoded audio data to generate moving image stream data. A schematic diagram (example) of the stream data is shown in FIG. In FIG. 6, stream data is shown in order from the top in the order of images that have been subjected to decoding / decompression processing. The upper three images show the navigation screen, and are compressed and encoded as I, B, and P pictures in order from the top, and the audio data is multiplexed with data corresponding to the external audio data of the encoded audio data. . The lower three images show photographed image data, and are compressed and encoded as I, B, and P pictures in order from the fourth image from the top, and the audio data corresponds to the audio data of the encoded audio data. Multiplexed using data. In FIG. 6, the upper three images are the same image, but different navigation images may be used. The stream data generated by the data generation unit 1A is output to the data recording unit 1B, and the data recording unit 1B records the stream data on the recording medium 1C.

  With the above processing, the same effect as in the first embodiment can be obtained.

  In this embodiment, the navigation image data is data for a predetermined time set in advance by a user or the like. However, the navigation 2 and the video camera 1 are provided with communication means such as USB, Bluetooth, and wireless LAN, and the navigation is performed. When the audio is over, a navigation audio end notification may be output from the navigation 2 to the external audio input unit 181 of the video camera 1, and the external audio input unit 181 may determine the end of data based on the end notification.

  A third embodiment will be described. The difference from the first embodiment is that the video camera 1 and the navigation 2 are connected by the AV cable 3 in the first embodiment, but the video camera 1 has a C-side wireless communication means 32 and the navigation 2 has an N-side wireless communication means. 31 is connected. The C-side wireless communication means 32 and the N-side wireless communication means 31 communicate with each other and exchange data using a wireless communication protocol such as infrared communication, wireless LAN, or Bluetooth (registered trademark).

  The shooting procedure in the image recording / playback system of the third embodiment is that the navigation image is transferred from the navigation image output unit 211 to the external image input unit 14 by the AV cable in the first embodiment. The second embodiment is the same as the first embodiment except that the navigation image is exchanged by the C-side wireless communication unit 32.

  With the above processing, the same effect as in the first embodiment can be obtained.

DESCRIPTION OF SYMBOLS 1 Video camera 11 Operation input part 12 Control part 13 Captured image input part 14 External image input part 15 Image input mediation part 16 Image memory part 18 Voice input part 181 External voice input part 182 Voice input mediation part 183 Voice memory part 1D Voice activation Part 2 Navigation 21 Navigation body part 211 Navigation image output part 212 Navigation voice output part 213 Navigation voice start part 22 Navigation display part 3 AV cable

Claims (12)

An imaging device,
An input means for inputting a plurality of image data taken by the user;
The position information where each of the plurality of image data is photographed and the plurality of image data are recorded in association with each other, and map image data corresponding to the position where each of the plurality of image data is photographed, Recording means for recording image data created as one moving image in association with the plurality of image data;
Bei to give a,
The first image of the image data created as the one moving image is the map image,
Imaging device. An imaging device,
An input means for inputting a plurality of image data taken by the user;
The map is a map corresponding to the position at which each of the plurality of image data is recorded, and the position information at which each of the plurality of image data is captured and the plurality of image data are recorded in association with each other. Map image data in which thumbnails of each of the plurality of image data are displayed in association with corresponding positions on the map based on position information associated with each of the image data, and the plurality of image data are associated with each other. Recording means for recording image data created as one moving image;
Bei to give a,
The first image of the image data created as the one moving image is the map image,
Imaging device. The imaging apparatus according to claim 2,
The image data of the map corresponds to the thumbnail of each of the plurality of image data on the movement path of the map indicating the movement path of the imaging device based on position information associated with each of the plurality of image data. Map image data displayed in association with the location.
Imaging device. It is an imaging device as described in any one of Claims 1-3 ,
Furthermore, the wireless communication means for wirelessly transferring the image data created as the one moving image to a server on the network,
Imaging device. A method of processing a captured image,
Receiving input of a plurality of image data photographed by the imaging device;
The position information where each of the plurality of image data is photographed and the plurality of image data are recorded in association with each other, and map image data corresponding to the position where each of the plurality of image data is photographed, Recording image data created as one moving image in association with the plurality of image data;
Bei to give a,
The first image of the image data created as the one moving image is the map image,
Processing method of captured image. A method of processing a captured image,
Receiving input of a plurality of image data photographed by the imaging device;
The map is a map corresponding to the position at which each of the plurality of image data is recorded, and the position information at which each of the plurality of image data is captured and the plurality of image data are recorded in association with each other. Map image data in which thumbnails of each of the plurality of image data are displayed in association with corresponding positions on the map based on position information associated with each of the image data, and the plurality of image data are associated with each other. Recording image data created as one moving image;
Bei to give a,
The first image of the image data created as the one moving image is the map image,
Processing method of captured image. A captured image processing method according to claim 6 , comprising:
The image data of the map corresponds to the thumbnail of each of the plurality of image data on the movement path of the map indicating the movement path of the imaging device based on position information associated with each of the plurality of image data. Map image data displayed in association with the location.
Processing method of captured image. A method of processing captured image according to any one of claims 5-7,
Furthermore, the method includes the step of wirelessly transferring the image data created as the one moving image to a server on the network.
Processing method of captured image. An imaging system,
An output device for outputting the acquired map information;
An imaging device for acquiring information of the map output from the output device,
The imaging device
An input means for inputting a plurality of image data taken by the user;
The position information where each of the plurality of image data is photographed and the plurality of image data are recorded in association with each other, and map image data corresponding to the position where each of the plurality of image data is photographed, Recording means for recording image data created as one moving image in association with the plurality of image data;
Bei to give a,
The first image of the image data created as the one moving image is the map image,
Imaging system. An imaging system,
An output device for outputting the acquired map information;
An imaging device for acquiring information of the map output from the output device,
The imaging device
An input means for inputting a plurality of image data taken by the user;
The map is a map corresponding to the position at which each of the plurality of image data is recorded, and the position information at which each of the plurality of image data is captured and the plurality of image data are recorded in association with each other. Map image data in which thumbnails of each of the plurality of image data are displayed in association with corresponding positions on the map based on position information associated with each of the image data, and the plurality of image data are associated with each other. Recording means for recording image data created as one moving image;
Bei to give a,
The first image of the image data created as the one moving image is the map image,
Imaging system. The imaging system according to claim 10 , wherein
The image data of the map corresponds to the thumbnail of each of the plurality of image data on the movement path of the map indicating the movement path of the imaging device based on position information associated with each of the plurality of image data. Map image data displayed in association with the location.
Imaging system. The imaging system according to any one of claims 9 to 11 ,
The imaging apparatus further includes wireless communication means for wirelessly transferring the image data created as the one moving image to a server on the network.
Imaging system.

Images