WO2006082886A1

WO2006082886A1 - Image editing device, image editing method, image editing program and computer readable recording medium

Info

Publication number: WO2006082886A1
Application number: PCT/JP2006/301757
Authority: WO
Inventors: Takeshi Sato; Kenichiro Yano; Koji Koga; Goro Kobayashi
Original assignee: Pioneer Corporation
Priority date: 2005-02-03
Filing date: 2006-02-02
Publication date: 2006-08-10
Also published as: JP4516111B2; CN100533477C; US20090027399A1; EP1845491A1; JPWO2006082886A1; CN101111863A; EP1845491A4

Abstract

An image editing device is provided with an input section (103) for receiving input of image data including information relating to date and time; an acquiring section (104) for acquiring information relating to a route and a time by which and at which a mobile object moved; and an associating section (105) for associating the image data with map information based on information relating to the date and the time of the image data received by the input section (103) and the information relating to the route and the time acquired by the acquiring section (104). The image editing device automatically edits the image data in time series or in route order.

Description

Image editing apparatus, image editing method, image editing program, and computer-readable recording medium

Technical field

The present invention relates to an image editing device that edits image data such as a photograph, an image editing method, an image editing program, and a computer-readable recording medium. However, use of the present invention is not limited to the above-described image editing apparatus, image editing method, image editing program, and computer-readable recording medium.

Background art

[0002] With the recent spread of digital still cameras (DSC) and digital video cameras (Digital Video Cameras: DVC), so-called electronic albums using still image data and moving image data. There is provided an electronic album creating apparatus that makes it possible to create and publish easily on a web page or the like. In such an electronic album creating apparatus, an electronic album is created as follows.

[0003] Specifically, for example, a server connected to the Internet is provided with program software that edits digital image data to create an electronic album, and the server includes image data captured by a digital camera, The photographing time data of the image, the position data acquired by the mopile terminal, and the time data when the position data was acquired can be received. Create a Lubam (see, for example, Patent Document 1 below) ₀

[0004] Patent Document 1: JP 2002-183742 A

Disclosure of the invention

Problems to be solved by the invention

However, in the electronic album creating apparatus described in Patent Document 1, the electronic album is created by associating, for example, the photographing time and photographing place of the image data with the program software in the server. As an example, there is a problem that the establishment of a connection environment with the system is indispensable and the entire device configuration becomes complicated. Means for solving the problem

[0006] The image editing apparatus according to the invention of claim 1 includes an input unit that receives input of image data including information related to date and time, an acquisition unit that acquires information about a route and time when the moving body has moved, and the input unit. And an associating means for associating the image data with map information based on the information on the date and time of the image data received by the information and the information on the route and the time acquired by the acquiring means. .

[0007] Further, the image editing method according to the invention of claim 10 includes an input step of inputting image data including information on date and time, an acquisition step of acquiring information on a route and time of movement of the moving body, and the input An association step of associating the image data with map information based on the information on the date and time of the image data input in the step and the information on the route and time acquired in the acquisition step. It is a feature.

[0008] An image editing program according to claim 11 causes a computer to execute the image editing method according to claim 10.

[0009] A computer-readable recording medium according to the invention of claim 12 records the image editing program according to claim 11.

Brief Description of Drawings

FIG. 1 is a block diagram showing an example of a functional configuration of an image editing apparatus that is useful for an embodiment.

FIG. 2 is a flowchart showing an example of an image editing process procedure of the image editing apparatus according to the embodiment.

FIG. 3 is an explanatory diagram showing an example of the inside of a vehicle equipped with an image editing apparatus similar to the embodiment.

[FIG. 4] FIG. 4 is a block diagram illustrating an example of a hardware configuration of an image editing apparatus that is effective in the embodiment.

FIG. 5 is a block diagram illustrating an example of an internal configuration of an image editing unit in the image editing apparatus according to the embodiment.

O

[FIG. 6] FIG. 6 shows an example of an internal configuration of an audio reproduction unit in an image editing apparatus that is effective in the embodiment.

It is a block diagram.

1—

FIG. 7 is a flowchart showing an example of an image editing process procedure of the image editing apparatus according to the embodiment.

[FIG. 8] FIG. 8 is a flowchart illustrating an example of another procedure for associating audio data in the image editing process of the image editing apparatus according to the embodiment.

FIG. 9 is a flowchart illustrating an example of another procedure for associating audio data in the image editing process of the image editing apparatus according to the embodiment.

FIG. 10 is an explanatory diagram of an example of an image data distribution process procedure in the image editing process of the image editing apparatus according to the embodiment.

FIG. 11 is an explanatory diagram of a specific processing example of the image editing processing of the image editing apparatus according to the embodiment.

Explanation of symbols

102 Sound collector

103 Input section

104 Acquisition Department

105 Association

106 Display

107 Detector

108 Control unit

109, 414 Audio playback section

310 Image editing device

412 Image Editor

510 Image editing processor

610 Audio playback processor

BEST MODE FOR CARRYING OUT THE INVENTION

With reference to the accompanying drawings, an image editing apparatus, an image editing method, and A preferred embodiment of an image editing program and a computer-readable recording medium recording the program will be described in detail.

[0013] (Embodiment)

(Functional configuration of image editing device)

First, the contents of an image editing apparatus that is relevant to the embodiment of the present invention will be described. FIG. 1 is a block diagram showing an example of a functional configuration of an image editing apparatus that is useful in an embodiment of the present invention. In FIG. 1, an image editing apparatus is mounted on a moving body such as a vehicle (including a four-wheeled vehicle and a two-wheeled vehicle), for example, and includes a photographing unit 101, a sound collecting unit 102, an input unit 103, and an acquiring unit 104. The association unit 105, the display unit 106, the detection unit 107, the control unit 108, and the audio reproduction unit 109 are included.

[0014] The imaging unit 101 captures an image. The image photographed by the photographing unit 101 may be an image taken inside or outside the vehicle. The photographing unit 101 is configured to be integral with or removable from the image editing apparatus. The sound collection unit 102 collects sound inside the vehicle, for example. Examples of the sound collected by the sound collection unit 102 include those collected from the sound field inside the vehicle.

The input unit 103 accepts input of image data including information related to date and time (for example, time stamp data). The input unit 103 also accepts input of image data of an image captured by the imaging unit 101 and audio data of the sound collected by the sound collection unit 102. The acquisition unit 104 acquires information regarding the route and time of travel of the vehicle. The acquisition unit 104 acquires behavior information regarding the behavior of the vehicle. Specifically, this behavior information is information indicating the moving or stopping state of the vehicle. For example, information on vehicle speed (speed information, acceleration information, angular velocity information, etc.), inclination angle information, lateral G (Gmvity ) Information and current location information that includes at least one of! /

The associating unit 105 uses the image data based on the information on the date and time of the image data received by the input unit 103, the vehicle route and time information acquired by the acquiring unit 104, and the behavior information. Is associated with map information. By the association in the associating unit 105, when and where the image data captured by the imaging unit 101 is captured. It is determined whether it is

Display unit 106 displays the image data associated by association unit 105.

For example, the display unit 106 may display the image data arranged in the order of time series in which the image data was taken or the order in which the vehicle passed. The detection unit 107 calculates the feature amount of the image image included in the image data of the image captured by the image capturing unit 101 and the feature amount of the audio parameter included in the sound data of the sound collected by the sound collection unit 102. To detect.

[0018] Specifically, the feature amount of the image image includes, for example, the feature amount of the face image of a person included in the image image of the image data. Specific examples of the feature amount of the voice parameter include a feature amount of a volume component (volume level), a time component (sounding duration), and a frequency component (frequency level).

The control unit 108 controls the photographing unit 101 based on the feature amount of the image image and the feature amount of the sound meter detected by the detection unit 107. Further, the control unit 108 controls the photographing unit 101 to take an image when the feature amount detected by the detecting unit 107 changes.

[0020] The audio reproduction unit 109 reproduces audio data. Further, when displaying the image data on the display unit 106, the audio reproduction unit 109 reproduces based on, for example, the feature amount detected by the detection unit 107 and the behavior information acquired by the acquisition unit 104. Select audio data. Examples of the sound reproduced by the sound reproducing unit 109 include music such as music and sound effects.

[0021] (Image Editing Processing Procedure of Image Editing Device)

Next, an image editing process procedure of the image editing apparatus according to the embodiment of the present invention will be described. FIG. 2 is a flowchart showing an example of an image editing process procedure of the image editing apparatus according to the embodiment of the present invention.

In the flowchart of FIG. 2, first, for example, an image of a person or a landscape is input from one or a plurality of photographing units 101 (see FIG. 1, the same applies hereinafter) by the input unit 103 (see FIG. 1, the same applies hereinafter). And image data including date and time information is input (step S 201). Next, the acquisition unit 104 (see FIG. 1, the same applies hereinafter) acquires information regarding the route and time the vehicle has traveled (step S 202). [0023] Then, based on the information on the date and time of the image data input in step S201 and the information on the route and time acquired in step S202 by the associating unit 105 (see FIG. 1, the same applies hereinafter), the image The data is associated with the map information (step S203). After associating the image data with the map information in this manner, the image data is displayed by the display unit 106 (see FIG. 1, the same applies hereinafter) (step S204). As a result, the image editing process according to this flowchart is completed.

[0024] Although illustration is omitted, the characteristics of the image image detected by the detection unit 107 (see FIG. 1, the same applies hereinafter) during the display processing of the image data by the display unit 106 in step S204. Volume and the parameter value of the voice parameter and the behavior information acquired by the acquisition unit 104, the audio playback unit 109 (see FIG. 1, the same applies hereinafter) selects the audio data to be played back, and the selected audio data May be played back. In addition, when the control unit 108 (see FIG. 1, the same applies hereinafter) changes the feature amount detected by the detection unit 107, the imaging unit 101 may be controlled to capture an image.

As described above, according to the image editing apparatus according to the embodiment of the present invention, the input image data is based on the information on the date and time of the image data and the information on the acquired route and time. Can be associated with map information without going through a server. For this reason, image data obtained during driving of the vehicle can be automatically linked to the passing point and passing time of the vehicle without complicating the configuration of the device, and edited in chronological order or passing route order. As a result, it is possible to reduce complicated labor and cost for image editing.

[0026] Next, examples that are useful for the embodiments of the present invention will be described in detail. Here, a case where the image editing apparatus according to this embodiment is applied to an in-vehicle navigation apparatus will be described as an example.

Example

[0027] (Description of inside of vehicle equipped with image editing device)

First, the interior of a vehicle equipped with an image editing apparatus that is useful in the embodiment of the present invention will be described. FIG. 3 is an explanatory view showing an example of the inside of a vehicle in which an image editing apparatus that is effective in the embodiment of the present invention is mounted. In FIG. 3, for example, the driver's seat 311 and the passenger seat A monitor 302a as the display unit 106 and a speaker 304 as an audio output device of the audio reproduction unit 109 in FIG. Further, a camera 305 as the photographing unit 101 and a microphone 306 as the sound collecting unit 102 in FIG.

[0028] The passenger seat 312 is provided with a monitor 302b as the display unit 106 toward the passenger on the rear seat 313. The image editing apparatus 310 (310a, 310b) includes these monitors 302 (302a, 302b), a speaker 304, a camera 305, and a microphone 306. It should be noted that the camera 305 and the microphone 306ί and the image editing device 310 (310a, 310b) may be mounted respectively. The image editing device 310 (310a, 310b) may be provided with a structure that can be attached to and detached from the vehicle.

[0029] (Hardware configuration of image editing apparatus)

Next, the hardware configuration of the image editing apparatus that is useful in the embodiment of the present invention will be described. FIG. 4 is a block diagram showing an example of a hardware configuration of an image editing apparatus that is useful in the embodiment of the present invention.

In FIG. 4, the image editing device 310 is detachably mounted on the vehicle as described above, and includes a control unit 400, a user operation unit (remote control and touch panel) 401, a display unit (monitor) 402, and the like. , Position acquisition unit (GPS, sensor) 403, recording medium 404, recording medium decoding unit 405, guidance sound output unit 406, communication unit 407, route search unit 408, route guidance unit 409, guidance Sound generation unit 410, speaker 411, image editing unit 412, image input / output IZF 413, audio reproduction unit 414, audio output unit 415, shooting unit 416, sound collection unit 417, audio input iZ F418, It is the composition which includes.

[0031] The control unit 400 controls the entire image editing apparatus 310, for example, and executes various arithmetic processes according to a control program, thereby comprehensively controlling each unit included in the image editing apparatus 310. The control unit 400 includes, for example, a CPU (Central Processing Unit) that executes predetermined arithmetic processing, a ROM (Read Only Memory) that stores various control programs, and a RAM (Random Access Memory) that functions as a work area for the CPU. It can be realized by a microcomputer composed of the above. [0032] In addition, the control unit 400, for example, at the time of route guidance of the vehicle, information on the current position of the vehicle (current position information) acquired by the position acquisition unit 403 and the recording medium 404 via the recording medium decoding unit 405 Based on the map information obtained in this way, the position on the map where the vehicle is traveling is calculated, and the calculation result is output to the display unit 402. The control unit 400 inputs and outputs information related to route guidance among the route search unit 408, the route guidance unit 409, and the guidance sound generation unit 410, and displays information obtained as a result of the display unit 402. And output to the guide sound output unit 406.

The user operation unit 401 outputs information input by the user such as characters, numerical values, and various instructions to the control unit 400. As the configuration of the user operation unit 401, various known forms such as a push button switch that detects physical pressing Z non-pressing, a touch panel, a keyboard, and a joystick can be employed. The user operation unit 401 may be configured to perform an input operation by sound using a microphone or the like that inputs sound of an external force such as the sound collection unit 417 described later.

[0034] The user operation unit 401 may be provided integrally with the image editing apparatus 310, or may be configured to be able to operate a positional force separated from the image editing apparatus 310, such as a remote controller. . Further, the user operation unit 401 may be configured in any one of the various forms described above, or may be configured in a plurality of forms. The user inputs information by appropriately performing an input operation according to the form of the user operation unit 401.

[0035] Information input by an input operation of the user operation unit 401 includes, for example, destination information regarding navigation. Specifically, for example, when the image editing apparatus 310 is provided in a vehicle or the like, a point set as a destination target by a person on the vehicle is set. Further, as information input to the user operation unit 401, for example, with respect to image editing, information on the display format in an electronic album of image data input to the image editing unit 412 from an image input / output IZF 413 to be described later can be cited. Specifically, for example, the display format of the electronic album desired by the person on board the vehicle is set.

Here, for example, when a touch panel is adopted as the form of the user operation unit 401, the touch panel is used by being stacked on the display screen side of the display unit 402. In this case, display By managing the display timing in the unit 402, the operation timing for the touch panel (user operation unit 401) and its position coordinates, the input information by the input operation is recognized. By adopting a touch panel stacked on the display unit 402 as a form of the user operation unit 401, it is possible to input a large amount of information without enlarging the form of the user operation unit 401. As this touch panel, various known touch panels such as a resistance film type and a pressure sensitive type can be adopted.

[0037] Display unit 402 includes, for example, a CRT (Cathode Ray Tube), a TFT liquid crystal display, an organic EL display, a plasma display, and the like. Specifically, the display unit 402 can be configured by, for example, a video IZF or a video display device connected to the video IZF, not shown. Specifically, the video IZF is a graphic controller that controls the entire display device, a buffer memory such as VRAM (Video RAM) that temporarily stores image information that can be displayed immediately, and a graphic controller. Based on the image information output from the control unit, it is composed of a control IC that controls display of the display device, a GPU (Graphics Processing Unit), and the like. The display unit 402 displays icons, cursors, menus, windows, or various information such as characters and images. Further, the display unit 402 displays image data edited by an image editing unit 412 described later.

[0038] The position acquisition unit 403 acquires current position information (latitude and longitude information) of the vehicle on which the image editing device 310 is mounted, for example, by receiving radio waves of artificial satellite power. Here, the current position information is information for receiving a radio wave from an artificial satellite and obtaining a geometric position with respect to the artificial satellite, and can be measured anywhere on the earth. The position acquisition unit 403 includes a GPS antenna (not shown). Here, the GPS (Global Positioning System) is a system that accurately obtains the position on the ground by receiving radio waves of four or more artificial satellite forces. Here, since it is a well-known technique, description about GPS is abbreviate | omitted. The position acquisition unit 403 can be configured by, for example, a tuner that demodulates radio waves that have also received artificial satellite force, an arithmetic circuit that calculates the current position based on the demodulated information, and the like.

[0039] It should be noted that the radio wave from the artificial satellite is 1. 57542GHz carrier, CZA (Coarse and Access) code and navigation messages are carried out using LI radio waves. As a result, the current position (latitude and longitude) of the vehicle on which the image editing apparatus 310 is mounted is detected. In detecting the current position of the vehicle, information collected by various sensors such as a vehicle speed sensor and a gyro sensor may be taken into consideration. The vehicle speed sensor detects the vehicle speed from the output shaft of the transmission of the vehicle on which the image editing device 310 is mounted.

[0040] In addition, when detecting the current position of the vehicle, information collected by various sensors such as an angular velocity sensor, a travel distance sensor, an inclination angle sensor, and a lateral G (Gravity) sensor may be taken into consideration. The angular velocity sensor detects the angular velocity when the vehicle rotates and outputs angular velocity information and relative orientation information. The mileage sensor calculates the number of pulses per one rotation of the wheel by counting the number of pulses of a pulse signal output with the rotation of the wheel in a given period, and the mileage based on the number of pulses per one rotation Output information. The inclination angle sensor detects the inclination angle of the road surface and outputs inclination angle information. The lateral G sensor detects lateral G, which is an outward force (gravity) generated by centrifugal force during vehicle cornering, and outputs lateral G information. The vehicle current position information acquired by the position acquisition unit 403 and the information detected by the vehicle speed sensor, gyro sensor, angular velocity sensor, mileage sensor, inclination angle sensor, and lateral G sensor are the vehicle behavior. Is output to the control unit 400 as behavior information regarding

The recording medium 404 records various control programs and various information in a state that can be read by a computer. The recording medium 404 accepts writing of information by the recording medium decoding unit 405 and records the written information in a nonvolatile manner. The recording medium 404 can be realized by HD (Hard Disk), for example. The recording medium 404 is not limited to HD. It can be attached to and detached from the recording medium decoding unit 405 such as DVD (Digital Versatile Disk) and CD (Compact Disk) instead of HD or in addition to HD. A medium having portability may be used as the recording medium 404. The recording medium 404 is not limited to the DVD and CD, but is attached to and detached from the recording medium decoding unit 405 such as a CD-ROM (CD-R, CD-RW), MO (Magno-Optical disk), and memory card. It is possible to use a portable medium that is portable. [0042] Note that the recording medium 404 stores an image editing program, a navigation program, image data, map information, and the like that realize the present invention. Here, the image data refers to a two-dimensional array value representing an image image related to a person or a landscape, for example. The map information includes background information representing features (features) such as buildings, rivers, and the ground surface, and road shape information representing road shapes. Rendered in 3D or 3D.

[0043] The background information includes background shape information representing the shape of the background and background type information representing the type of the background. The background shape information includes information indicating, for example, representative points of features, polylines, polygons, coordinates of features, and the like. The background type information includes, for example, text information indicating the name, address, telephone number, and the like of the feature, and type information indicating the type of the feature such as a building 'river'.

[0044] The road shape information is information relating to a road network having a plurality of nodes and links. The node is information indicating an intersection where a plurality of roads intersect, such as a three-way “crossroad” and a five-way. A link is information indicating a road connecting nodes. Some links have shape interpolation points that allow the expression of curved roads. Road shape information has traffic condition information. The traffic condition information is information indicating the characteristics of the intersection, the length (distance) of each link, vehicle width, traveling direction, traffic prohibition, road type, and the like.

[0045] The characteristics of this intersection include, for example, complex intersections such as three- and five-way intersections, intersections where roads branch at shallower angles, intersections around destinations, junctions on expressways, junctions, and route deviation rates. For example, high intersections. The route deviation rate can be calculated by, for example, the past driving history force. Examples of road types include highway, toll road, and general road.

In the present embodiment, for example, the power for recording image data and map information on the recording medium 404 is not limited to this. The image data and the map information may be provided outside the image editing apparatus 310, not limited to those recorded only with those provided integrally with the hardware of the image editing apparatus 310. In that case, the image editing apparatus 310 acquires image data via the network through the communication unit 407, for example. In addition, the image editing device 310 uses the communication unit 407, for example, to map information via the network. To get. The image data and map information acquired in this way may be stored in the RAM of the control unit 400, for example!

[0047] The recording medium decoding unit 405 controls reading and writing of information on the recording medium 404. For example, when HD is used as the recording medium 404, the recording medium decoding unit 405 is an HDD (Hard Disk Drive). Similarly, when DV D or CD (including CD-R, CD-RW) is used as the recording medium 404, the recording medium decoding unit 405 is a DVD drive or a CD drive. When a CD-ROM (CD-R, CD-RW), MO, memory card, etc. is used as the writable and detachable recording medium 404, information can be written to various recording media and various recording media. A dedicated drive device or the like that can read the information stored in the recording medium may be used as the recording medium decoding unit 405 as appropriate.

[0048] The guidance sound output unit 406 reproduces navigation guidance sound by controlling output to the connected speaker 411. There may be one or more speakers 411. Specifically, the guide sound output unit 406 can be realized by an audio IZF (not shown) connected to the audio output speaker 411. More specifically, the audio IZ F is, for example, a DZ A converter that performs DZA conversion of digital audio data, an Ό / A converter power, an amplifier that amplifies the analog audio signal that is output, and an AZD conversion of the analog audio signal. It can be configured with AZD converter and force.

[0049] The communication unit 407 performs communication with other image editing apparatuses. The communication unit 407 of this embodiment may be a communication module that communicates with a communication server (not shown) via a base station (not shown), such as a mobile phone. It may be a communication module that performs direct wireless communication with the device. Here, the wireless communication is communication performed using radio waves or infrared rays' ultrasonic waves without using a wire line as a communication medium. Standards that enable wireless communication include various technologies such as wireless LAN, IrDA (Infrared Data Association), HomeRF (Home Radio Frequency), BlueTooth, etc. In this embodiment, various known wireless communication technologies are used. Can be used. Note that a wireless LAN can be used as a preferable example in terms of information transfer speed and the like. [0050] Here, the communication unit 407 may receive road traffic information such as traffic jams and traffic regulations on a regular basis (even if irregular). The reception of road traffic information by the communication unit 407 may be performed at the timing when the VICS (Vehicle Information and Communication system) center ~~ road traffic information is delivered, or the road traffic information is periodically sent to the VICS center. You may do it by requesting. The communication unit 407 can be realized as, for example, an AM ZFM tuner, a TV tuner, a VICSZ beacon receiver, and other communication devices.

[0051] It should be noted that rvicsj, which is a well-known technology and omits detailed explanation, is a real-time transmission of road traffic information such as traffic jams and traffic regulations edited and processed at the VICS center. It is an information communication system that displays characters on the in-vehicle device. As a method of transmitting road traffic information (VICS information) edited and processed at this VICS center to the navigation device, there is a method of using “beacon” and “FM multiplex broadcasting” installed on each road. “Beacons” include “electric beacons” mainly used on expressways and “optical beacons” used on major general roads. When “FM multiplex broadcasting” is used, road traffic information in a wide area can be received. When using VISICON, receive necessary road traffic information at the place where the vehicle is located, such as detailed information on the most recent road based on the vehicle (vehicle) position! It becomes possible. When the communication method with another image editing apparatus is different from the communication method for receiving image data and road traffic information, the communication unit 407 includes a plurality of communication means corresponding to each. Get ready.

[0052] The route search unit 408 calculates an optimal route from the current position to the destination based on the current position information of the vehicle acquired by the position acquisition unit 403 and the destination information input by the user. To do. The route guidance unit 409 includes information on the guidance route searched by the route search unit 408 or route information received by the communication unit 407, current position information acquired by the position acquisition unit 403, and a recording medium 404 Based on the map information obtained via the decoding unit 405, real-time route guidance information is generated. The route guidance information generated by the route guidance unit 409 is output to the display unit 402 via the control unit 400. [0053] Guide sound generation section 410 generates tone and voice information corresponding to the pattern. In other words, based on the route guidance information generated by the route guidance unit 409, the virtual sound source corresponding to the guidance point is set and the voice guidance information is generated, and the guidance sound is transmitted via the control unit 400. Output to the output unit 406.

The speaker 411 reproduces (outputs) a navigation guidance sound output from the guidance sound output unit 406 and a sound output from the audio output unit 415 described later. For example, a headphone or the like may be provided for the speaker power 411 so that the sound output sound of the guidance sound or sound is appropriately changed so that the sound field of the guidance sound or sound output from the entire vehicle interior is not obtained. Oh ,.

The image editing unit 412 performs image editing processing such as image data acquired through the image input / output IZF 413 from the photographing unit 416 and the communication unit 407, which will be described later, and image data recorded in the recording medium 404. Specifically, the image editing unit 412 is configured by a GPU, for example. The image editing unit 412 performs processing for creating electronic album (hereinafter referred to as “album”) data using image data in accordance with a control command from the control unit 400. Here, the album data refers to image data taken by the photographing unit 416 such as a digital still camera (DSC) or a digital video camera (DVC) on the display screen of the display unit 402. Digital data that can be viewed like a photo album or viewed and edited on a personal computer.

Image Input / Output IZF 413 inputs / outputs image data input / output to / from image editing unit 412 from the outside. Image input / output IZF413 uses USB (Universal Serial Bus), IEEE 1394 to transfer image data from recording medium 404 that stores image data captured by DSC or DVC, for example, or image data stored in DSC or DVC, for example. (Institut e of Electrical and Electronic Engineers 1394) and image data input from the communication unit 407 by communication such as infrared rays are output to the image editing unit 412 and the image data output from the image editing unit 412 is output to the recording medium 404. And output to the communication unit 407. Note that the image input / output IZF 413 preferably has a controller function for controlling read Z write of the recording medium 404 when inputting / outputting image data to / from the recording medium 404. Further, the image input / output IZF 413 may have a function of a communication controller that controls communication with the communication unit 407 when inputting / outputting image data to / from the communication unit 407. [0057] The audio reproduction unit 414 selects audio data obtained from the recording medium 404 through the recording medium decoding unit 405, audio data obtained from the communication unit 407 through the control unit 400, and the like, and reproduces the selected audio data. To do. The audio playback unit 414 plays back audio data stored in a storage device such as an audio database (hereinafter referred to as “audio DB”) 611 (see FIG. 6) described later. Audio data to be played back includes audio data such as music music and sound effects. In addition, when the image editing device 310 includes an AM / FM tuner or a TV tuner, the audio reproduction unit 414 may be configured to reproduce radio audio, for example.

The audio output unit 415 controls the output of the audio output from the speaker 411 based on the audio data selected and reproduced by the audio reproduction unit 414. Specifically, for example, sound volume adjustment and equalizing processing is performed to control the sound output state. The audio output control by the audio output unit 415 is performed by, for example, an input operation from the user operation unit 401 or a control by the control unit 400.

[0059] The imaging unit 416 includes the camera 305 mounted on the vehicle in FIG. 3 and an external imaging device such as the above-described DSC and DVC, and includes a photoelectric conversion element such as C-MOS or CCD. Take images inside or outside the vehicle. The photographing unit 416 is connected to the image editing device 310 by wire or wirelessly, and takes a picture of a person who gets on the vehicle, for example, according to a photographing command from the control unit 400. Image data of an image photographed by the photographing unit 416 is output to the image editing unit 412 via the image input / output IZF 413.

[0060] The sound collection unit 417 is configured by the microphone 306 or the like mounted on the vehicle in FIG. 3, for example, collecting sound such as the utterance sound of a person boarding the vehicle from the sound field inside the vehicle. To do. The voice input IZF 418 converts the voice collected by the sound collection unit 417 into digital voice data and outputs it to the control unit 400. More specifically, the voice input IZF 418 can also be configured with power such as an AZD converter that converts input analog voice data into digital voice data. In addition, the audio input IZF 418 may include a filter circuit that filters digital audio data, an amplification circuit that amplifies analog audio data, and the like. Here, the control unit 400 described above is based on image data captured by the imaging unit 416 and output from the image editing unit 412, or audio data collected by the sound collection unit 417 and output from the audio input IZF 418. To determine the atmosphere inside the vehicle. Specifically, the determination of the atmosphere inside the vehicle is made by detecting changes in features such as the facial expression and voice of the person on board. For this reason, the control unit 400 may be configured to have a DSP (Digital Signal Processor) function, for example!

Note that, specifically, the photographing unit 101 in FIG. 1 realizes its function by, for example, the photographing unit 416, and the sound collecting unit 102 realizes its function by, for example, the sound collecting unit 417. Further, specifically, the input unit 103 in FIG. 1 realizes its function by, for example, an image input / output IZF 413 and an audio input IZF 418, and the acquisition unit 104 realizes its function by, for example, a position acquisition unit 403.

[0063] In addition, the functions of the associating unit 105, the detecting unit 107, and the control unit 108 in FIG. 1 are specifically realized by, for example, the control unit 400 and the image editing unit 412. Furthermore, specifically, the display unit 106 in FIG. 1 realizes its function by, for example, the display unit 402, and the audio reproduction unit 109 has its function by, for example, the audio reproduction unit 414, the audio output unit 415, and the speech force 411. Realize.

Here, the internal configuration of the image editing unit 412 and the audio reproduction unit 414 will be described. FIG. 5 is a block diagram showing an example of the internal configuration of the image editing unit in the image editing apparatus according to the embodiment of the present invention. FIG. 6 is a block diagram showing an example of the internal configuration of the audio playback unit in the image editing apparatus according to the embodiment of the present invention.

In FIG. 5, an image editing unit 412 includes an image editing processing unit 510, a display control unit 511, an image recognition unit 512, an image storage unit 513, a person recognition unit 514, "Person DB") 515. The image editing processing unit 510 is connected to the image capturing unit 416 (see FIG. 4; the same applies hereinafter) through the image input / output IZF 413, the image data input to the image editing unit 412 from the outside, and the recording medium decoding unit 405 (see FIG. 4; the same applies to the following). ) And the control unit 400 (see FIG. 4, the same applies hereinafter), and performs image editing processing of image data input to the image editing unit 412 from the recording medium 404 (refer to FIG. 4 and the same applies hereinafter). The image editing processing unit 510 reads out image data stored in an image storage unit 513 to be described later. Perform image editing processing. The contents of the image editing process include, for example, editing image data into album data.

The display control unit 511 performs control for displaying the image data output from the image editing processing unit 510 as an album form on the display screen of the display unit 402. The image recognition unit 512 recognizes what image image is included in the image data based on the image data input to the image editing processing unit 510. The image storage unit 513 stores the image data input to the image editing processing unit 510.

[0067] The person recognizing unit 514 is stored in the person DB 515 when the image image in the image data input to the image editing processing unit 510 includes an image related to a person. The image related to the person is read out, and the person represented by the image is recognized. Specifically, the recognition process is performed, for example, by face authentication based on a human face image. Since face authentication is a known technique, description thereof is omitted here. The person DB 515 stores image data including image images of persons on the vehicle, and personal identification data such as the age and sex of these persons.

[0068] Note that the image editing unit 412 detects the image data of the image data recognized by the image recognition unit 512, the feature amount of the image image related to the person recognized by the person recognition unit 514, and sends it to the control unit 400. Output. The feature quantities of these image images are detected, for example, color tone data of the image image or emotion parameter power of the human face image. Specifically, the color tone data indicates which color tone of the entire image is close to red, blue, green, etc., and the emotion parameter indicates which facial expression of the person's face image is close (feeling emotional). Show.

On the other hand, in FIG. 6, an audio playback unit 414 includes an audio playback processing unit 610, an audio data base (hereinafter referred to as “audio DB”) 611, a music selection history database (hereinafter referred to as “music selection history DB”). 612). The audio reproduction processing unit 610 performs selection / reproduction processing of audio data input to the audio reproduction unit 414 and audio data stored in the audio DB 611 !. Also, the audio reproduction processing unit 610 performs audio data selection 'reproduction processing in association with image data in album data created by, for example, the image editing unit 412 (see FIG. 4, the same applies hereinafter). The audio data association here is the image in the album data. It may be performed based on feature quantities such as time stamp data included in the image data, color tone data of the image image, and human face image image.

The audio DB 611 stores audio data to be reproduced by the audio reproducing unit 414. The sound data stored in the sound DB 611 may be the sound data input to the sound reproducing unit 414 even if the recording medium 404 (see FIG. 4; the same applies hereinafter) or the communication unit 40 7 (see FIG. 4; the same applies hereinafter). Alternatively, the audio data provided in the bullying image editing device 310 may be used. The music selection history DB 612 stores information related to the music playback history and music selection history when the audio data played back by the audio playback unit 414 is music data. This music selection history DB 612 stores, for example, information related to the reproduction history of music played during driving and information related to music selection history when the image editing apparatus 310 is mounted on a vehicle.

[0071] (Image Editing Processing Procedure of Image Editing Device)

Next, an image editing process procedure of the image editing apparatus according to the embodiment of the present invention will be described. FIG. 7 is a flowchart showing an example of an image editing process procedure of the image editing apparatus according to the embodiment of the present invention. In FIG. 7, first, an image inside the vehicle is taken by the imaging unit 416 (see FIG. 4, the same applies hereinafter) provided inside the vehicle (step S 701), and the sound collecting unit provided inside the vehicle. Board the vehicle with 417 (see Figure 4, the same applies below)! / Sounds from the passenger (hereinafter referred to as “passenger” ヽぅ) are collected (step S702).

[0072] Image data of an image photographed by the photographing unit 416 is input to an image editing unit 412 (see FIG. 4 and the same below) through an image input / output IZF413 (see FIG. 4 and so on), and is collected by a sound collecting unit 417. The collected audio data is input to the control unit 400 (see FIG. 4 and the same below) through the audio input IZF418 (see FIG. 4 and the same below), and the image editing unit 412 and the control unit 400 input the image data. The feature amount of the image image and the feature amount of the sound parameter of the sound data are detected (step s703). In step S 703, information regarding the feature amount of the image image detected by the image editing unit 412 is output to the control unit 400.

[0073] After detecting the feature amount of the image and the feature amount of the audio parameter, the control unit 400 determines whether or not the atmosphere in the vehicle has changed based on the detected feature amount ( Step S704). Judgment of whether or not there is a change in the atmosphere in the vehicle is, for example, whether the detected feature value of the image changes from an emotion parameter representing “smile” to an emotion parameter representing “crying face” This is done by judging that the feature value changes from a frequency component representing “laughing voice” to a frequency component representing “screaming voice”.

In step S704, if control unit 400 determines that there is no change in the vehicle interior (step S704: No), the process returns to step S701, and the processing from step S701 to step S704 is repeated. If it is determined in step S704 that the atmosphere in the vehicle has changed (step S704: Yes), the image editing unit 412 generates image data including time stamp data photographed by the photographing unit 416 through the image input / output IZF413. Obtain (step S705).

[0075] Then, in step S705, the image data is acquired, and the control unit 400 also acquires the current position information of the vehicle with the position acquisition unit 403 (see FIG. 4; the same applies hereinafter) force (step S706), and the recording medium Through the decoding unit 405 (see FIG. 4, the same applies hereinafter), the recording medium 404 (see FIG. 4; the same applies hereinafter) also acquires map information (step S707), and further acquires information on the route and time of travel of the vehicle ( Step S708).

[0076] In step S708, after acquiring information related to the traveled route and time, the control unit 400 collates the time stamp data of the image data acquired by the image editing unit 412 with the information related to the traveled route and time, A point on the map where the vehicle passes at the time indicated by the time stamp data of the image is detected, and the image data is associated with the map information (step S709).

[0077] Then, after associating the image data with the map information, the image editing unit 412 creates album data using the image data (step S710). After creating the album data in this manner, the position acquisition unit 403 or the like acquires behavior information relating to the vehicle behavior such as information relating to the vehicle speed and inclination angle information (step S711).

The behavior information acquired in this way is output to the audio reproduction unit 414 (see FIG. 6; the same applies hereinafter) through the control unit 400, and the audio reproduction unit 414 acquires album data from the image editing unit 412. The audio playback processing unit 610 (see FIG. 6; the same applies hereinafter) allows the audio data from the audio DB 611 (see FIG. 6; the same applies hereinafter) and the music selection history DB 612 (see FIG. 6; same applies hereinafter) The audio data is associated with the album data by referring to information on the power selection history (step S712).

[0079] Here, associating the audio data, for example, based on the map information and behavior information associated with the album data, the terrain and road type at the time of image capture are determined, and the determined terrain and road type are determined. Audio data such as matching music is read from the audio DB611 and associated. In addition, referring to the above-mentioned feature values of the image and the feature values of the audio parameters, the sound data suitable for these feature values may be associated.

[0080] In step S712, after associating the audio data with the album data, the image editing unit 412 and the control unit 400 determine whether the album data is completed (step S713). If it is determined that the album data is completed! /, NA! / (Step S713: No), the process returns to step S701, and the processing from step S701 to step S713 is repeated. If it is determined that the album data has been completed (step S713: Yes), the series of image editing processing by this flowchart is terminated.

[0081] Here, another process of associating audio data with album data in step S712 will be briefly described. FIG. 8 and FIG. 9 are flowcharts showing an example of another procedure for associating audio data in the image editing process of the image editing apparatus according to the embodiment of the present invention. FIG. 8 shows the association process based on the time stamp data of the image data, and FIG. 9 shows the association process based on the tone data of the image image of the image data.

In FIG. 8, first, the audio reproduction processing unit 61 0 (see FIG. 6, the same applies hereinafter) of the audio reproduction unit 414 (see FIG. 6, the same applies hereinafter) is selected from the music selection history DB 612 (refer to FIG. 6, the same applies hereinafter). For example, the information about the reproduction history of the music that has been reproduced by the image editing device 310 (see FIG. 4, the same applies hereinafter) is acquired (step S801). After acquiring the information regarding the playback history, the audio playback processing unit 610 refers to the time stamp data of the image data in the album data (step S802).

[0083] In step S802, after referring to the time stamp data, the audio reproduction processing unit 610 selects the audio data of the music having information regarding the reproduction history reproduced at the time closest to the referred time stamp data. (Step S803). This way the audio data After the selection, the audio reproduction processing unit 610 associates the selected audio data with the album data (step S804). In step S804, for example, the chorus portion (highlight portion) of the selected audio data may be associated with the album data to associate the audio data.

On the other hand, in FIG. 9, an audio reproduction processing unit 610 (see FIG. 6, the same applies hereinafter), for example, an image editing processing unit 510 (see FIG. 5; Refer to the feature value of the color tone data as the feature value of the image in the entire data (step S901). Then, the audio reproduction processing unit 610 selects audio data corresponding to the feature amount of the referred tone data from the audio DB 611 (see FIG. 6, the same applies hereinafter) (step S902).

In this step S902, the sound data is selected, for example, when the color of the image of the entire image data is blue, the sound data of a sad atmosphere is selected, and when the color is green, the sound of the atmosphere of the healing atmosphere is selected. If the audio data is selected and it is red, the uptempo music data is selected. After selecting the audio data in this way, the audio reproduction processing unit 610 associates the selected audio data with the album data (step S903). In step S903, for example, the chorus portion (highlight portion) of the selected audio data is associated with the album data so as to associate the audio data.

Note that the selection of audio data in step S902 may be performed based on, for example, emotion parameters represented by the face image of the image data. In this case, for example, when the face image shows joy, select the voice data with a tone of brightness and atmosphere, and when the image shows anger, select the voice data with a tone of intense atmosphere and enjoy it. When !! is displayed, the voice data of up-tempo music is selected.

Next, a specific example of the image editing process of the image editing apparatus that is useful in the embodiment of the present invention will be described. 10 and 11 are explanatory diagrams showing a specific processing example of the image editing processing of the image editing apparatus according to the embodiment of the present invention. As shown in FIG. 10, the image editing unit 412 (see FIG. 5; the same below) of the image editing device 310 (see FIG. Section 403 (see Fig. 4; the same applies hereinafter) acquired by the control unit 400 (see Fig. 4; the same applies hereinafter) as well as vehicle current position information, route and time information, and recording medium 404 (Fig. 4). Refer to the map information input through the force control unit 400, and on the route from the starting point S to the end point E of the vehicle, the shooting unit 416 (see FIG. Part 417 (Refer to Fig. 4; the same applies hereinafter) Image data when it is determined that the atmosphere in the vehicle has changed based on the feature values of the force image data and the image data and sound parameters of the sound data. get.

[0088] In the case shown in Fig. 10, it is determined that the atmosphere in the vehicle has changed at photo acquisition points A to D, and image data has been acquired. Then, for example, the image editing unit 412 includes the time stamp data of the image data acquired at the photo acquisition points A to D, the current position information of the vehicle acquired at the photo acquisition points A to D, and the route of the vehicle. Based on the information about the time and the time, album data is created by associating the image data acquired at the photo acquisition points A to D with the map information. Since this album data is associated with audio data such as music, music can be automatically played back as appropriate.

[0089] The album data created by the image editing unit 412 in this way is, for example, a double-page album on the display screen of the display unit 402 (see Fig. 4; the same applies hereinafter) as shown in Fig. 11. It can be displayed in the form. In the displayed album data 1110, for example, image data 1120, 1130, 1140, 1150 acquired at photo acquisition points A to D (see FIG. 10, the same shall apply hereinafter) can be displayed in chronological order. The displayed image data 1120, 1130, 1140, and 1150 are the image images 1121, 1131, 1141, and 1151 and the scenery A to D outside the vehicle at image acquisition points A to D. Show 122, 1132, 1142, 1152! / ,.

[0090] Each of the displayed image data 1120, 1130, 1140, 1150 includes place information obtained from the associated map information, and text information representing the time when the image was taken or the vehicle passed. It is good to display. In FIG. 11, for example, the image data 1120 is taken at the time “8:14 am”, and the photograph acquisition point A taken is “near Yorii Station”. Similarly, the image data 1130 is taken at the time “8:37 am”, and the taken photo acquisition point B is “near Nagatoro”. Also, image data 1140 is taken at the time “1:20 pm”, and the photograph acquisition point C is “near Chichibu Station”. Further, the image data 1150 is taken at the time “2:53 pm”, and the photograph acquisition point D taken is “near Masamaru Pass”. Here, for example, the image data 1120, 1130, 1140, and 1150 may be displayed in the order of the route that the vehicle has passed.

As described above, the image editing apparatus according to the present embodiment uses the time stamp data of the image data captured when the atmosphere in the vehicle changes, and information on the route and time the vehicle has moved. Based on this, album data can be created by associating image data with map information without using a server or the like. For this reason, image data can be automatically edited in chronological order or route order to create album data, reducing the effort of image editing.

In addition, the image editing apparatus according to the present embodiment can associate the image data of the album data with the audio data without using a server or the like, so that it is possible to improve entertainment properties and reduce the labor and cost of the image editing process. Can be reduced.

[0093] As described above, according to the image editing apparatus, the image editing method, the image editing program, and the computer-readable recording medium according to the present invention, images are automatically captured as appropriate! / As a result, an electronic album can be created by automatically associating the acquired image data with map information and audio data as appropriate.

Note that the image editing method described in the present embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read by the computer. Further, this program may be a transmission medium that can be distributed through a network such as the Internet.

Claims

The scope of the claims

[1] An input means for accepting input of image data including information on date and time,

Based on acquisition means for acquiring information on the route and time of movement of the moving body, information on the date and time of the image data received by the input means, and information on the route and time acquired by the acquisition means An association means for associating the image data with map information;

An image editing apparatus comprising:

2. The image editing apparatus according to claim 1, further comprising display means for displaying the image data associated by the association means.

[3] Furthermore, photographing means for photographing an image;

Sound collecting means for collecting sound;

Detection that detects the feature amount of the image image included in the image data of the image photographed by the photographing means and the feature amount of the sound parameter contained in the sound data of the sound collected by the sound collecting means. Means,

3. The image editing apparatus according to claim 1, further comprising a control unit that controls the photographing unit based on a feature amount detected by the detection unit.

4. The image editing apparatus according to claim 3, wherein the detection unit detects a feature amount of the person's face image from an image related to the person included in the image data.

[5] Furthermore, an audio reproducing means for reproducing audio data is provided,

The acquisition means acquires behavior information relating to the behavior of the moving object,

The audio reproducing means reproduces audio data to be reproduced based on the feature amount detected by the detecting means and the behavior information acquired by the acquiring means when displaying the image data on the display means. The image editing apparatus according to claim 3, wherein selection is performed.

6. The display unit displays text information included in the map information based on the map information together with the image data associated by the association unit. 3. The image editing apparatus according to 3.

[7] The detection unit detects a feature amount of an emotion parameter of the person's face image as the feature amount of the image image, and features of a volume component, a time component, and a frequency component as the feature amount of the voice parameter 5. The image editing apparatus according to claim 4, wherein the amount is detected.

8. The image editing apparatus according to claim 3, wherein when the feature amount detected by the detection unit is changed, the control unit captures the image by the imaging unit.

[9] The acquisition means acquires at least one of! /! Deviation as information about the speed of the moving object, inclination angle information, lateral G (Gravity) information, and current position information as the behavior information. The image editing apparatus according to claim 5, wherein the image editing apparatus is characterized.

[10] An input process for inputting image data including date and time information;

Based on an acquisition step of acquiring information on a route and time of movement of the mobile body, information on the date and time of the image data input in the input step, and information on the route and time acquired in the acquisition step An associating step for associating the image data with map information;

An image editing method comprising:

[11] An image editing program that causes a computer to execute the image editing method according to claim 10.

[12] A computer-readable recording medium in which the image editing program according to claim 11 is recorded.