JP5102708B2 - Composite image photographing system and composite image photographing method - Google Patents

Description

  The present invention relates to a composite image capturing system and a composite image capturing method for capturing a digital image, such as a mobile phone with a camera and a digital camera.

  In recent years, with the development of digital technology, devices for taking digital images, such as camera-equipped mobile phones and digital cameras, have been developed, and the miniaturization has improved portability and the diversification of functions.

As one of such functions, there are functions such as writing characters and pictures in the shot image, adding a frame image such as a frame, etc., by recognizing the face part of the shot person, A technique has also been developed in which a frame and a template are matched with the face portion (for example, Patent Document 1).
JP 2006-113741 A

  However, users of digital cameras have increased portability and can easily shoot at various locations, so there is a desire to synthesize frames and templates associated with the shooting location. It is expected that it will be a trigger for regional revitalization in each region by distributing local unique content such as famous sights, specialties and characters in each region.

  Therefore, the present invention solves the above problems, and in a device for taking a digital image, such as a mobile phone with a camera or a digital camera, additional contents associated with the shooting location can be obtained. It is an object of the present invention to provide a composite image capturing system and a composite image capturing method capable of diversifying services provided in a region.

In order to solve the above problems, the present invention acquires an image capturing unit that captures an image, position information indicating the position of the own device at the time of image capturing, and a position information acquiring unit that records the position information. An image selection unit that selects region image data associated with the position information from the region image storage unit based on the position information at the time of shooting recorded by the position information acquisition unit; The image captured by the image capturing unit is used as a basic image, and an image compositing unit that synthesizes the regional image data selected by the image selecting unit with the basic image, and detects the face portion of the person. The face detection unit that calculates the coordinate position in the basic image and each part constituting the face part detected by the face detection unit are extracted as vector data, and the features of the face specified by the vector data are managed And a management information generation unit for generating a broadcast, local image storage unit, and the position information, the part library integrated facial parts associated with the management information, stored as part of the regional image data, the image The selection unit extracts parts from the parts library based on the position information and the management information, and the image synthesis unit uses the coordinate position of the face part detected by the face detection unit as a reference for the region for the basic image. A composite image photographing system characterized by combining image data and generating a character by combining extracted parts .

In another aspect of the invention, an image capturing step for capturing an image by the image capturing unit, position information indicating the position of the own device at the time of capturing, a position information acquiring step for recording, and area image data are associated with the position information in advance. Based on the position information at the time of shooting recorded by the position information acquisition unit, the image selection step for selecting the region image data associated with the position information and the face portion of the person are detected In addition, a face detection step for calculating a coordinate position in the basic image of the detected face portion and an image captured by the photographing unit as a basic image, and the regional image data selected by the image selection unit with respect to the basic image an image combining step of combining, and the respective parts constituting the face portion detected by the face detection step is extracted as vector data, it is specified by the vector data To generate a feature of the face as the management information, the part associated with the positional information and the management information, and a management information generation step of integrated parts library as part of the regional image data, the image synthesis step Based on the coordinate position of the face part detected in the face detection step, the regional image data is synthesized with the basic image, and the parts are extracted from the parts library based on the position information and management information. Then, a composite image photographing method characterized by combining the extracted parts to generate a character .

According to such an invention, it is possible to synthesize regional image data associated with the current location where the image was taken, for example, a frame or template associated with the location where the image was taken. Each region's unique content related to specialties and characters can be distributed only in that region, which can be a trigger for regional development in each region. Further, prior to image composition, the face portion of a person is detected, the coordinate position of the detected face portion in the basic image is calculated, and the image position is determined based on the coordinate position of the detected face portion. Since the regional image data is synthesized with the basic image, it is possible to match a frame or template such as a so-called “face-up signboard” depicting characters in each region with the face portion of the subject.

  In the above invention, the management information can be shared and synchronized between the plurality of devices by transmitting and receiving between the plurality of devices, and at the time of character generation, the management information and the management information by other sampling It is preferable to perform matching with and display the degree of match.

  In the above invention, it is preferable to perform facial expression recognition for recognizing a predetermined facial expression of a person to be photographed prior to imaging, and to automatically perform the photographing process at the moment when the facial expression is recognized. In this case, for example, you can automatically release the shutter at the moment when the subject in the field of view of the digital camera's viewfinder laughs, and the natural expression is different from shooting with a camera. Can be captured.

  In the above invention, the image taken by the image pickup unit is stored in the data storage unit in association with the position information acquired by the position information acquisition unit, and the position information of the stored image is displayed as an index on the display interface. It is preferable to retrieve the corresponding image from the data storage unit by selecting the displayed index. At this time, the display interface reads the map information corresponding to the accumulated position information of each image from the map information storage unit that stores and holds the map information, and the position corresponding to each position information on the read map information. The index of each image is displayed. In this case, retrieval of captured image data can be facilitated by using position information acquired at the time of capturing. By using additional information such as the date and time at the time of shooting as an index together with the position information, more various searches can be made possible.

  In the above invention, the display interface expresses the map information as a graphic of the planet displayed on the touch panel, and the index is displayed superimposed on the position corresponding to the latitude and longitude of the planet, and the planet is revolved by a touch operation. Or it is preferable that it can be selected by rotating.

  Further, in the above invention, in the parts library, sentence information is added for each part accumulated in the parts library, and in the character generation by the image composition unit, the sentence is determined according to the combination of the synthesized parts. The synthetic image photographing method according to claim 11, wherein a sentence is constructed by information.

  As described above, according to the present invention, in a device for taking a digital image, such as a mobile phone with a camera or a digital camera, it is possible to acquire additional content associated with the shooting location and to synthesize it with the taken image. This makes it possible to diversify the services provided in each region.

[First Embodiment]
(Structure of composite image shooting system)
Exemplary embodiments of a composite image photographing system according to the present invention will be described below in detail with reference to the accompanying drawings. FIG. 1 is a conceptual diagram showing an overall configuration of a composite image photographing system according to the present embodiment. The “module” used in the description refers to a functional unit that is configured by hardware such as an apparatus or a device, software having the function, or a combination thereof, and achieves a predetermined operation. .

  In this embodiment, as shown in FIG. 1, the composite image photographing system is roughly composed of a digital camera 1a or camera-equipped mobile phone 1b used by a user, and a regional image distribution server 3 installed on the Internet 2. .

  The regional image distribution server 3 provides local images such as background data and wallpaper data of local attractions and specialties, as well as frames and templates such as “signboards” depicting the characters of each region. A server that distributes content including image data in a limited region. In this embodiment, the server serves as a regional image storage unit that stores content data such as regional image data in association with location information. For example, a Web server The content server. The content server is a server computer that transmits information such as an HTML (HyperText Markup Language) file, an image file, or a music file in a document system such as the WWW (World Wide Web) or software having the function. Information such as documents and images is stored, and the information is transmitted through the Internet 2 in response to a request from the digital camera 1a or the mobile phone 1b.

  Each of the digital camera 1a and the camera-equipped mobile phone 1b has an imaging function for optically capturing a digital image, and also includes a position information acquisition function for acquiring and recording position information indicating the position of the own device at the time of shooting. ing. As this position information acquisition function, as shown in FIG. 1, for example, a method of detecting the position of the own device by a signal from the satellite 21 such as GPS (Global Positioning System), or a radio base station 22 of a mobile phone A method of detecting the position based on the intensity of the radio wave from, etc. is considered.

  FIG. 2 is a block diagram illustrating an internal configuration of the composite image capturing unit 10 relating to the composite image capturing function of the digital camera 1a or the mobile phone 1b.

  First, the composite image capturing unit 10 includes an image capturing unit 102, a shutter control unit 106, a face detection unit 107, and a data storage device 108 as image capturing system modules.

  The imaging unit 102 is a module for taking a digital image scientifically. The imaging unit 102 executes an imaging process according to control from the shutter control unit 106, and the captured image data is sent to the face detection unit 107. The shutter control unit 106 also includes an automatic shooting mode in which a shooting process is automatically executed at a moment when the facial expression recognition unit 107a in the face detection unit 107 recognizes a specific facial expression such as a smiling face of a subject.

  The face detection unit 107 extracts a certain geometric shape formed by eyes, nose, mouth, etc., detects a human face part, and calculates a coordinate position of the detected face part in the basic image It is a module to do. This face detection process is executed from the previous stage of the photographing operation. As shown in FIG. 4A, the detected face is highlighted as a marking 107b on the finder screen, and the distance to the subject is measured. It also works with an autofocus function that performs focusing according to distance. Further, the face detection unit 107 has a facial expression recognition unit 107a that recognizes a predetermined facial expression such as a smile of a person to be photographed, and recognizes the face part and the facial expression from the monitor image at the time of photographing. The recognition result is input to the shutter control unit 106. The shutter control unit 106 inputs a control signal for causing the imaging unit 102 to execute a shutter operation in accordance with the input recognition result.

  The data storage device 108 is a storage device that stores various types of data. The captured basic image data, regional image data acquired from the network, composite image data obtained by combining these, and processing of each image data. Edited data is stored.

  Further, the composite image photographing unit 10 includes a position information acquisition unit 103, an image acquisition unit 104, and a map information acquisition unit 105 as communication modules.

  The position information acquisition unit 103 is a module that acquires and records position information indicating the position of the own device at the time of photographing. As described above, for example, a signal from the satellite 21 such as GPS (Global Positioning System). Is used to detect the position of the device itself, or the position is detected based on the radio wave intensity from the radio base station 22 of the mobile phone. The acquired position information is stored and held in the data storage device 108 in association with the basic image as metadata of the basic image data.

  In the present embodiment, the position information acquired by the position information acquisition unit 103 is transmitted to the Internet 2 through the communication I / F 101 and received by the regional image distribution server 3. The regional image distribution server 3 selects and returns content such as regional image data associated with the received position information. The returned regional image data is stored in the data storage device 108 through the image acquisition unit 104. The image acquisition unit 104 is a module that downloads regional image data associated with position information from the regional image distribution server 3.

  The map information acquisition unit 105 is a module that downloads map information from a content server installed on the Internet 2 and stores and holds the map information. The acquired map information is stored in the data storage device 108.

  Further, the composite image photographing unit 10 includes an edit processing unit 110, an edit data generation unit 111, and an image composition unit 112 as edit processing modules. In the present embodiment, operations such as editing processing and display processing can be performed using a GUI 116 as shown in FIGS.

  The editing processing unit 110 is a module that executes various editing processes according to the operation of the operator. In conjunction with the GUI 116, photo retouching processing such as drawing a picture on an image with a touch pen or writing characters or the like. In addition, it is possible to perform an image composition process for compositing an image such as a frame image or a background image with a taken basic image.

  The edit data generation unit 111 is a module that generates edit data as a result of the edit operation by the edit processing unit 110, and the generated data is stored in the data storage device 108. The image synthesis unit 112 is a module provided as part of the editing function of the editing data generation unit 111. The image captured by the imaging unit 102 is used as a basic image, and the image acquisition unit 104 acquires the basic image. Synthesize regional image data. The synthesizing process by the image synthesizing unit 112 can also be controlled by an editing operation on the editing processing unit 110.

  When the image composition unit 112 performs composition, the regional image data is superimposed on the basic image with reference to the coordinate position of the face portion detected by the face detection unit 107. In this image composition, the image composition unit 111a uses the coordinate position of the face part detected by the face detection unit 107 as a reference, as shown in FIG. 4 (b) or (c), for the captured basic image D1. Then, the regional image data D2 or D3 with the character head covering as a motif is synthesized.

  These operations can be performed using the GUI 116 described above. In the present embodiment, a basic image D1 related to shooting (or editing) is displayed on the GUI 116, and a plurality of icons 1162 of the acquired area image data are listed in the icon area 1161, and these icons 1162 are displayed as For example, an icon can be selected by clicking with the pointer 1163.

  The composite image photographing unit 10 includes an operation signal acquisition unit 114, a display information generation unit 115, an index generation unit 109, and a data detection unit 113 as user interface modules.

  The operation signal acquisition unit 114 is a module that acquires an operation signal from the operation device 116 a based on a user operation, and inputs an execution command corresponding to the operation signal to the editing processing unit 110. Here, the operation device 116a includes a touch panel, operation buttons, an acceleration sensor, and the like.

  The display information generation unit 115 is a module that generates image data displayed on the image display unit 116b of the GUI 116. At the time of shooting, the display information generation unit 115 displays a finder image captured by the imaging unit 102 or at the time of editing. The change in the image such as an icon based on the user operation signal acquired by the operation signal acquisition unit 114 is displayed.

  In particular, the display information generation unit according to the present embodiment has a function of displaying position information of an image stored in the data storage device 108 as an index icon I31 as shown in FIG. In this function, map information corresponding to the accumulated position information of each image is read from the data storage device 108, and each content is placed at the coordinate position corresponding to the position information of each content on the map W3 that has been read and developed. The index icon I31 is displayed. The index generation unit 109 generates index information (icon) for displaying the image captured by the imaging unit 102 and the position information acquired by the position information acquisition unit 103 in association with each other, and inputs the index information to the display information generation unit 115. Then, the display information generating unit 115 superimposes the index icon on the map W3.

  The data detection unit 113 is a module that searches for and reads data stored in the data storage device 108, and reads the corresponding data by selecting an index icon displayed on the map on the GUI 116.

  In the present embodiment, a mobile phone having a camera function has been described as an example. As this camera function, for example, as shown in FIG. It may be a compact mobile phone, or may be a pendant type 1c that is independent of the main body of the mobile phone 1b and is connected to only the camera portion by a cord as shown in FIG.

  As shown in FIG. 10, by selecting the automatic shooting mode as shown in FIG. 10, the user himself / herself does not have to worry about the shutter operation, and the people around him can enjoy the toast or karaoke scene. A commemorative photo is automatically taken at the time when the person smiles.

(Configuration of GUI)
FIG. 5 is an explanatory diagram showing the screen configuration of the GUI 116 described above. In this embodiment, the GUI 116 is provided in the mobile phone 1b, and the mobile phone 1b includes a front portion 11 and a back portion 12 that can slide in the horizontal direction in the figure. The GUI 116 is interlocked with a touch panel disposed on the entire front surface of the front surface portion 11, and a graphic is displayed on the touch panel by the image display portion 116 b and is given to the touch panel surface by the operation device 116 a. The pressure position is detected, and the coordinate position of the pressure point on the screen and its movement are acquired as an operation signal.

  In the GUI 116, an execution command assigned to an area by touching a predetermined area on the screen is acquired as an operation signal. Icons and pop-up menus are arranged in the predetermined area, and the user can input an arbitrary execution command by selecting and touching these icons. In particular, in this embodiment, as shown in FIG. 5, by tracing the screen with a finger or a touch pen, the movement distance and movement speed of the pressure point are detected from the start position and the end position of the operation, and the operation is performed. The execution command can be input accordingly. Note that normal operation buttons and direction keys are arranged on the front surface of the rear surface portion 12 that is slid and protrudes from the front surface portion 11. By operating these buttons, an input operation to the GUI can be performed. ing.

  In addition, the menu displayed on the GUI 116 in the present embodiment has a hierarchical format, and can be shifted to a lower menu (zoom up and zoom down) by a selection operation on the upper menu. In particular, in the GUI 116, as shown in FIG. 6, menu icons are represented as planets floating in space, and command groups for each function are grouped and assigned to each planet. As shown in FIG. 6A, the user selects a group of commands as a planet on the planet display screen W1 that is the upper menu, and zooms up the planet, thereby displaying the earth display screen W2 that is the lower menu. You can drill down to a more detailed menu. Each of these planets is arranged in a ring like the galaxy and the solar system, for example, and each planet can be revolved and rotated by the touch operation. Can be operated. These screen transitions can also be performed by a pop-up menu M1 such as a zoom switching icon as shown in FIG.

  In the earth display screen W2, as shown in FIGS. 6 (b) and 7, the earth (earth) is included as one of the planets, and commands or commands that can be associated with position information such as latitude and background on the earth are displayed. Data can be displayed as index icons I3 to I5 on the planet. In FIG. 7, news occurring in various places on the earth is displayed as a balloon icon I5, and captured image data is displayed as a camera icon I4 at positions corresponding to related latitude and longitude. For example, when information about the address of the mail sender is acquired as the index icon, the mail icon I3 can be displayed at the latitude and longitude corresponding to the address.

  It should be noted that icons I1 and I2 for calling up menus are arranged on the screen, and when this is clicked, the zoom switching menu M1 and the display switching menu M2 for switching the type of index icon displayed on the screen. Can be called. In addition, a slide operation slider 1165 is displayed on the earth display screen, and the earth can be zoomed up and down in a stepless manner by sliding the slider. Transition to W3 is possible.

  On the earth display screen W2, by selecting an index icon displayed on the earth, the ground surface (home display screen W3) as shown in FIG. 6 (d) can be further zoomed in. On the screen W3, an index icon associated with the position information is displayed together with more detailed map information.

  On the home display screen W3, as shown in FIG. 8, the image stored in the data storage device 108 is displayed as an index icon corresponding to the position information. Specifically, the map information corresponding to the stored position information of each image is read from the data storage device 108 and corresponds to the position information of each content on the read map information as shown in FIG. The index icon I31 of each content is displayed superimposed on the coordinate position. By clicking the index icon I31, an image D photographed in the area as shown in FIG. 5C can be called up and displayed. The image D can also be directly called from the camera navigation menu M3 as shown in FIG.

(Synthetic image shooting method)
By operating the composite image capturing system having the above configuration, the composite image capturing method of the present invention can be implemented. FIG. 3 is a flowchart showing the operation of the composite image photographing system according to this embodiment.

First, an image is taken by the imaging unit 102 (S101). In this embodiment, in the present embodiment, an “automatic shooting mode” in which the finder image is always monitored by the imaging unit 102 may be executed. In this mode, the facial expression recognition unit 107a performs facial expression recognition on a finder image taken by the imaging unit 102, for example, a predetermined facial expression such as a smile of a person to be photographed, and the moment when the smile is recognized. In addition, the photographing process is automatically executed.
In step S101, the image data photographed by the imaging unit 102 is recorded in the data storage device 108 as a basic image (S102), and a face is detected in the basic image (S103). When detecting the face part, the face detection unit 107 calculates the coordinate position of the detected face part in the basic image.

  Further, in response to this shooting operation, the position information of the own device at the time of shooting is acquired and recorded, and this position information is transmitted to the regional image distribution server 3 (S104). The regional image distribution server 3 that has received this positional information (S201) searches the regional image storage database for regional image data associated with the positional information based on the received positional information at the time of shooting (S202). The selected image is returned (S203).

  The camera side that has received the area image data (S105) synthesizes the received area image data with the basic image (S106). In this image composition, the image composition unit 111a synthesizes the regional image data with the basic image using the coordinate position of the face portion detected by the face detection unit 107 as a reference.

  The image data thus photographed and accumulated can be subjected to other editing operations such as photo retouching processing (S107), and by finishing the editing operation ("N" in S108), The edited image data is saved (S109).

  In storing this image data, in this embodiment, the image data is stored in the data storage device 108 in association with the position information. At this time, the position information of the accumulated image is generated as index information (S110) and displayed as an icon on the GUI 116. Then, by selecting the index displayed on the GUI 116, the corresponding image can be retrieved from the data storage device 108 and read out. At this time, the display information generation unit 115 reads map information corresponding to the accumulated position information of each image, and displays an index icon of each image at a position corresponding to the position information on the read map. .

  The data captured and accumulated in this way can be read and viewed on the GUI 116 by selecting an index icon by a selection operation such as a touch operation.

(Composite image shooting program)
The above-described system and method according to the present embodiment can be realized by executing a composite image photographing program described in a predetermined language on a computer. That is, by installing this composite image photographing program on the IC chip of the information processing terminal using a mobile phone or the like and executing it on the CPU, a system having the above functions can be easily constructed. This program can be distributed through a communication line, for example.

  According to the computer-readable recording medium in which the program is recorded, the above-described system and method can be implemented using a general-purpose computer or a dedicated computer, and the program can be stored, transported, and installed. It can be done easily.

(Operations and effects according to this embodiment)
According to the present embodiment, the area image data associated with the captured current position, for example, a frame or template such as a “face-up signboard” depicting local characters associated with the shooting location, It can be combined and superimposed on the face part of the subject, and each region's unique content can be distributed only in that region, thereby triggering regional development in each region.

  In the present embodiment, the automatic shooting function enables the shutter to be automatically released at the moment when the subject in the field of view of the digital camera finder laughs and the camera is held. You can capture a natural expression different from that.

  Furthermore, in the present embodiment, since the captured image data can be searched and read by the index icon associated with the position information, the image search can be easily performed using the position information acquired at the time of shooting. it can. By using additional information such as the date and time at the time of shooting as an index together with the position information, more various searches can be made possible.

[Second Embodiment]
(System configuration)
Next, the second embodiment board of the present invention will be described. FIG. 11 is a conceptual diagram illustrating an overall configuration of a composite image capturing system according to the embodiment of the second embodiment. In the present embodiment, the same components as those in the first embodiment described above are denoted by the same reference numerals, and the functions and the like are the same unless otherwise specified, and the description thereof is omitted.

  As shown in FIG. 11, the present embodiment is characterized in that the edit data generation unit 111 includes a character generation unit 111a for automatically generating a two-dimensional character. This character generation unit 111b is a management information generation unit that extracts each part constituting the face portion detected by the face detection unit 107a as vector data, and generates the feature of the face specified by the vector data as management information. It has the function of.

  The character generation unit 111b has a function of causing the image acquisition unit 104 to acquire a parts library based on the position information and the management information, and to extract parts from the acquired part library. In addition, the image synthesizing unit 111a has a function of synthesizing the extracted parts and generating a character.

  The management information can be generated, for example, in conjunction with the composite image shooting function of the digital camera 1a or the mobile phone 1b as in the first embodiment described above, and the generated management information is other attached information. At the same time, it is stored in the server 10 as data indicating the characteristics of the character unique to the user, or transferred through a communication function or storage medium, so that it can be shared with other personal computers, home game machines, portable game machines, etc. Can be made.

  The attached information includes, for example, information that characterizes the character, such as the character's name, gender, function, ability, training information, personal information of the subject itself when the character is generated, operation history for the character, fortune-telling / diary -Related information such as e-mail transmission / reception history when the character is regarded as an alternate subject. Note that this subject is not limited to the owner of the digital camera 1a or the mobile phone 1b, and any photographed person can be automatically generated as a character, and the attached information is an input interface of each device. Can be entered through and stored in association with the corresponding management information.

  In the present embodiment, on the server 10 side of the present embodiment, a part library in which face parts associated with position information and management information are accumulated is stored as part of regional image data. For example, the database of the server 10 includes a parts library in which various parts for representing characters are stored as regional images, and transmits necessary parts in response to a request from the user. This parts library includes basic parts such as eyes, eyebrows, nose, mouth, contour, ears, and hairstyles, moles, accessories (glasses, earrings, earrings, etc.), etc. to express various characters. Various parts such as parts are included. In the parts library, text information such as fortune telling is added to each part accumulated in the parts library, and in the character generation by the character generation unit 111b, according to the combination of the synthesized parts, Fortune-telling sentences are constructed from the sentence information. This fortune-telling text information can be used as the attached information associated with the management information of the corresponding character.

  The character generation unit 111b is a module that automatically generates a character resembling the detected face based on the face image data detected by the face detection unit 107 from the captured image. Specifically, the character generation unit 111b receives a face image detected from the image captured by the imaging unit 102 from the face detection unit 107, and arranges information on the eyes and nose, for example, plane coordinates, inclination, size, and color. , Etc. are extracted as management information. The extracted management information is stored in the data storage device 108 and transmitted to the server 10 through the image acquisition unit 104 and the communication I / F 101. The image acquisition unit 104 transmits the management information together with the part request signal, receives additional information such as a parts library and fortune telling text distributed in response to the request, and stores the additional information in the data storage device 108. The accumulated parts library is sequentially read out in response to a request from the editing data generation unit 111 and is used for character generation processing and character editing processing.

  In the present embodiment, the parts library is associated with the position information, and a “local character” corresponding to the current position of the own device can be generated. In detail, position information is used as the selection priority of the parts library when creating an automatically generated character. For example, if you take in Africa, black type face parts are preferentially selected, or if you take in UK , White type face parts are preferentially selected. At this time, the position information may be used as the attached information indicating the character characteristics, for example, as the character's birthplace.

  In addition to the face parts described above, the parts library also includes the character's body, clothes, accessories, and other elements that make up the character. These are also linked to location information and named after the region. A character is to be generated. For example, when creating an automatically generated character, the constituent elements of the character are expanded to the body, clothes, accessories, etc., and clothes and accessories tailored to the region are attached. For example, if you take a picture at the seaside, it becomes a surfer style, if you take a picture in the snowy mountains, it becomes a skier style, and if you take it in a downtown area, it will be chosen as a flashy fashion and makeup.

  Furthermore, as a parts library, the above-mentioned clothes and accessories can be further expanded to include branded goods that are tied up with brand stores. For example, if you take a picture in the store of the brand shop, you can get clothes of the brand's original design, or if you take it in the event venue, you can get clothes and accessories of related designs.

(Automatic character generation process)
The automatic character generation process can be executed by operating the system having the above configuration. In the present embodiment, this automatic character generation process is executed as a subroutine for image composition in step S106 described in the first embodiment. FIG. 12 is a sequence diagram showing the procedure of automatic character generation processing.

  First, when a photographing operation is executed by the camera (FIG. 3: S101), face detection is executed (S106), and a synthesis process with the acquired area image is performed (S106). At this time, automatic character generation processing is started using the face image obtained by face detection.

  First, the face image is analyzed, outline vector data is created for each part such as eyes, nose, mouth, and eyebrows from the face photograph (S301), and plane coordinates are calculated at the center of the vector data (S301). S302).

  Next, management information is calculated based on the vector data and the plane coordinates (S303). Specifically, the inclination of the sample part is matched with the vector data, and the size is calculated from the vertical and horizontal sizes of the outermost periphery of the vector data. The management information calculated in this way is transmitted to the server 10 (S304 and S401).

  The server 10 searches the parts library based on the position information received in the previous step S201 and the management information received in step S401 (S402). Then, the retrieved parts are collected, parts necessary for expressing the user's face are extracted, a parts library unique to the user is generated, and transmitted to the user side (S403). At this time, the parts included in the user-specific parts library include parts and accessories associated with the current position unique to the local area.

  Further, in the present embodiment, fortune telling and the like can be distributed by using the user's facial features specified from the management information as additional information of the user-specific parts library. This fortune-telling can also be associated with the user's location information, making fortune-telling unique to the area. As this fortune-telling service, for example, fortune-telling sentences associated with the characteristics of regions and parts are associated as additional information in the parts library.

  Upon receiving and acquiring such a part library (S305), the user side collates the received part library with the vector data based on sampling points (S306), and selects a part (S307). A character is generated (S309).

  At this time, additional information associated with each part is also extracted in accordance with the selection of the part, and is configured as the above-mentioned additional information unique to the character (308). Specifically, fortune-telling sentences are extracted from the parts library, and correlated fortune-telling sentences are extracted from the arrangement information of the parts. Furthermore, text is extracted from the fortune-telling data and keywords are generated based on supplementary information such as the user's date of birth and name. In the case of compatibility fortune-telling, an accumulated database of partner information may be collated. In this way, the final fortune-telling text corresponding to the request is constructed by combining the extracted fortune-telling text and keywords.

  In this way, attached information such as fortune-telling sentences constructed after the generated character is stored in the data storage device 108 together with information about the generated character (management information and the like) (S310). The generated character can be changed within the range of the parts library acquired in step S305, and if the part is replaced due to the change, the fortune-telling associated with the new part is also possible. Updated. It should be noted that the management information and attached information related to each character and the related parts library stored in the data storage device 108 are sequentially created when they are newly created or updated through the communication interface 101 or periodically. 10 can be uploaded to or written out via a storage medium, and can be shared and synchronized with other devices via the server 10 or the storage medium.

(Example of change)
Furthermore, the present invention is not limited to the embodiment described above, and various modifications can be made.

  For example, in automatic character generation processing, matching processing with similar celebrities and entertainers may be added. Specifically, management information obtained by sampling facial features such as celebrities is stored in a database, and when searching the parts library in step S402, matching is performed with the management information of the celebrities stored in the database. Depending on the degree of match, steps such as ranking the degree of similarity to a certain entertainer or introducing a celebrity who is most similar may be added.

  Furthermore, you may make the character produced | generated by this automatic character production | generation function appear as a local frame frame demonstrated in the said 1st Embodiment. For example, an auto-generated character with a big Buddha bracket may be drawn in the image frame, and the photographed person may appear normally in the center of the image along with his local portrait character.

  As another function, an automatically generated character that has already been created may be replaced using face recognition. That is, an additional function such as superimposing the portrait character generated by the automatic character generation function on the coordinate position of the face detected by the face detection unit 107 is provided. For example, when a friend is photographed, various editing functions such as attaching the face of his / her automatically generated character to the friend's body or photographing the Great Buddha to make it an automatically generated character can be loaded.

  In addition, as an index icon generated by the index generation unit 109, an automatically generated character image can be used, and the index icon displayed on the map of the interface can be represented by a portrait. Since this index icon is associated with the location information of the shooting location, by including the local character generated by the character generation unit 111b as index information, the local character (local fashion) is displayed on the map or the like. And characters with local accessories).

  The index icon represented by this character is suitable for information sharing in the server 10 or the like. That is, the automatically generated character here is quantized with parts and management information rather than bitmap data such as a normal photograph, so the data size can be reduced. Using this lightened data, when sharing a photo on a map, the data transmission size from the server can be reduced and displayed as an easy-to-understand icon.

  In this embodiment, since the character is automatically generated, the data of each part for expressing the character can be miniaturized. In detail, in a system that manually selects parts one by one based on user operations, management information such as reducing the number of parts or setting the position of parts in stages to prioritize user operability. However, if it is set automatically as in this embodiment, the user operation will not be complicated even if it is not stepped. The parameter data for can be set in detail. As a result, when linked with fortune-telling, the fortune-telling evaluation is not stepwise and can be fed back with fine information such as a fashion degree of 23% and a compatibility degree of 97%.

  Furthermore, in this embodiment, since a caricature character can be automatically generated and collected as management information on the server 10 side, highly accurate data can be accumulated from a large number of users. It can be used for compatibility hobbies such as introducing good people.

  In addition, since it can be linked to the date of birth (age and constellation) strokes, colors, etc., you can use the date of birth to switch parts library data, color of constellations, strokes, and accessories (Feng Shui) System) or the like.

(Effects of automatic character generation function)
According to the automatic character generation function according to this embodiment, a character is created by automatically extracting facial features from a photographed face photo, so that it requires skill, time, and labor like a portrait. Because it does not force you to work, you can easily use the creation of characters such as avatars for users who have never wanted to do so.

  In the present embodiment, character management information is transmitted and received in a personal computer, a home game machine, and a portable game machine through the server 10 and various storage media, and information related to the character is shared and synchronized between these devices. By or by taking in, it can be continuously used as the same user-specific character appearing in an avatar or game in each device or medium.

  Furthermore, in this embodiment, since automatic character generation and fortune-telling are linked, the objectivity of fortune-telling can be ensured. In other words, it is an objective decision because the user is fortune-telling based on the photographed face photo rather than selecting any part that is his / her own characteristic. Since this is an objective judgment, the user's psychology is greatly different from that of fortune-telling. In addition to this, there are also advantages in terms of usage, such as anthropomorphized fortune tellers appearing in the GUI, and it is easy to eliminate arbitrary elements when matching users.

It is a conceptual diagram which shows the whole structure of the composite image imaging | photography system which concerns on embodiment which concerns on embodiment. 1 is a block diagram illustrating an internal configuration of a composite image capturing unit 10 relating to a composite image capturing function of a digital camera 1a or a mobile phone 1b according to a first embodiment. It is a flowchart figure which shows operation | movement of the composite image imaging system which concerns on 1st Embodiment. It is explanatory drawing of the image composition process which concerns on 1st Embodiment. It is explanatory drawing which shows the example of operation of GUI which concerns on 1st Embodiment. It is explanatory drawing which shows the screen transition of GUI which concerns on 1st Embodiment. It is a screen block diagram which shows the example of the planet display which concerns on 1st Embodiment. It is explanatory drawing which shows the operation screen of image browsing which concerns on 1st Embodiment. It is explanatory drawing which shows the example of a change of the imaging means which concerns on 1st Embodiment. It is explanatory drawing which shows the Example of the automatic imaging | photography mode which concerns on 1st Embodiment. It is a block diagram which shows the internal structure of the composite image imaging | photography unit 10 which concerns on the composite image imaging | photography function of the digital camera 1a or the mobile telephone 1b which concerns on 2nd Embodiment. It is a flowchart figure which shows operation | movement of the composite image imaging system which concerns on 2nd Embodiment.

Explanation of symbols

D1 ... Basic image D2, D3 ... Regional image data I3-I5 ... Index icon M1-M3 ... Menu icon W1 ... Planet display screen W2 ... Earth display screen W3 ... Home display screen 1a ... Digital camera 1b ... Mobile phone 2 ... Internet 3 DESCRIPTION OF SYMBOLS ... Regional image distribution server 10 ... Composite image photographing unit 11 ... Front part 12 ... Back part 21 ... Satellite 22 ... Wireless base station 102 ... Imaging part 103 ... Position information acquisition part 104 ... Image acquisition part 105 ... Map information acquisition part 106 ... Shutter control unit 107 ... face detection unit 107a ... facial expression recognition unit 107b ... marking 108 ... data storage device 109 ... index generation unit 110 ... edit processing unit 111 ... edit data generation unit 111a ... image composition unit 111b ... character generation unit 112 ... image Synthesizer 113 ... Data detector 14 ... operation signal acquisition unit 115 ... display information generation unit 116 ... GUI
116a: Operation device 116b: Image display unit

Claims (12)

An imaging unit for taking an image;
A position information acquisition unit for acquiring and recording position information indicating the position of the own device at the time of shooting;
A regional image storage unit that stores regional image data in association with location information;
Based on the position information at the time of shooting recorded by the position information acquisition unit, an image selection unit that selects the region image data associated with the position information from the region image storage unit;
An image combined with the image captured by the image capturing unit as a basic image, and the region image data selected by the image selection unit is combined with the basic image ;
A face detection unit for detecting a face portion of a person and calculating a coordinate position of the detected face portion in the basic image;
A management information generation unit that extracts each part constituting the face portion detected by the face detection unit as vector data, and generates a feature of the face specified by the vector data as management information ;
The regional image accumulation unit accumulates a part library in which parts associated with the position information and the management information are accumulated as a part of the regional image data,
The image selection unit extracts parts from a parts library based on the position information and the management information,
The image composition unit synthesizes the regional image data with the basic image based on the coordinate position of the face part detected by the face detection unit, and synthesizes the extracted parts to generate a character. A composite image photographing system characterized by the above. The management information can be shared and synchronized between these devices by transmitting and receiving between a plurality of devices, and when generating the character, the management information and other sampling management information The combined image photographing system according to claim 1, wherein matching is performed and the degree of coincidence is displayed . A facial expression recognition unit for recognizing a predetermined facial expression of a person to be photographed;
The composite image photographing system according to claim 1, wherein the imaging unit has an automatic photographing function that automatically performs photographing processing at the moment when the facial expression recognition unit recognizes the facial expression. . A data storage unit that stores an image captured by the imaging unit in association with the position information acquired by the position information acquisition unit;
A map information storage unit for storing and holding map information;
A display interface for displaying the stored image position information as an index;
A data search unit that reads out the corresponding image from the data storage unit by selecting the index displayed on the display interface;
Further comprising
The display interface reads map information corresponding to the accumulated position information of each image from the map information storage unit, and the position of each image in the position corresponding to each position information on the read map information. The composite image capturing system according to claim 1, which has a function of displaying an index. In the display interface, the map information is expressed as a graphic of the planet displayed on the touch panel, and the index is displayed superimposed on a position corresponding to the latitude and longitude of the planet, and the planet is displayed by a touch operation. The composite image photographing system according to claim 4 , wherein the composite image photographing system is selectable by revolving or rotating .   In the parts library, sentence information is added to each part accumulated in the parts library, and in the character generation by the image composition unit, the sentence information is written according to the combination of the synthesized parts. The composite image photographing system according to claim 5, wherein: An imaging step of capturing an image by the imaging unit;
A position information acquisition step for acquiring and recording position information indicating the position of the own device at the time of shooting;
An image for selecting the regional image data associated with the location information based on the location information at the time of shooting recorded by the location information acquisition unit from the regional image storage unit that has previously accumulated the regional image data in association with the location information. A selection step;
A face detecting step for detecting a face portion of a person and calculating a coordinate position of the detected face portion in the basic image;
An image synthesis step of synthesizing the regional image data selected by the image selection unit with the basic image that is an image captured by the imaging unit ;
Each part constituting the face portion detected in the face detection step is extracted as vector data, the feature of the face specified by the vector data is generated as management information, and the position information and the management information are A management information generation step of accumulating the associated parts in a parts library as a part of the regional image data ,
In the image synthesis step, the area image data is synthesized with the basic image on the basis of the coordinate position of the face portion detected in the face detection step, and the position information and the position information are extracted from the part library. A composite image photographing method , wherein parts are extracted based on the management information, and characters are generated by synthesizing the extracted parts . The management information can be shared and synchronized between these devices by transmitting and receiving between a plurality of devices, and when generating the character, the management information and other sampling management information The combined image photographing method according to claim 7, wherein matching is performed and the degree of coincidence is displayed . Prior to the imaging step, further comprising a facial expression recognition step for recognizing a predetermined facial expression of a person to be photographed,
8. The composite image photographing method according to claim 7, wherein in the imaging step, photographing processing is automatically executed at the moment when the facial expression is recognized in the facial expression recognition step. A data storage step of storing an image captured by the imaging unit in a data storage unit in association with the position information acquired by the position information acquisition unit;
A data search step of displaying the position information of the stored image as an index on the display interface and reading the corresponding image from the data storage unit by selecting the displayed index;
Further including
The display interface reads map information corresponding to the accumulated position information of each image from a map information storage unit that stores and holds the map information, and sets the position corresponding to each position information on the read map information. The composite image capturing method according to claim 7, wherein the index of each image is displayed. In the display interface, the map information is expressed as a graphic of the planet displayed on the touch panel, and the index is displayed superimposed on a position corresponding to the latitude and longitude of the planet in the data search step, The composite image capturing method according to claim 10 , wherein selection is possible by revolving or rotating the planet by a touch operation .   In the parts library, sentence information is added to each part accumulated in the parts library, and in the character generation by the image composition step, the sentence information is written according to the combination of the synthesized parts. The composite image capturing method according to claim 11, wherein: