JP2014057118A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image which is obtained by retrieving a network and is adjusted to environment of a self-terminal.SOLUTION: Only the number of images minus one, which are used for multi-display, are selected from a plurality of image files (step S17). A thumbnail image in response to display performance of a display section is generated from the plurality of selected image files (step S18). The thumbnail image generated with a photographed image as a center is multi-displayed (step S19). When there are eight thumbnail images, the photographed image is displayed in the center of the display section by processing in the step S19, and the eight thumbnails being the retrieved images are displayed at a periphery. Thus, a user can compare and appreciate the photographed image and the thumbnails.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing an image reproduced as a search result.

2. Description of the Related Art Conventionally, there is a technique for searching for images belonging to a predetermined range of a search key through a network using a position or date / time added to an arbitrary image as a search key.
Further, as described in Patent Document 1, after adding the above technique to an imaging apparatus, a technique for generating a thumbnail image as a search result on the server side and acquiring the thumbnail image has been proposed.

JP 2002-044506 A

However, the display resolution of current imaging devices differs depending on the model depending on specifications and performance. Also, from the perspective of a terminal having a communication function and an imaging function, a multi-function communication terminal can also be an object of an image search service. And diverse.
Therefore, as described in Patent Document 1, even if the thumbnail image is generated with a unique size on the server side, the display performance of the device on the reception side varies, so it is difficult to see depending on the display performance. There was a problem.

  The present invention has been made in view of the above problems, and an object of the present invention is to make it possible to obtain an image obtained by searching a network that matches the environment of the terminal itself.

  In order to solve the above-described problems, the present invention provides a captured image acquisition unit that acquires a captured image, a communication unit, and an image captured by the captured image acquisition unit connected to an external communication network via the communication unit. An instruction detecting unit for detecting a search instruction for a related image of the image, a related image acquiring unit for acquiring a related image via the external communication network as a result of the search instruction by the instruction detecting unit, and the related image acquiring unit An image processing apparatus comprising: processing means for processing so that the related image acquired by the method can be reproduced in association with the captured image.

  Further, the present invention is an image processing method using an apparatus having a communication unit and a reproduction unit, which is connected to an external communication network via the communication unit, and issues a related image search instruction for the previously acquired photographed image. An instruction detecting step for detecting, a related image acquiring step for acquiring a related image via the external communication network as a result of a search instruction in the instruction detecting step, and a related image acquired in the related image acquiring step And a processing step of processing the image so as to be reproduced in association with the captured image.

  Further, the present invention provides an instruction for connecting a computer included in an apparatus including a communication unit and a reproduction unit to an external communication network via the communication unit, and detecting a search instruction for a related image with respect to the captured image acquired in advance. As a result of the search instruction in the instruction detection step, the detection means, the related image acquisition means for acquiring the related image via the external communication network, and the related image acquired by the related image acquisition means are associated with the captured image. Provided is a program characterized by functioning as a processing means for processing so that it can be reproduced.

  According to the present invention, it is possible to obtain an image obtained by searching a network that matches the environment of the own terminal, and thus an image obtained from the network can be clearly displayed on the own terminal. .

1 is a block diagram illustrating an overall configuration of an image search system according to an embodiment of the present invention. It is a figure explaining the image of the image search system concerning the embodiment. It is a schematic block diagram which shows the structure of a digital camera. It is a schematic block diagram which shows the structure of a smart phone. It is a flowchart which shows the process sequence of the embodiment. It is a figure which shows the example of a display screen.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing the overall configuration of the image search system according to the present embodiment.
An image search site 10 including a service server 11 is connected to a smartphone 1 and a PC (Personal Computer) 3 via the Internet 500. The digital camera 2 can be connected to the smartphone 1 wirelessly such as Bluetooth (registered trademark). The smartphone 1 installs an application program and operates to cooperate with the image search site 10. In addition, a relatively light application program is installed in the digital camera 2 to operate in cooperation with the smartphone 1.
It is also possible to directly connect the digital camera 2 and the service server 11 to prepare a route for exchanging information without going through the smartphone 1.

  Further, the service server 11 includes a database in which a large number of image files are organized and stored for each shooting date / time, shooting position, shooting condition, and the like, and a plurality of image files having the same shooting date / time, shooting position, shooting condition, and the like. It is possible to search. These image files are collected from the Internet 500, and a URL (Uniform Resource Locator) indicating the storage location of the information resource on the Internet 500 is added to each image file.

  FIG. 2 is a diagram for explaining an image of the image search system according to the present embodiment. The smartphone 1 and the PC 3 are connected via the image search site 10, and the digital camera 2 and the smartphone 1 are wirelessly connected as described above. Thereby, the image file of the image photographed with the digital camera 2 is transmitted to the smartphone 1, and the image file related to the image file received by the smartphone 1 connected to the image search site 10 is retrieved. can do. A cloud service can be used as the image search site 10.

  FIG. 3 is a schematic block diagram showing the configuration of the digital camera 2. Reference numeral 240 denotes a display unit, which includes an LCD that is a display device and a drive circuit that drives the LCD. Reference numeral 254 denotes a display control unit that controls the display state of the display unit 240. An input unit 256 includes various keys such as a shutter key and switches.

  Reference numeral 250 denotes a GPS unit that measures the current position using a known GPS (Global Positioning System). In other words, the GPS unit 250 receives radio waves transmitted from a plurality of positioning satellites (GPS satellites) with an antenna, thereby acquiring position data including latitude and longitude indicating the current position, and shooting the acquired position data. The location information indicating the location is supplied to the CPU 202.

  A CPU 202 controls the entire digital camera 2. A program memory 204 stores a control program 2041, an application program 2042, and other programs 2043 in advance. The control program 2041 is a basic control program for controlling the entire digital camera, and the application program 2042 is a program for cooperating with the smartphone 1 and the service server 11. Reference numeral 258 denotes a communication I / F that performs a link with the smartphone 1 using a wireless communication device such as Bluetooth (registered trademark).

  The imaging unit 260 includes an image sensor that is driven by a driver, a lens that forms an object on the image sensor, and a unit circuit that receives analog corresponding to the optical image of the object from the image sensor. The unit circuit includes a CDS circuit that removes noise included in the output signal of the image sensor by correlated double sampling, an A / D converter that converts the image signal from which noise has been removed to a digital signal, and the like. The converted imaging signal is output to the CPU 202.

  The CPU 202 converts the imaging signal into YUV data, and in the REC through mode, the video signal is converted every time one frame of data (image data) is accumulated, and is sent to the display unit 240 to display the screen as a through image. . In the still image shooting mode, using the shutter key operation as a trigger, the CPU 202 instructs to switch from the through image shooting mode to the still image shooting mode, and the image data obtained by the shooting processing in this still image shooting mode is compressed and It is encoded and recorded in the external memory 230 as a still image file of a predetermined format.

  FIG. 4 is a schematic block diagram illustrating the configuration of the smartphone 1. In other words, reference numeral 340 denotes a display unit, which includes an LCD that is a display device and a drive circuit that drives the LCD. Reference numeral 354 denotes a display control unit that controls the display state of the display unit 340. Reference numeral 356 denotes an input unit, which includes various keys and switches.

  Reference numeral 302 denotes a CPU that controls the entire smartphone 1 and the PC 3. A program memory 304 stores a control program 3041, an application program 3042, and other programs 3043 in advance. The control program 3041 is a basic control program for controlling the entire smartphone 1, and the application program 3042 is a program for cooperating with the digital camera 2 and cooperating with the service server 11. Reference numeral 358 denotes an I / F that makes a Bluetooth (registered trademark) link with a portable terminal, and 260 denotes an I / F that makes a link with the service server 11 using Wi-Fi or the like.

  The resolution (horizontal × vertical) of the LCD included in the display unit 340 is, for example, 320 × 480 px, and information indicating the display performance of the display unit 340 such as the resolution is stored in the program memory 304 in advance. The CPU 302 can acquire the display performance of the display unit 340 stored in the program memory 304 in advance by executing processing according to the control program 3041. Therefore, the CPU 302 functions as a display performance acquisition unit that acquires the display performance of the display unit in the portable device of the present embodiment.

  Reference numeral 357 denotes an audio processing unit, which includes an audio decoding processing unit connected to a speaker and an audio code processing unit connected to a microphone. The speaker reproduces the ringing tone and received voice decoded by the voice decoding processing unit. The microphone detects the input voice and inputs it to the voice code processing unit, and the input voice signal is encoded by the voice code processing unit and transmitted to the outside as the transmitted voice.

  The smartphone 1 is also provided with a GPS unit 350, an imaging unit 370, and an external memory 330 having the same configuration as the GPS unit 250, the imaging unit 260, and the external memory 230 of the digital camera 2 described above.

Next, the operation of the present embodiment will be described. The user carries the digital camera 2 and directs the lens toward the subject and presses the shutter key to take a picture at a desired location. Then, the CPU 202 converts the imaging signal output from the imaging unit 260 into image data, and records the image data in the external memory 230 as a still image file in a predetermined format (for example, JPEG (Joint Photographic Experts Group) method). At this time, the CPU 202 records the following items as EXIF (Exchangeable Image File Format) information.
(1) Shooting position information that is the position of the digital camera 2 itself acquired by the GPS unit 250.
(2) Shooting date and time information that is the date and time of shooting that the CPU 202 is timing.
(3) Shooting condition information such as an aperture, white balance, and shutter speed set when the CPU 202 controls the imaging unit 260 and the like during shooting.

  Therefore, an image file in which the items (1), (2), and (3) are embedded in EXIF is recorded in the external memory 230 of the digital camera 2. When the digital camera 2 in which such an image file is stored in the external memory 230 and the smartphone 1 are in a state in which wireless communication can be performed with each other, the smartphone 1 follows the program stored in the program memory 304 according to the program in FIG. Processing is executed as shown in the flowchart.

  That is, the CPU 302 of the smartphone 1 reads an image file recorded in the external memory 230 through communication with the digital camera 2, reproduces the read image file, and displays a captured image on the display unit 340 (step 340). S1). Next, it is determined whether or not a search instruction by a predetermined operation at the user input unit 356 has been detected (step S2). If a search instruction is detected, the communication I / F 360 is activated and connected to the service server 11 of the image search site 10 via the wireless communication unit and the communication network (step S3). The shooting position information embedded in the image displayed on the display unit 340 in the step S1 is read out, and this is used as a search key to request the service server 11 to search for an image (step S4).

  Then, the service server 11 receives the shooting position information transmitted from the smartphone 1, searches the database for an image file of an image shot at the same position as the shooting position indicated by the received information, and transmits the image file to the smartphone 1. To do. The smartphone 1 acquires an image transmitted from the service server 11 as a search result, and stores it in a work area of a RAM included in the CPU 302 (step S5). Then, based on the number of received image files, it is determined whether or not the number of images in the search result is “0” (step S6). If the number of images as a search result is “0”, the display unit 340 displays that the search result is 0 together with the photographed image (step S7), and the process according to this flowchart ends.

  If the number of images in the search result is not “0”, whether or not there is an image obtained as a search result that is greater than or equal to a predetermined display size, greater than or equal to the data amount, less than the predetermined display size, or less than the data amount. Is determined (step S8). If there is an image obtained as a search result that is larger than the predetermined display size, larger than the data amount, smaller than the predetermined display size, or smaller than the data amount, among the images stored in the RAM in step S5 Exclusion processing such as adding an identifier indicating exclusion to an image corresponding to these is executed (step S9).

  Therefore, images that are difficult to display on the display unit 340 are excluded in advance in the processing in step S9, and thus the searched images can be displayed appropriately.

  Subsequently, it is determined whether or not the number of images after the exclusion process has become “0” (step S10). When the number of remaining images becomes “0”, the number of images searched and hit together with the captured image is displayed on the display unit 340 (step S11). At this time, in step S7 described above, the search result “0” is displayed, but in step S11, the number of images hit by the search is displayed. Therefore, in both steps S7 and S11, the searched image is not displayed, but the user can know that the image could not be searched by the display in step S7, and the image search can be performed by the display in step S11. Although it was possible, it can be known that an image cannot be displayed due to the display performance of the display unit 340 of the smartphone 1.

  Therefore, when the display in step S11 is performed, the CPU 302 of the smartphone 1 determines whether or not a storage instruction by an operation by the user with the input unit 356 is detected (step S12). If a save instruction is detected, an image file in which the URL of the search result image file is described in the EXIF comment tag is generated (step S13). That is, an image file including the hit image is generated, and the URL of each image file is described in the EXIF comment tag.

  Here, the search result image file used in step S13 is excluded in step S9 described above for reasons such as being difficult or impossible to display on the display unit 340 of the smartphone 1. However, even if display is difficult or impossible on the display unit 340 of the smartphone 1, it can be displayed on the PC 3. Therefore, if the image file is generated in step S13, the image file can be sent to the PC 3 via the Internet 500 at a later date and can be viewed on the PC 3. In addition, each URL can be accessed from the PC 3 via the Internet 500 to view or acquire the original image.

  On the other hand, if the determination in step S10 is NO and the number of images after the exclusion process is not “0”, it is further determined whether or not the number of image files after the exclusion process is plural (step S14). . If the determination in step S14 is NO and the number of image files after the exclusion process is “1”, the captured image and the search result image are displayed on the display unit 340 so that they can be identified. (Step S15). As a result, as illustrated in FIG. 6A, the captured image A based on the captured image file is larger on the display unit 340 of the smartphone 1, and the related image a based on the retrieved image file is larger than the captured image A. Is also displayed small. Therefore, the user can know from this display form that there is only one image file remaining after the exclusion process, and that the image a is displayed smaller than the image A.

  After that, it is determined whether or not a save instruction by an operation by the user with the input unit 356 is detected (step S16). If the save instruction is detected, the process proceeds to step S21 to be described later.

  Note that the display form in step S15 is not limited to that illustrated in FIG. 6A, and the captured image A and the related image a are visually displayed, for example, by adding different color frames to the captured image A and the related image a. Any display form may be used as long as it can be identified.

  If the determination in step S14 is YES and the number of image files after the exclusion process is plural, only the number of images used for multi-display −1 is selected from these plural image files (step S17). That is, if the number of images used for multi-display in this embodiment is “9”, “9-1 = 8”, so “8” image files are selected from the image files after the exclusion process. . Thereby, the image display space for one sheet for displaying a picked-up image can be ensured in the display part 340. FIG.

  When the number of image files after exclusion processing is less than or equal to the number of images used for multi-display, for example, when the number of images used for multi-display is “9” and the number of image files after exclusion processing is “6”. Selects all of these “6” excluded image files.

  Next, a thumbnail image based on the display performance of the display unit 340 is generated from the selected plurality of image files (step S18). As described above, information indicating the display performance of the display unit 340 such as the resolution is stored in the program memory 304 in advance, and a thumbnail image based on this display performance is generated. Therefore, it is possible to obtain an image suitable for the environment of its own terminal from images obtained by searching the network.

  Thereafter, the thumbnail images generated centering on the photographed image are displayed in a multi display (step S19). Therefore, assuming that there are eight thumbnail images, the photographed image A is displayed at the center of the display unit 340 as shown in FIG. The eight thumbnails a to h that are the images thus displayed are displayed. As a result, the user can view and compare the captured image A and the thumbnails a to h.

  In addition, since the captured image A and the thumbnails a to h are images having the same capturing position, it is highly possible that the captured image A and the thumbnails a to h are captured at the same sightseeing spot, for example. Therefore, the photographed image A and the thumbnails a to h are highly likely to be a set of images obtained by photographing the landscape of the same tourist spot in different directions and images taken of buildings of the same tourist spot from different directions. This makes it possible to improve the interest in image display.

  Subsequently, it is determined whether or not a save instruction by an operation by the user with the input unit 356 has been detected (step S20). If a save instruction is detected, an image file in MPF (Multi-Picture Format) format is generated with the captured image as the main image and the related image as the sub-image (step S21). At this time, the URL of the image file of the search result is described in the EXIF comment tag.

  Thereafter, the generated MPF image file is stored in the external memory 330 and stored (step S22), and the process is terminated. Therefore, an MPF format image file in which the captured image is a main image and a related image is a sub-image is recorded in the external memory 330, which can be reproduced and viewed later on the PC 3.

(Other embodiments)
[1] In the above-described embodiment, a case where nine images including the main image are displayed in a multi-display is shown. However, the number of images to be displayed in a multi-display is the image remaining after the exclusion process in step S10. It may be determined according to the number of sheets. For example, the greater the number of images remaining after the exclusion process, the greater the number of images to be displayed in multi display.
In this way, it is possible to display a larger number of searched images, that is, images related to the captured image, in a limited display area.

  In addition, it is not necessary to display a photographed image and related images (related images) on the same screen. For example, a photographed image is displayed on the first page, and one or a plurality of related images are displayed in association with each other. You may make it do.

[2] In the embodiment, the related image is searched using the photographed position as a search key. However, the position information of the main subject included in the photographed image is acquired, and the position information is used as related information. The image may be searched. In this case, the main subject is extracted from the captured image using image recognition technology, and information indicating the main subject is sent from the smartphone 1 to the service server 11. Then, after detecting the position of the sent subject on the service server 11 side, an image file having a main subject at the same position is searched and transmitted to the service server 11 side. Therefore, the image file of the main subject having the same position as the photographed main subject is sent from the service server 11 to the smartphone 1.
Here, since the main subject of the photographed image is a subject that the user of the digital camera 2 is interested in as a photographing target, the image that the user is interested in can be collected and displayed.

  [3] It should be noted that the main subject itself may be used as a search key instead of the position information of the main subject as a search key. In this case as well, the main subject is extracted from the captured image using image recognition technology, and information indicating the main subject is sent from the smartphone 1 to the service server 11. Then, on the service server 11 side, an image file having the same main subject as the sent subject is searched and transmitted to the service server 11 side. Therefore, an image file having the same subject as the photographed main subject as the main subject is sent from the service server 11 to the smartphone 1.

  Accordingly, in this case as well, as in the case of [2], the main subject of the photographed image is a subject that the user of the digital camera 2 is interested in as a photographing target. Can be displayed.

  [4] In the above [3], the main subject may be a human face. In this case, since the person who is the main subject of the photographed image is a person who is interested in the user of the digital camera 2, a famous person, an entertainer, etc., an image of the person who is interested in the user is displayed. Can be collected and displayed.

  [5] An image may be searched using the date and time of shooting as a search key. In this case, for example, when a rainbow or tornado occurs at a certain date and time and the user shoots it with the digital camera 2, images obtained by shooting the same rainbow or tornado from different shooting positions and shooting directions are searched and displayed in multiple. become. Therefore, it is possible to grasp the rainbow and tornado including the geographical situation from a plurality of images obtained by photographing the same rainbow and tornado from different positions and directions.

  [6] The image may be searched by using the shooting conditions such as aperture, white balance, shutter speed, etc. as search keys. In this case, although the scene is completely different, an image having the same shooting condition as that of the shot image is searched and displayed in multiple. Therefore, even if the shooting conditions are the same, it is possible to learn how the captured image is expressed if the scene is different, thereby contributing to improvement of the user's shooting technique.

  [7] In the embodiment, the digital camera 2 captures an image and transmits the captured image to the smartphone 1. However, since the smartphone 1 itself has the imaging unit 370, the smartphone 1 captures an image. You may make it perform the image search in the service server 11. FIG. In this way, similar related images can be displayed without using the digital camera 2.

[8] In the embodiment, the captured image data is transmitted from the digital camera to the smartphone 1 wirelessly such as Bluetooth (registered trademark). However, the image data is transmitted via a communication network or a wireless LAN. May be. In this case, the information device on the receiving side, such as the smartphone 1 or the mobile phone, has a name or a mail address, FaceBook, Flickr, Mixi, etc. for specifying the user (transmission user) of the other device that has transmitted the captured image data. Acquire specific information of the sending user such as user IDs, nicknames, terminal device addresses, URLs of blogs, personal homepages, etc. for SNS (Social Networking Service) and image sharing services, etc. Based on the specific information of the sending user, a server on the web, an image database, or a site of a predetermined SNS or image sharing service is searched, and an image (such as a face photo or an avatar) related to the sending user is searched. Acquire the copy of the captured image data and the image data related to the sending user, Create and record a new image data file.
Alternatively, an MPF image data file is generated, stored, and recorded using the captured image data as main image data and the transmission user image data as sub-image data.

  [9] Furthermore, in [8], not only the location information and user identification information related to the transmitting user but also the location information and user identification information related to the receiving user are acquired or stored in advance, and the captured image is mainly stored. Using the image city, the transmission user image, the image around the shooting location (or transmission location), the reception user image, the image around the reception location, etc. as sub-images, a file of MPF image data is generated, or the image data is Encoded and stored.

  [10] or image data transmitted with a Dane mail attachment, or image data transmitted or transmitted from a friend or member of an SNS or image sharing service such as FaceBook, Flickr, Mixi, etc. In accordance with the service and type, or according to the setting information set by the user for each service or each address, the related information in a predetermined format is extracted and acquired, and based on this, the predetermined related image data is searched or acquired. Then, the received (or transmitted) captured image data is used as a main image, and the additional acquired image data is used as a sub-image, or an MPF image data file is generated, or they are included in one image. An image in which a plurality of images are laid out is generated, and the image is encoded and stored and recorded.

As mentioned above, although embodiment of this invention was described, this invention is not limited to these, The invention described in the claim and its equal range are included.
The claims appended at the time of filing this application will be appended below.

<Claim 1>
A captured image acquisition means for acquiring a captured image;
Communication means;
An instruction detection unit that connects to an external communication network via the communication unit and detects a search instruction for a related image of the captured image acquired by the captured image acquisition unit;
As a result of the search instruction by the instruction detection unit, a related image acquisition unit that acquires a related image via the external communication network;
Processing means for processing so that the related image acquired by the related image acquisition means can be reproduced in association with the captured image;
An image processing apparatus comprising:

<Claim 2>
The related image acquired by the related image acquisition means further comprises a determination means for determining whether the output size or the data amount is within a predetermined range,
The image processing apparatus according to claim 1, wherein the processing unit processes the related image determined to be positive by the determination unit so that the reproduction unit can reproduce the related image.

<Claim 3>
The reproduction means includes display means,
It further comprises display performance acquisition means for acquiring the display performance of this display means,
The image processing apparatus according to claim 1, wherein the processing unit processes the related image based on the display performance acquired by the display performance acquisition unit.

<Claim 4>
The image processing apparatus according to claim 3, wherein when the related image acquisition unit acquires a plurality of the related images, the processing unit processes the acquired plurality of related images so that they can be multi-displayed.

<Claim 5>
4. The method according to claim 3, wherein when the related image acquisition unit acquires a plurality of the search results, the processing unit processes the related image so that multi-display according to the acquired number can be performed. Image processing device.

<Claim 6>
The image processing apparatus according to claim 4, wherein the processing unit processes the captured image and the related image so that multi-display can be performed after adding the captured image to a display layout.

<Claim 7>
Shooting position information indicating the shooting position is associated with the shot image,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the photographed image based on the photographing position information.

<Claim 8>
Further comprising location information acquisition means for acquiring location information of the main subject included in the captured image,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the captured image based on the location information.

<Claim 9>
The shooting image is associated with shooting date / time information indicating the shooting date / time,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the captured image based on the shooting date and time information.

<Claim 10>
2. The main subject itself included in the captured image is information related to the captured image, and the instruction detection unit detects a search instruction for a related image of the captured image based on the information. 7. The image processing device according to any one of items

<Claim 11>
The image processing apparatus according to claim 10, wherein the main subject is a human face.

<Claim 12>
The captured image is associated with shooting condition information indicating a shooting condition,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the photographed image based on the photographing condition information.

<Claim 13>
The imaging apparatus according to claim 1, wherein the processing unit processes data in a file format in which a main image is the captured image and a sub-image is the related image.

<Claim 14>
An image processing method using an apparatus having a communication unit and a reproduction unit,
An instruction detection step of connecting to an external communication network via this communication means and detecting a search instruction for a related image for the captured image acquired in advance,
As a result of the search instruction in this instruction detection step, a related image acquisition step of acquiring a related image via the external communication network;
A processing step for processing the related image acquired in the related image acquisition step so that the related image can be reproduced in association with the captured image;
An image processing method comprising:

<Claim 15>
A computer included in an apparatus including a communication unit and a reproduction unit,
An instruction detection unit for connecting to an external communication network via the communication unit and detecting a search instruction for a related image for the captured image acquired in advance;
As a result of the search instruction in this instruction detection step, related image acquisition means for acquiring a related image via the external communication network,
Processing means for processing the related image acquired by the related image acquisition means so as to be reproduced in association with the captured image;
A program characterized by functioning as

1 Smartphone 2 Digital camera 3 PC
10 Image Search Site 11 Service Server 202 CPU
230 External Memory 240 Display Unit 250 GPS Unit 260 Imaging Unit 302 CPU
304 Program memory 330 External memory 340 Display unit 350 GPS unit 356 Input unit 360 Imaging unit 360 Communication I / F
500 Internet 2041 Control Program 2042 Application Program 3041 Control Program 3042 Application Program

Claims (15)

A captured image acquisition means for acquiring a captured image;
Communication means;
An instruction detection unit that connects to an external communication network via the communication unit and detects a search instruction for a related image of the captured image acquired by the captured image acquisition unit;
As a result of the search instruction by the instruction detection unit, a related image acquisition unit that acquires a related image via the external communication network;
Processing means for processing so that the related image acquired by the related image acquisition means can be reproduced in association with the captured image;
An image processing apparatus comprising: The related image acquired by the related image acquisition means further comprises a determination means for determining whether the output size or the data amount is within a predetermined range,
The image processing apparatus according to claim 1, wherein the processing unit processes the related image determined to be positive by the determination unit so that the reproduction unit can reproduce the related image. The reproduction means includes display means,
It further comprises display performance acquisition means for acquiring the display performance of this display means,
The image processing apparatus according to claim 1, wherein the processing unit processes the related image based on the display performance acquired by the display performance acquisition unit.   The image processing apparatus according to claim 3, wherein when the related image acquisition unit acquires a plurality of the related images, the processing unit processes the acquired plurality of related images so that they can be multi-displayed.   4. The method according to claim 3, wherein when the related image acquisition unit acquires a plurality of the search results, the processing unit processes the related image so that multi-display according to the acquired number can be performed. Image processing device.   The image processing apparatus according to claim 4, wherein the processing unit processes the captured image and the related image so that multi-display can be performed after adding the captured image to a display layout. Shooting position information indicating the shooting position is associated with the shot image,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the photographed image based on the photographing position information. Further comprising location information acquisition means for acquiring location information of the main subject included in the captured image,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the captured image based on the location information. The shooting image is associated with shooting date / time information indicating the shooting date / time,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the captured image based on the shooting date and time information.   2. The main subject itself included in the captured image is information related to the captured image, and the instruction detection unit detects a search instruction for a related image of the captured image based on the information. 6. The image processing device according to any one of items 6 to 6.   The image processing apparatus according to claim 10, wherein the main subject is a human face. The captured image is associated with shooting condition information indicating a shooting condition,
The image processing apparatus according to claim 1, wherein the instruction detection unit detects a search instruction for a related image of the photographed image based on the photographing condition information.   The imaging apparatus according to claim 1, wherein the processing unit processes data in a file format in which a main image is the captured image and a sub-image is the related image. An image processing method using an apparatus having a communication unit and a reproduction unit,
An instruction detection step of connecting to an external communication network via this communication means and detecting a search instruction for a related image for the captured image acquired in advance,
As a result of the search instruction in this instruction detection step, a related image acquisition step of acquiring a related image via the external communication network;
A processing step for processing the related image acquired in the related image acquisition step so that the related image can be reproduced in association with the captured image;
An image processing method comprising: A computer included in an apparatus including a communication unit and a reproduction unit,
An instruction detection unit for connecting to an external communication network via the communication unit and detecting a search instruction for a related image for the captured image acquired in advance;
As a result of the search instruction in this instruction detection step, related image acquisition means for acquiring a related image via the external communication network,
Processing means for processing the related image acquired by the related image acquisition means so as to be reproduced in association with the captured image;
A program characterized by functioning as

Images