JP2009188870A - Image reproduction system and image reproduction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reproduction device and an image reproduction method for widening the applicable range of an image, and for generating an image in a short time, and for giving a feeling of expectation or pleasure to a user. <P>SOLUTION: An image reproduction system includes: an image recording part 2 for recording image data corresponding to a plurality of photographic images; an image selection part 3 for selecting an image corresponding to a representative image from among the image data recorded in the image recording part 2; a similar image retrieval part 8a for retrieving a content package including the image data corresponding to the image similar to the representative image; and a slide show part 6 for reproducing the image corresponding to the image data included in the content package retrieved by the similar image retrieval part 8a as part of the image configuring a slide show with the plurality of photographic images at a display part 5. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image reproduction device and an image reproduction method, and more particularly, to an image reproduction system and an image reproduction method for enjoying an image taken by an imaging device such as a digital camera.

  In recent years, imaging devices such as digital cameras have become widespread. When enjoying an image captured by the imaging apparatus, not only printing but also viewing on a screen of a personal computer (PC), a mobile phone, a television, or the like can be easily performed. In these devices, as a method of appreciating a photographed image, a slide show technique for sequentially reproducing images on a screen is performed in the same manner as sequentially viewing printed photographs. However, a slide show is a situation that is not so popular at present because it takes time to finish watching all of the images simply by sequentially switching the images and is monotonous.

In order to increase the added value to the slide show technique as described above, Patent Document 1 discloses a slide show reproduction method in which group photos of face images constituting a slide show are combined and inserted into the slide show. That is, in this reproduction method, an order is created from representative images of a group with a large number of face images, and a group scene is generated by determining the image size, and this group scene is inserted into the last scene of the slide show. .
JP 2007-249371 A

  The slide show reproduction method described in Patent Document 1 cannot be applied to images other than face images. In addition, it is necessary to newly generate a representative image, and it takes time and effort to generate the representative image. Furthermore, the representative image is limited to a prediction range of “a group photo of face images”, and does not give the user a sense of excitement and enjoyment.

  The present invention has been made in view of such circumstances, and has a wide range of applicable images, can be generated in a short time, and can provide a user with a sense of expectation and enjoyment. An object of the present invention is to provide an image reproduction method.

  In order to achieve the above object, an image reproduction system according to a first aspect of the present invention corresponds to a recording unit in which image data corresponding to a plurality of images is recorded, and a representative image among the image data recorded in the recording unit. A representative image selection unit for selecting an image, a search unit for searching for a content package including image data corresponding to an image similar to the representative image, and an image corresponding to the image data included in the content package searched by the search unit And a slide show playback unit that plays back the plurality of images as part of the images constituting the slide show.

The image playback system according to a second invention is the image playback system according to the first invention, wherein the slide show playback unit uses the head image included in the content package searched by the search unit as a part of images constituting the slide show. Play back images.
In the image reproduction system according to a third aspect of the present invention, in the first aspect, the slide show reproduction unit uses the jacket included in the content package searched by the search unit as a part of the images constituting the slide show. Play the image.

In the image reproduction system according to a fourth aspect of the present invention, in the first aspect, the slide show reproduction unit uses the image corresponding to the image data included in the content package searched by the search unit as a head image, and the plurality of images. The music data included in the content package is played back while playing back.
According to a fifth aspect of the present invention, in the image reproduction system according to the first aspect, the slide show reproduction unit includes a part of an image constituting the slide show with an image corresponding to the image data included in the content package searched by the search unit. The music data included in the content package is reproduced while reproducing the plurality of images.

  According to a sixth aspect of the present invention, there is provided an image reproduction method for selecting image data corresponding to a representative image from a recording unit in which image data corresponding to a plurality of images is recorded, and corresponding to an image similar to the representative image. Searching for a content package that includes image data to be played, and replaying the plurality of images with an image corresponding to the image data included in the content package searched by the search unit as a part of an image constituting a slide show, .

  An image reproduction method according to a seventh aspect of the present invention is a content package including image data corresponding to an image similar to a representative image included in a plurality of images from a recording unit in which image data corresponding to the plurality of images is recorded. And a playback unit that plays back the content included in the searched content package on the same device as the device that plays back the plurality of images.

  According to an eighth aspect of the present invention, there is provided an image reproduction method including a content package including image data corresponding to an image similar to a representative image included in a plurality of images from a recording unit in which image data corresponding to the plurality of images is recorded. And a step of reproducing the content included in the searched content package on the same device as the device that reproduces the plurality of images.

  An image display method according to a ninth aspect of the present invention is a content package including content data corresponding to data related to a representative image included in a plurality of images from a recording unit in which image data corresponding to the plurality of images is recorded. And a step of reproducing the content included in the searched package on the same device as the device that reproduces the plurality of images.

  An image reproduction server according to a tenth aspect of the invention receives a representative image designated from captured images, temporarily stores the representative image, an image storage unit that stores content images, and the above A search unit that compares the representative image with the image stored in the image storage unit and searches for an auxiliary image similar to the representative image; and a communication unit that outputs the auxiliary image searched by the search unit. .

  An image reproduction server according to an eleventh aspect of the present invention is the server according to the tenth aspect, further comprising an image processing unit that detects and removes characters from the content image, and the search unit removes characters by the image processing unit. The auxiliary image is searched using the obtained image.

According to a twelfth aspect of the present invention, there is provided an image reproduction server according to the tenth aspect, wherein a photographed image storage unit that receives and records a photographed image and a representative image storage unit that stores a representative image designated from the photographed images. And a sequential playback unit for reproducing and displaying the representative image after the auxiliary image is reproduced and displayed, and a transmission unit for transmitting images ordered by the sequential playback unit. .
In the image reproduction server according to a thirteenth aspect, in the twelfth aspect, the sequential reproduction unit reproduces and displays the auxiliary image at the beginning or end of the slide show.
According to a fourteenth aspect of the present invention, in the image reproduction server according to the twelfth aspect, the image accumulation unit further accumulates music related to the image of the content. Send the data.
According to a fifteenth aspect of the present invention, in the image reproduction server according to the twelfth aspect, the image accumulating unit further accumulates a moving image related to the image of the content. Later, the video data is transmitted.

  An image reproduction server according to a sixteenth aspect of the present invention compares an image storage unit that stores content images with a plurality of captured images and images stored in the image storage unit, and uses an image with a high degree of similarity as an auxiliary image. A search unit for searching, and a communication unit for outputting the auxiliary image searched by the search unit.

  An image reproduction program according to a seventeenth aspect of the present invention compares a representative image designated from photographed images with a content image, searches for an auxiliary image similar to the representative image, and displays a slideshow when displaying the slideshow. After the auxiliary image is reproduced and displayed, the representative image is reproduced and displayed.

  An image reproduction program according to an eighteenth aspect of the invention designates a representative image from photographed images, compares the designated representative image with a content image, and searches for an auxiliary image similar to the representative image. The representative image is transmitted to an external device, the auxiliary image is received from the external device, and the auxiliary image is reproduced and displayed when the slide show is reproduced and displayed, and then the representative image is reproduced and displayed.

  ADVANTAGE OF THE INVENTION According to this invention, the range of the image which can be applied is wide, can produce | generate in a short time, and can provide the image reproduction apparatus and image reproduction method which can give expectation and pleasure to a user.

  A preferred embodiment will be described below using an image reproduction system to which the present invention is applied according to the drawings. FIG. 1 is a block diagram showing a configuration of an image reproduction system including a camera 10, a personal computer 20, and a television 30 according to the first embodiment of the present invention.

  The camera 10 is a digital camera that captures a subject and acquires an image. The personal computer 20 can be connected to the camera 10 and stores image data of a jacket such as a music CD as will be described later. The television 30 can be connected to the camera 10 and reproduces and displays the slide show based on the image data of the slide show generated in the camera 10. The connection between the camera 10 and the personal computer 20 includes not only a wired connection using a cable or the like, a wireless connection such as radio waves or infrared rays, but also an exchange via a storage medium such as a USB memory.

  The camera 10 includes a photographing unit 1, an image recording unit 2, an image selection unit 3, a communication and control unit 4, a display unit 5, a slide show unit 6, a sequence control unit 7, a similar image search unit 8, a face detection character detection and processing unit. 8 a and an auxiliary content recording unit 9. The communication and control unit 4 performs overall control of the camera 10 and performs communication when connected to the personal computer 20 and the television 30.

  The photographing unit 1 includes an optical system for forming a subject image, an image sensor that photoelectrically converts the subject image, an image processing unit that processes a signal output from the image sensor, and the like, and outputs image data. The image recording unit 2 records the image data output from the photographing unit 1. The display unit 5 performs display such as reproduction display of an image recorded in the image recording unit 2. The image selection unit 3 selects a representative image in response to an instruction from the operation member by the photographer.

  The similar image search unit 8 searches for an auxiliary image similar to the representative image instructed by the image selection unit 3 from images recorded in the personal computer 20 or the like. In the present embodiment, a photographed image is compared with an image of a content jacket such as a music CD, and an image similar to the photographed image is searched.

  The face detection character detection and processing unit 8a performs face detection and character removal processing from the image. Content jackets often include characters such as labels, publisher names, as well as images, titles, authors and artist names. Therefore, the character detection unit in the face detection character detection and processing unit 8a removes characters and the like from the image, and the similar image search unit 8 compares the photographed image of the jacket such as a music CD with the photographed image taken. An image similar to the image (that is, an auxiliary image) is searched.

  The auxiliary content recording unit 9 records the auxiliary image searched by the similar image search unit 8. Also, information related to the auxiliary image, for example, music (audio data) recorded in content such as a music CD, text data of lyrics of the music, and the like are also recorded.

  The slide show unit 6 causes the display unit 5 to perform a slide show by combining the representative image or the captured image recorded in the image recording unit 2 with the auxiliary image recorded in the auxiliary content recording unit 9. The order control unit 7 controls the order of reproduction and display of images when performing a slide show.

  The personal computer 20 includes a communication and control unit 24 and a content jacket recording unit 21. The communication and control unit 24 performs overall control of the personal computer 20, and controls communication when connected to the communication and control unit 4 of the camera 10. The content jacket recording unit 21 stores information related to content such as LP images, content jacket images such as music CDs and DVDs, book cover images, and music and lyrics. That is, the content includes various things such as existing comics, anime, television programs, movies, music, etc., and the content jacket recording unit 21 stores images and related information related to these contents. Yes.

  In the present embodiment, the camera 10 acquires these pieces of information from the content jacket recording unit 21 via the communication and control unit 4 and the communication and control unit 24 of the personal computer 20. However, in addition to this method, it may be obtained from an external server or the like from the personal computer 20 via a network such as the Internet, or the camera 10 directly accesses an external server or the like via a network such as the Internet. You may make it acquire such information. In addition to this method, a recording medium on which such information is recorded, such as a memory card, may be used.

  Next, the operation in this embodiment will be described with reference to FIG. First, as shown in FIG. 2A, shooting is performed using the camera 10. Subsequently, the camera 10 is connected to the personal computer 20 as shown in FIG. At this time, the user selects a representative image while viewing the captured image reproduced on the display unit 8. The similar image search unit 8 or the like of the camera 10 searches for an auxiliary image similar to the representative image, and records the searched auxiliary image in the auxiliary content recording unit 9.

  The images taken by the camera 10 can be enjoyed in a normal frame-by-frame playback display or slide show on the display unit 5 of the camera 10, but are connected to the television 30 as shown in FIG. You can enjoy watching with many people. In addition, when playing back and displaying on the television 30, as shown in FIG. 2D, the content (auxiliary image) that everyone knows is played back and displayed as an introductory image, and preparations are made for the beginning. I can get it.

  The slide show shown in FIG. 2 (d) does not play back only one frame of the shot image, but when a plurality of shot images are selected and viewed as a slide show, a music CD or the like retrieved as auxiliary content (auxiliary image) is displayed. Display a combination of jackets. That is, as shown in FIG. 2D, the jacket image 110 similar to the captured image 100 is displayed before the captured image 100 as shown in FIG. By displaying such auxiliary content (in this case, the jacket image 110), it is possible to attract the viewer's interest rather than merely displaying the captured image 100.

  If the music recorded on this music CD is used as BGM, it can be used to further enhance the atmosphere. That is, the music on the music CD that is the source of the auxiliary image is the same jacket music as the atmosphere of the photographed image, so it often has a natural and harmonious feeling, and it is possible to save the trouble of selecting the music. .

  Further, when searching for similar images using the jacket image 110 of the music CD jacket as shown in FIG. 3A as a search target, the character portion is removed as shown in FIG. Only the portion 111 is extracted. Then, the photographed image 100 as shown in FIG. 3C is compared with the image portion 111 as shown in FIG. 3B, and an image similar to the photographed image 100 is searched.

  When searching for similar images, information on the position and size of the face in the screen may be used. In recent years, in the field of cameras, research on techniques for specifying the position of a face from within a screen has been performed in order to adjust the focus and exposure, and such techniques may be applied.

  Next, the operation of the camera 10 in the present embodiment will be described using the flowchart shown in FIG. When the camera control flow is entered, it is first determined whether or not the playback mode is set (S1). The camera 10 can use two modes, a playback mode and a shooting mode. If the playback mode is not set as a result of the determination, it is determined that the mode is a shooting mode, and then the operation state of the release button is detected to determine whether or not to perform shooting (S2).

  As a result of the determination, if it is not shooting, the process returns to step S1, whereas if it is shooting, a shooting operation is executed (S3). In the photographing operation, the photographing unit 1 acquires image data of a subject image and records the image data in the image recording unit 2. When the photographing operation ends, the process returns and returns to step S1.

  If the result of determination in step S1 is playback mode, the operating state of the playback button or the like is detected and it is determined whether or not playback is to be performed (S11). As a result of the determination, if reproduction is to be performed, the process proceeds to step S12 and subsequent steps, and the reproduction mode is executed. In the playback mode in this embodiment, in addition to normal playback display and normal slide show, playback display by a slide show combined with auxiliary content is possible.

  Next, it is determined whether to perform database (DB) communication (S13). When the user of the camera 10 obtains an auxiliary image as auxiliary content, the user performs an operation to that effect. In this step, it is determined whether or not the operation has been performed. If DB communication is performed as a result of the determination, it is subsequently determined whether or not to search for similar images (S21). The determination here is whether or not the user has started a search start operation.

  If the result of determination in step S21 is that search has not started, processing returns to step S1. On the other hand, when the search is started, a representative image is designated (S22). In this step, the captured images recorded in the image recording unit 2 are sequentially reproduced and displayed on the display unit 5, and the user designates a representative image from the reproduced and displayed captured images according to his / her preference.

  When the designation of the representative image is completed, an image similar to the selected representative image is searched for (S23). The subroutine for determining the degree of coincidence for searching for similar images will be described later with reference to FIG. Subsequently, with respect to the representative image designated in step S22, the degree of image coincidence with all target images is determined, and it is determined whether or not the processing has been completed (S25). As a result of the determination, if it is not the end, the target image is changed (S27), the process returns to step S23, and the image matching degree is determined for the target image changed in step S27.

  If it is determined in step S25 that the determination of the degree of coincidence with all target images has been completed, an image having the maximum degree of coincidence is then acquired (S26). Since the degree of coincidence is digitized for each target image in the image coincidence degree determination subroutine in step S23, the target image having the highest degree of coincidence among the target images for which the degree of coincidence has been determined is determined in this step. Acquired and used as an auxiliary image. When the matching degree maximum image is acquired, the process returns and is executed from step S1. Note that the representative image that is reproduced and displayed together with the auxiliary image in the slide show is normally one frame, but when there are a plurality of representative images, the selection of the representative image is repeated a plurality of times.

  If the result of determination in step S13 is not DB communication, it is next determined whether or not it is a slide show (S14). If the result of determination is not a slide show, normal playback display, for example, playback display of the selected image from thumbnail playback is performed (S15). On the other hand, in the case of a slide show, by executing the steps after step S31, the auxiliary image (the image acquired in step S26) is reproduced as if it were an opening screen of the slide show, and then the representative image is reproduced and displayed. Then, sequentially, the captured images are reproduced and displayed in the form of a slide show according to the time series of the captured images.

  In the slide show, first, it is determined whether or not an image has been received (S31). That is, in step S13, DB communication is performed, and it is determined whether or not an auxiliary image has been acquired in step S26.

  As a result of the determination, if the image has been received, it is determined whether or not to perform external output (S32). In the case where a television 30 is connected to the camera 10 and a large number of people are watching as shown in FIGS. 2C and 2D, in this step, the slide show image data is externally output to an external device such as the television 30. Do. As a result of the determination, in the case of external output, the image data such as an auxiliary image, a representative image, and a captured image is output from the communication and control unit 4 to an external device such as the television 30. When the external image output ends, the process returns and the process is executed from step 1.

  If the result of determination in step S32 is that there is no external output, auxiliary images are reproduced (S33), and then representative images are reproduced (S34). When the reproduction of the representative image is completed, a slide show of the captured images is sequentially performed in order of the shooting date and time (S35).

  As described above, prior to the normal slide show, the auxiliary image similar to the representative image is reproduced and displayed, and then the representative image is reproduced and displayed. That is, as shown in FIG. 2D, the auxiliary image is reproduced first to enhance the feeling of starting the slide show, and then the reproduction of the captured image is started as shown in FIG. The image that appears after the auxiliary image is a representative image from which a similar image is retrieved. By displaying the representative image, the viewer understands why the auxiliary image is displayed. The order control unit 7 in FIG. 1 performs such rearrangement.

  If the result of determination in step S31 is that the image has not been received, the auxiliary image has not been searched, so steps S33 and S34 are skipped, and the time-series slide show in step S35 is executed. When the time-series slide show ends, the process returns and is executed from step S1.

  Next, the image matching degree determination subroutine in step S23 will be described with reference to the flowchart shown in FIG. When determining the degree of image coincidence, it should be noted that the DB image stored in the database (DB) such as the content jacket recording unit 21 has a different aspect ratio from the photographed image.

  For example, it is difficult to directly compare the degree of coincidence between a square image such as an LP record or a music CD jacket and a 4: 3 image of a normal digital camera. Therefore, in the present embodiment, the comparison is made after cutting out the central portion of the photographed image in accordance with the aspect ratio of the DB image. In addition, the similarity between the images is compared within the cut-out range, the face position is detected, and the comparison is performed using the image from which characters are removed.

  When the image matching degree determination subroutine is entered, first, the number and positions of the faces of the representative image designated in step S22 are determined (S101). Subsequently, based on the determination result in step S101, it is determined whether or not there is a face in the representative image (S102). As a result of the determination, if there is a face, an image having a face is selected from images recorded in the content jacket recording unit 21 as a database (DB) (S103). In this step, one target image is selected from a large number of images. In step S27 described above, the next target image is selected. In step S25, it is determined whether the processing has been completed for all target images.

  Subsequently, the image is compared with the representative image within a range that matches the aspect ratio of the image recorded in the content jacket recording unit 21 (S104). Then, an image having the same face position and the same number of faces is selected within the range in which the aspect ratio is taken into consideration with the image recorded in the content jacket recording unit 21 (S105).

  If the result of determination in step S102 is that there is no face in the representative image, an image recorded in the content jacket recording unit 21 and having no face is selected from the images (S111). Then, it is compared with the representative image within a range that matches the aspect ratio of the image recorded in the content jacket recording unit 21 (S112).

  When the process in step S105 or S112 is completed, it is next determined whether or not a character exists in the selected image (S106). As described above, if characters and the like are mixed in the image, the accuracy of the similarity determination is lowered. Therefore, the degree of coincidence is determined in a portion excluding the characters and the like in advance.

  If the character is present as a result of the determination in step S106, the degree of coincidence is determined in a portion other than the character portion (S107). If the character is not present in step S106, step S107 is skipped, Proceed to step S108.

  Subsequently, the degree of coincidence similar to the representative image is determined over the entire range in accordance with the aspect ratio of the DB image recorded in the content jacket recording unit 21 (S108). When the coincidence determination is completed, the process returns to the original routine.

  As described above, in the first embodiment of the present invention, before the captured image is reproduced and displayed in the slide show, the representative image and the auxiliary image similar to the representative image are reproduced and displayed. The auxiliary image is searched from the content jacket recording unit 21 in which the DB image is recorded. By reproducing and displaying the auxiliary image and the representative image, the viewer's interest can be attracted, and the user has a sense of expectation. And can give fun.

  Further, in the present embodiment, since the face image does not need to exist in the representative image, the range of applicable images is wide, and the representative image is simply specified from the captured image. It can be generated in a short time without image processing for creation.

  Furthermore, in the present embodiment, the degree of coincidence is determined within a range corresponding to the aspect ratio of the DB image, and the degree of coincidence is determined by ignoring the character portion. It is possible to determine a high degree of coincidence.

  In the present embodiment, the DB image is a cover image such as a jacket such as a music CD or a book. However, the DB image is not limited to this. For example, It may be an image taken by a third party recorded in an external server, or other various images such as illustrations and paintings.

  In this embodiment, the representative image and the auxiliary image are reproduced and displayed only at the beginning before the time-series slide show. However, the present invention is not limited to this. The representative image and its auxiliary image may be reproduced and displayed. Furthermore, in the present embodiment, the camera 10 and the personal computer 20 are linked to each other. However, the image selection unit 3, the similar image search unit 8, the face detection character detection and processing unit 8 a, the slide show unit 6, the order of the camera 10. The functions of the control unit 7 and the auxiliary content recording unit 9 may be executed by the personal computer 20 and a program installed therein.

  Next, a second embodiment of the present invention will be described with reference to FIGS. In the first embodiment, similar images are searched in the camera 10, but in the second embodiment, similar images are searched by an external device such as a server via a network such as the Internet. .

  FIG. 6 is a block diagram illustrating a configuration of an image reproduction system including the camera 10 and the server 40 according to the second embodiment. The camera 10 includes a photographing unit 1, an image recording unit 2, an image selection unit 3, a communication and control unit 4, a display unit 5, a slide show unit 6, an order control unit 7, and an auxiliary content recording unit 9. The functions of these units are the same as those of the first embodiment shown in FIG.

  The server 40 that functions as a database includes a content jacket recording unit 21, a similar image search unit 22, a face detection character detection and processing unit 23, a communication and control unit 24, and a representative image temporary storage unit 25. The communication and control unit 24 performs overall control of the server 40 and also communicates with external devices such as the camera 10 through the Internet network or the like.

  Since the similar image search unit 22 and the face detection character detection and processing unit 23 are the same as those in the first embodiment shown in FIG. It functions to detect the position and number, and to extract the character part and remove it from the image. The similar image search unit 22 searches for an image similar to the representative image from the images recorded in the content jacket recording unit 21.

  The representative image temporary storage unit 25 is a storage unit for temporarily storing the representative image designated by the image selection unit 3 of the camera 10. The similar image search unit 22 described above performs a search using the representative image stored in the representative image temporary storage unit 25.

  Thus, on the camera 10 side, the auxiliary content recording unit 9, the slide show order control unit 7, and the slide show unit 6 that sequentially reads out and reproduces data from the image recording unit 2 and the auxiliary content recording unit 9 for the slide show. The server 40 side has a function of temporarily storing a representative image sent from the camera 10 and a function of searching for a similar image. When searching for similar images, as in the first embodiment, the search is performed in consideration of whether the image is a person image, the position and number of faces, and the character portion of the auxiliary content is ignored. .

  Next, the operation in this embodiment will be described with reference to FIG. As shown in FIG. 7, the camera 10 captures the subject 11, designates a representative image 12 a from the photographed images 12 captured at this time (image No. 4 in the figure), and stores the representative image 12 a on the network 19. To the server 40 via The server 40 searches for a similar image similar to the representative image from the content jacket recording unit 21 as a database, and sets the image having the highest degree of matching as the auxiliary image 22.

  When the auxiliary image 22 is searched, the server 40 transmits the auxiliary image 22 to the camera 10. When the camera 10 starts the slide show, first, the searched auxiliary image 22 is reproduced and displayed, and then the representative image 12a is reproduced. 1-No. Four captured images 12 are reproduced.

  Further, since the auxiliary image 22 is a published content, accompanying information (A to D in the figure) such as accompanying information attached by the manufacturer to the content and user's impression and opinion is attached. In the case of a music CD, music data is also included as accompanying information. Therefore, it is possible to perform a slide show while reproducing the music recorded on the music CD along with the reproduction display of the auxiliary image 22. Further, a music piece having the same atmosphere may be searched and reproduced from other information such as a song title, composer, artist, etc. even if it is not recorded on the music CD.

  Next, the operation of the camera 10 in the present embodiment will be described using the flowchart shown in FIG. The same processing steps as those in the camera control in the first embodiment shown in FIG. 4 are given the same step numbers, and detailed descriptions thereof are omitted. In the first embodiment, the search for the auxiliary image is performed in the camera 10, but in the second embodiment, the search is performed in the server 40, and this difference will be mainly described.

  Steps S1 to S15 executed in the camera control flow are the same as those in the first embodiment. If the result of determination in step S13 is DB communication, if it is DB communication, it is determined whether or not acquisition of an auxiliary image and related information is started (S21A). This determination detects whether or not an operation member for starting acquisition (not shown) has been operated by the user.

  If the auxiliary image or the like is not acquired as a result of the determination in step S21A, the acquired information is changed to acquire another information (S29), and the process returns to step S1. In step S29, if the auxiliary image acquired in step S26A, which will be described later, or the related information is not satisfied, a condition for acquiring information is set / changed in order to further acquire information. On the other hand, if acquisition of an auxiliary image or the like is started as a result of the determination in step S21A, a representative image is designated (S22), and the designated representative image is transmitted to the server 40 (S23A). The representative image is designated by the user from the captured images reproduced and displayed on the display unit 5.

  When the transmission of the representative image ends (S25 → Yes), the image (auxiliary image) having the highest degree of coincidence and information related to the auxiliary image are acquired from the server 40 (S26A). That is, since the server 40 searches for auxiliary images and the like, the camera 10 waits for the search in the server 40 to end and the search results to be transmitted. Get related information. As related information, if the auxiliary image is an image of a music CD jacket, it is music recorded on the music CD.

  If the result of determination in step S14 is not a slide show, it is determined whether or not an image has been received (S31). In the second embodiment, the auxiliary image search is performed in the external server 40, and in this step, it is determined whether or not the auxiliary image is received from the server 40 in step S26A.

  If the result of determination in step S31 is that an image has been received, as described with reference to FIG. 7, the auxiliary image is reproduced and displayed, and for example, a music CD is reproduced using the related information (S33A). The representative image is reproduced and displayed (S35A). When the reproduction display of the auxiliary image and the representative image is finished, a normal slide show for sequentially reproducing the photographed images in order of the photographing date and time is executed (S35A).

  If the result of determination in step S31 is that no image has been received, there is no auxiliary image. Therefore, in step S35A, a normal slide show is executed, and when the normal slide show ends, the process returns and executes from step S1. .

  As described above, the camera 10 according to the second embodiment does not perform the image search, transmits the representative image to the server 40 (S24), acquires the auxiliary image retrieved from the server 40 (S26), and this auxiliary image. Is used to execute a slide show (S33 to S35). In the network communication state, an image is transmitted to the outside via the network (S23A), and further different information is acquired from the obtained acquisition information (S29). When the searched auxiliary image or the like is not liked, or when the music is not liked, it can be transmitted in step S29 that further information is acquired.

  Also in the second embodiment, as in the first embodiment, a slide show may be performed in which the television 10 can be captured by the camera 10 and an auxiliary image is reproduced and displayed. In this case, after step S31, it is determined whether to perform external output, and an image is output to the outside according to the determination result.

  Next, the operation of the server 40 in this embodiment will be described using the flowchart shown in FIG. If this flow is entered, it is first determined whether or not a representative image has been acquired (S51). As described above, since the camera 10 transmits the representative image in step S23A, it is determined whether or not the representative image is received in this step.

  When the representative image is acquired as a result of the determination in step S51, the acquired representative image is temporarily stored in the representative image temporary storage unit 25 (S52). Subsequently, the characters are removed from the target image by the face detection character detection and processing unit 23 (S53). That is, characters and the like are described in the jacket image such as a music CD recorded in the content jacket recording unit 21, and by removing these characters and the like, the accuracy of determining the degree of coincidence of the images is improved. ing.

  Subsequently, the face position of the target image is detected by the face detection character detection and processing unit 23, and the degree of coincidence between the representative image and the target image is determined in consideration of the face position (S54). Here, a numerical value corresponding to the degree of coincidence is obtained. When the coincidence determination is completed, it is determined whether or not the coincidence determination has been completed for all the target images in the content jacket recording unit 21 (S55). As a result of the determination, if it has not been completed, the image is changed to the next target image (S57), the process returns to step S53, and the determination is repeated as described above.

  As a result of the determination in step S55, when all the target images have been determined, the image with the highest degree of coincidence (auxiliary image) and related information (for example, recorded music) of the auxiliary image are obtained. It transmits to the camera 10 (S56).

  Subsequently, it is determined whether or not an image NG is received from the camera 10 (S71). That is, when the user of the camera 10 is not satisfied with the auxiliary image transmitted from the server 40, the fact is transmitted by changing the acquired information in step S29. If the image NG is received as a result of the determination, the image having the next highest matching degree and its related information are transmitted to the camera 10 (S72), and the process returns to step S71.

  If the result of determination in step S71 is that the image is not NG, it is next determined whether or not the related information is OK (S73). If the user of the camera 10 is not satisfied with the related information transmitted from the server 40, the fact is transmitted in step S29. If the result of determination is that it is not OK, other related information related to the auxiliary image is transmitted (S74), and the process returns to step S73. As a result of the determination in step S73, if the related information is OK, the process returns.

  If the result of determination in step S51 is that a representative image has not been acquired, it functions as a normal image recording and browsing server. First, it is determined whether an image for recording has been acquired (S61). As a result of the determination, if a recording image is acquired, the image is recorded in the recording image recording unit in the server 40 (S62), and the process returns.

  If the result of determination in step S61 is that an image for recording has not been acquired, it is determined whether or not to view the image recorded in the server 40 (S63). If the result of the determination is that the image is not appreciating, the process returns as it is. On the other hand, if the image is appreciating, the image is displayed according to the access (S64), and the process returns.

  As described above, in the second embodiment of the present invention, when the server 40 acquires the representative image, the representative image is temporarily stored, and a music CD having a similar jacket is searched (S51 → S54). The auxiliary image thus transmitted is transmitted to the camera 10 (S56). At this time, related information is also transmitted together. If the user performs an operation not to like the auxiliary image transmitted, if it is determined in steps S71 and S73, another auxiliary image and related information are transmitted (S71 and S74). Further, it has an image bank function for recording a mere photographed image, and stores the transmitted image (S62) and reproduces it in response to access (S63 → S64). The server 40 may omit the image bank function or may have other functions.

  Next, a third embodiment of the present invention will be described with reference to FIGS. 10 and 11. In the first and second embodiments, the auxiliary image is reproduced and displayed at the beginning of the slide show, and a still image is independently displayed and reproduced as the auxiliary image. In the third embodiment, the auxiliary image is reproduced and displayed last, and after the auxiliary image is reproduced, a moving image related to the auxiliary image is continuously reproduced and displayed.

  FIG. 10 is a block diagram illustrating a configuration of an image reproduction system including the camera 10, the television 30, and the server 40 according to the third embodiment. The camera 10 includes a photographing unit 1, an image recording unit 2, an image selection unit 3, a communication and control unit 4, a display unit 5, and an auxiliary data unit 6A. Except for the auxiliary data part 6A, the function of each part is the same as that of the first embodiment shown in FIG. 1, and therefore the detailed description of the same part is omitted.

  The auxiliary data unit 6A is a storage unit that stores the shooting date and time and the shooting location of the representative image in association with the image. The information such as the shooting date and time is transmitted together with the representative image when transmitting the representative image to the server 40, and the information such as the shooting date and time is also taken into consideration when the server 40 searches for the auxiliary image.

  The server 40 serving as a database includes a content jacket recording unit 21, a similar image search unit 22, a communication and control unit 24, a representative image and data temporary storage unit 25A, an image recording unit 26, a related information search unit 27, and a sequential playback. Unit 28 and network distribution unit 29. Since the content jacket recording unit 21 and the like are the same as those in the second embodiment, different parts will be mainly described.

  In the same manner as in the first embodiment, the content jacket recording unit 21 stores a jacket image of a moving image such as a DVD and related video in addition to a music CD or the like. In the case of a moving image such as a DVD, in addition to the jacket image, the top image of the moving image is also stored for retrieval. Similarly, in the case of content with images such as cartoons and animations, the first image is also stored for retrieval in addition to the cover image. The similar image and data search unit 22 searches the content jacket recording unit 21 for an image similar to the representative image and outputs it as an auxiliary image. The representative image and data temporary storage unit 25A stores the representative image transmitted from the camera 10 and information such as the shooting date and time stored in the auxiliary data unit 6 and transmitted in association with the representative image.

  The image recording unit 26 records a captured image or the like that is stored in the image recording unit 2 of the camera 10 and reproduced and displayed in a slide show. The related information retrieval unit 27 retrieves information related to the auxiliary image (a jacket such as a music CD or a video DVD), for example, a composer name or an artist name related to the music CD.

  The sequential playback unit 28 uses the captured image recorded in the image recording unit 26, the similar image, the auxiliary image searched by the data search unit 22, and the related information searched by the related information search unit 27, to generate these images. And the order of playback and display of information. The net distribution unit 29 is connected to the television 30 via a network such as the Internet, and transmits captured images, auxiliary images, and the like sequentially output by the reproduction unit 28 to the television 30. The television 30 is a large-screen television, and reproduces and displays images and information distributed from the network distribution unit 29.

  Next, the operation in this embodiment will be described with reference to FIG. A representative image 12 a (also an image No. 4 in the figure) is designated from the photographed images 12 photographed by the camera 10 and transmitted to the server 40 via the network 19. In addition to the representative image 12a, all the captured images (No. 1 to No. 3 in the figure) used for the slide show are also transmitted. The server 40 searches for similar images from the content jacket recording unit 21 serving as a database, and sets the image with the highest degree of matching as the auxiliary image 22.

  The server 40 searches for similar images in the similar image and data search unit 22 and searches for related information. Then, the sequential playback control unit 28 sequentially transmits captured images to the television 30 via the network distribution unit 29 and the network 19 in the order of photographing date and time, and the television 30 reproduces and displays the captured images. Then, at the end of the slide show, the representative image 12a is transmitted, and then the auxiliary image 22 similar to the representative image 12a is reproduced and displayed.

  In this embodiment, an auxiliary image 22 based on a DVD jacket image similar to the representative image 12a is transmitted, and after the auxiliary image 22 is reproduced and displayed, the moving image 120 recorded on the DVD is reproduced. In this example, after a slide show of photos that are memories of a trip to Vienna, a moving image 120 based on a DVD about Vienna is played.

  In the first and second embodiments, the example in which the auxiliary image 22 corresponding to the representative image 12a is displayed at the top has been described. However, in the present embodiment, the auxiliary image 22 is reproduced last and the viewer has a strong impression. Leave a slide show. Of course, in addition to the last reproduction, the representative image 12a may be reproduced at a place aimed at excitement during the slide show. The reproduction timing may be arbitrarily set by the user. Further, the reproduction order of the auxiliary image 22 may be determined according to the reproduction order of the representative image 12a. Further, instead of being reproduced and displayed on the television 30, it may be reproduced and displayed by the camera 10 as in the first and second embodiments.

  Further, after the last image of the slide show based on the photographed image 12 is displayed, the movie or documentary set in the same place after finishing watching the searched DVD video, for example, a travel photograph. By making it possible to appreciate, it is possible to inspire more travel and indulge in memories.

  Next, the modification of 3rd Embodiment of this invention is demonstrated using FIG.12 and FIG.13. In the first to third embodiments, one image is sequentially displayed on the display unit. However, in the first modification, a plurality of images are displayed. That is, as shown in FIG. 12, when the screen of the television 30 is a large screen, it is possible to display a plurality of images on one screen, so there is no need to stick to time division display. An application in which a plurality of photographed images are arranged on the large screen of the television 30 and a DVD is reproduced at the same time while switching between them is also possible. The configuration of the modified example of the third embodiment is the same as the configuration of the third embodiment shown in FIG.

  In the first to third embodiments, the user designates the representative image and acquires the auxiliary image. However, by executing the auxiliary content determination flowchart shown in FIG. Alternatively, the auxiliary image may be automatically selected.

  In the auxiliary content determination flow, the first photographed image is first selected (S201). Next, an image similar to the selected photographed image is searched from the content jacket recording unit 21 (S203). A content (image) having a similarity equal to or greater than a predetermined value is temporarily recorded (S203). Next, the next image is selected from the captured images (S204).

  If another image is selected, it is next determined whether or not there is a photographed image (S205). As a result of the determination, if there is a next photographed image, the process returns to step S202, the above steps are repeated, and content having a similarity equal to or greater than a predetermined value is recorded. If the result of determination in step S205 is that there are no captured images, that is, all the captured images have been searched, the content that has been searched a predetermined number of times or more and temporarily recorded in step S203 is used as auxiliary content (auxiliary image). Transmit (S206). As for transmission, when playback and display is performed with the camera 10 as in the first and second embodiments, to the camera 10 when being reproduced and displayed over the network as in the third embodiment, to the television 30 when distributed over the network. Send.

  In this way, content that can be searched many times can be said to be auxiliary content that is highly relevant to a captured image. For example, if you are traveling to Vienna, you will be able to select movies and music that are related to the land without fail because the captured images contain many of the features of Vienna. Become.

  As described above, in each embodiment of the present invention, a representative image is compared with an image related to content, a similar image is searched as an auxiliary image, and when a slide show is performed, the auxiliary image is reproduced and displayed. The image is being played back. For this reason, even if it is an image other than a face, it is possible to search for an auxiliary image, so that the range of applicable images is wide. Further, since it is not necessary to generate a representative image, it can be processed in a short time. Furthermore, since the auxiliary image is an image related to the content, it is possible to give the user a sense of expectation and enjoyment.

  In the present embodiment, the digital camera is used as the photographing device. However, the camera may be a digital single-lens reflex camera or a compact digital camera, and may be a mobile phone or a personal digital assistant (PDA). Of course, a camera built in Personal Digital Assist) may be used.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is a block diagram illustrating a configuration of an image reproduction system including a camera and a personal computer according to a first embodiment of the present invention. In the image reproduction system concerning 1st Embodiment of this invention, it is a figure which shows the use condition of the image reproduction system by a user, (a) is a mode that it has image | photographed, (b) is a camera's image | photographed image. (C) is a state of enjoying a slide show on a television, and (d) is a view showing a state of reproducing and displaying an auxiliary image in the slide show. FIG. 3 is a diagram showing an example of removing characters in order to search for an auxiliary image similar to a representative image in the image reproduction system according to the first embodiment of the present invention, wherein (a) is an image of a jacket such as a music CD. (B) is the figure which removed the character from the image of (a), (c) shows a representative image. It is a flowchart which shows the operation | movement of the camera control in the camera concerning 1st Embodiment of this invention. It is a flowchart which shows the operation | movement of the image matching degree determination in the camera concerning 1st Embodiment of this invention. It is a block diagram which shows the structure of the image reproduction system which consists of a camera and server concerning 2nd Embodiment of this invention. In the image reproduction system concerning 2nd Embodiment of this invention, it is a figure which shows the relationship between the flow of an image etc., and reproduction | regeneration display. It is a flowchart which shows the operation | movement of the camera control in the camera concerning 2nd Embodiment of this invention. It is a flowchart which shows the operation | movement of the server control in the server concerning 2nd Embodiment of this invention. It is a block diagram which shows the structure of the image reproduction system which consists of a camera and a server concerning 3rd Embodiment of this invention. In the image reproduction system concerning 3rd Embodiment of this invention, it is a figure which shows the relationship between the flow of an image etc., and reproduction | regeneration display. It is a figure which shows the relationship between the flow of an image etc., and a reproduction | regeneration display in the image reproduction system concerning the 1st modification of 3rd Embodiment of this invention. It is a flowchart which shows the operation | movement of auxiliary content determination in the image reproduction system concerning the 2nd modification of 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging | photography part, 2 ... Image recording part, 3 ... Image selection part, 4 ... Communication and control part, 5 ... Display part, 6 ... Slide show part, 6A ... Auxiliary data section, 7 ... Order control section, 8 ... Similar image search section, 8a ... Face detection character detection and processing section, 9 ... Auxiliary content recording section, 10 ... Camera, 12. ..Photographed image, 12a ... representative image, 20 ... personal computer, 21 ... content jacket recording unit, 22 ... auxiliary image, 23 ... face detection character detection and processing unit, 24 ... Communication and control unit, 25 ... representative image temporary storage unit, 25A ... representative image and data temporary storage unit, 26 ... image recording unit, 27 ... related information search unit, 28 ... sequentially Reproduction part, 29 ... Net distribution part, 30 ... Television, 40 ... Server , 100 ... photographed image, 110 ... jacket image, 111 ... image portion, 120 ... Video

Claims (18)

A recording unit in which image data corresponding to a plurality of images is recorded;
A representative image selection unit for selecting an image corresponding to the representative image from the image data recorded in the recording unit;
A search unit for searching for a content package including image data corresponding to an image similar to the representative image;
A slide show playback unit for playing back the plurality of images, with an image corresponding to the image data included in the content package searched by the search unit as a part of an image constituting the slide show;
An image reproduction system comprising:   2. The image reproduction according to claim 1, wherein the slide show reproduction unit reproduces the plurality of images by using a head image included in the content package searched by the search unit as a part of images constituting the slide show. system.   2. The image reproduction system according to claim 1, wherein the slide show reproduction unit reproduces the plurality of images by using a jacket included in the content package searched by the search unit as a part of images constituting the slide show. .   The slide show reproduction unit reproduces music data included in the content package while reproducing the plurality of images with an image corresponding to the image data included in the content package searched by the search unit as a head image. The image reproduction system according to claim 1.   The slide show reproduction unit reproduces the plurality of images while using the image corresponding to the image data contained in the content package searched by the search unit as a part of the images constituting the slide show, and the music included in the content package. The image reproduction system according to claim 1, wherein data is reproduced. Selecting image data corresponding to a representative image from a recording unit in which image data corresponding to a plurality of images is recorded;
Searching for a content package containing image data corresponding to an image similar to the representative image;
Replaying the plurality of images with an image corresponding to the image data included in the content package searched by the search unit as a part of an image constituting a slide show;
An image reproduction method comprising: A search unit for searching for a content package including image data corresponding to an image similar to a representative image included in the plurality of images from a recording unit in which image data corresponding to the plurality of images is recorded;
A playback unit that plays back the content included in the searched content package on the same device as the device that plays back the plurality of images;
An image reproduction method comprising: Searching a content package including image data corresponding to an image similar to a representative image included in the plurality of images from a recording unit in which image data corresponding to the plurality of images is recorded;
Reproducing the content included in the retrieved content package on the same device as the device that reproduces the plurality of images;
An image reproduction method comprising: Searching a content package including content data corresponding to data related to a representative image included in the plurality of images from a recording unit in which image data corresponding to the plurality of images is recorded;
Reproducing the content included in the searched package on the same device as the device that reproduces the plurality of images;
An image reproduction method comprising: A temporary storage unit that receives a representative image designated from the captured images and temporarily stores the representative image;
An image storage unit that stores content images;
A search unit for comparing the representative image with the image stored in the image storage unit and searching for an auxiliary image similar to the representative image;
A communication unit that outputs the auxiliary image searched by the search unit;
An image playback server characterized by comprising:   The image processing unit for detecting and removing characters from the image of the content, wherein the search unit searches for the auxiliary image using an image from which characters have been removed by the image processing unit. 10. The image reproduction server according to 10. A captured image storage unit for receiving and recording captured images;
A representative image storage unit for storing a representative image designated from the captured images;
A sequential playback unit for reproducing and displaying the representative image after the auxiliary image is reproduced and displayed in a slide show reproduction display;
A transmission unit for transmitting images ordered by the sequential reproduction unit;
The image playback server according to claim 10, further comprising:   13. The image reproduction server according to claim 12, wherein the sequential reproduction unit reproduces and displays the auxiliary image at the beginning or end of the slide show.   13. The image reproduction unit according to claim 12, wherein the image accumulation unit further accumulates music related to the content image, and transmits the music reproduction data when reproducing the slide show. server.   13. The image storage unit according to claim 12, wherein the image storage unit further stores a moving image related to the image of the content, and transmits the moving image data after reproducing and displaying the image when reproducing the slide show. Image playback server. An image storage unit that stores content images;
A search unit that compares a plurality of captured images with images stored in the image storage unit, and searches for images with high similarity as auxiliary images;
A communication unit that outputs the auxiliary image searched by the search unit;
An image playback server characterized by comprising: Compare the representative image specified from the captured images with the content image, search for auxiliary images similar to the representative image,
In the playback display of the slide show, after the auxiliary image is reproduced and displayed, the representative image is reproduced and displayed.
An image reproduction program characterized by the above. Designate a representative image from the captured images,
In order to compare the designated representative image with the content image and to search for an auxiliary image similar to the representative image, the representative image is transmitted to an external device,
Receiving the auxiliary image from the external device;
In the playback display of the slide show, after the auxiliary image is reproduced and displayed, the representative image is reproduced and displayed.
An image reproduction program characterized by the above.