JP2009038443A - Image reproducing device, image reproducing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reproducing device which can present a slide show that would attract the interest of the viewers, and to provide an image reproduction method and a program. <P>SOLUTION: The leading-role part is extracted from captured images 101 to 124, ..., and generates a leading role image 44. The leading role image 44 thus generated is composited with the captured image and composite slide show is performed in a scroll format. In the composite slide show, the captured image moves along the scroll direction and the leading-role image 44 is displayed at a fixed position in the frame 51 (corresponding to the display part in the display section 22). When the leading role image 44 is composited with the captured image, the compounding is inhibited, when the captured image is a macro image 120, a short-distance captured image 121, or an image 122 where the figure occupies a large portion. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to an image reproduction device, an image reproduction method, and an image reproduction program for reproducing an image photographed by an imaging device such as a digital camera.

In the past, photographs taken with a camera were printed and pasted on albums for enjoyment. At the opportunity to show still images taken on a trip to family and friends, it is necessary to devise an interesting way to prevent them from getting bored. As a representative device for this purpose, there is a slide show, and a method of replaying while switching desired images is enjoyed widely, and this function is also installed in digital cameras which have been widely used in recent years.

The slide show function has been improved in various ways, not only by switching and reproducing images, but also in combination with music or using image composition. In recent years, image processing technologies such as face recognition technology and image processing functions have been rapidly developed. As an example in which a composition function is added to a slide show, Patent Document 1 discloses a facial expression generation apparatus that changes facial expressions to be synthesized while performing a slide show. Patent Document 2 discloses an electronic album display system that executes a slide show in which an arbitrary caption can be added to an image. Furthermore, as a slide show format, a slide show in a display format in which one frame of a continuous film is switched by scrolling the whole image instead of switching one frame at a time is also used.
JP 2003-281563 A JP-A-2005-341512

The above-mentioned patent document 1 and patent document 2 have gone on a trip, such as displaying an image with added value, for example, a place name when traveling, by changing a facial expression or adding a caption. It can be enjoyed even by those who do not have it, and doubles the enjoyment when viewing images. However, in these patent documents, it is an image as it is taken, and it is difficult to provide a new taste to the user.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image reproduction apparatus, an image reproduction method, and an image reproduction program capable of performing a slide show that attracts what is seen. To do.

In order to achieve the above object, an image reproduction apparatus according to the first invention includes an extraction unit that extracts a main part from an image selected from still images to be reproduced, and a main part that generates a main part image from the extracted main part. The generation unit, a composition suitability determining unit that determines whether each still image in the still image to be reproduced is an image suitable for compositing the leading image, and the composition suitability determining unit determine that it is suitable for composition A compositing unit that synthesizes the leading image with the still image that has been determined, and a leading composite image for an image that is determined to be suitable for combining by the combining suitability determining unit, and an image that is determined to be unsuitable for combining A scroll display unit for scrolling and reproducing images that are not to be played as a series of still image sequences is provided.

According to a second aspect of the present invention, in the image reproduction apparatus according to the first aspect, the composition unit synthesizes the main character image so that the main character image is displayed at a predetermined position on the screen with respect to the scrolling still image. .
In the image reproduction apparatus according to the third invention, in the first invention, the composition suitability determination unit belongs to the main character portion that is the origin of the main image and the shooting date and time of the still image to be determined. The appropriateness of the composition is determined by comparing the shooting date and time of the still image.
Furthermore, in the image reproduction apparatus according to the fourth invention, in the first invention, the composition suitability determination unit belongs to a still image shooting location to be determined and a main character part that is a source of the main image. The suitability of the composition is determined by comparing the still image shooting location.
Further, in the image reproducing apparatus according to the fifth invention, in the first invention, the composition suitability determining unit may compose the image when the image to be determined is an image in which a person occupies a large area. Judged as inappropriate.

Furthermore, in the image reproducing apparatus according to the sixth invention, in the first invention, the composition suitability determining unit may determine that the image to be judged is not suitable for composition if the image to be judged is an image obtained by macro mode photography or short-distance photography. Judge.
Furthermore, in the image reproducing apparatus according to the seventh aspect based on the first aspect, the leading image generation unit generates a plurality of leading portions extracted from the still image as one leading image.
Furthermore, in the image reproducing apparatus according to the eighth invention according to the first invention, when the compositing unit composes a main character image based on the plurality of main character parts, the synthesized still image is the main character part. If it is one of the original images, only the main image that does not belong to the synthesized still image is synthesized.
Furthermore, an image reproduction apparatus according to a ninth aspect of the present invention includes the operation unit instructing a size and a position with respect to the screen when the main image is combined in the first aspect. The leading role image is synthesized with the size and position based on the instruction.
Furthermore, in the image reproduction apparatus according to the tenth invention, in the first invention, the main image generation unit separately generates a plurality of main characters extracted from a still image as a plurality of main images.
The synthesizing unit sequentially synthesizes the plurality of leading images with respect to each still image with respect to the still images determined to be suitable for the synthesis.

In order to achieve the above object, an image reproduction method according to an eleventh aspect of the present invention extracts a main part from an image selected from still images to be reproduced, generates a main part from the extracted main part, and reproduces the main part. It is determined whether each still image in the still image is suitable for combining the main character image, and the main character image is combined with the still image determined to be suitable for the combination, and is suitable for the combination. The main composite image is reproduced for the image determined to be unsatisfactory, and the image that is not combined for the image determined to be unsuitable for synthesis is scrolled and reproduced as a series of still image sequences.

In order to achieve the above object, an image reproduction program according to a twelfth aspect of the present invention extracts a main part from an image selected from still images to be reproduced, generates a main part from the extracted main part, and performs the reproduction. It is determined whether each still image in the still images to be combined is an image suitable for synthesizing the leading image, and the leading image is synthesized with the still image determined to be suitable for the synthesis, The main composite image is reproduced for an image determined to be suitable, and the non-composited image is reproduced for an image determined to be unsuitable for composition by scrolling the screen as a series of still image sequences.

In the twelfth aspect of the invention, the image reproduction program according to the thirteenth aspect synthesizes the main character image so that the main character image is displayed at a fixed position with respect to the scrolling still image.
In the twelfth aspect of the invention, the image reproduction program according to the fourteenth aspect of the invention determines whether or not the leading image is suitable for composition by determining the shooting date and time of the still image to be determined and the origin of the leading image. The appropriateness of the composition is determined by comparing the shooting date and time of the still image to which the protagonist part becomes.
In the image reproduction program according to the fifteenth invention, in the twelfth invention, the determination as to whether or not the main image is suitable for composition is based on the location of the still image to be determined and the origin of the main image. The suitability of the composition is determined by comparing with the shooting location of the still image to which the main part becomes.
According to a sixteenth aspect of the present invention, in the twelfth aspect of the invention, the determination as to whether or not the leading image is suitable for composition is based on an image in which a person occupies a large area. In this case, it is determined that the composition is unsuitable.

In the twelfth aspect of the invention, the image reproduction program according to the seventeenth aspect of the invention determines whether or not the main image is suitable for composition by determining whether the image to be judged is an image obtained by macro mode photography or short-distance photography. In such a case, it is determined that the composition is inappropriate.
According to an eighteenth aspect of the invention, in the twelfth aspect of the invention, the main image is generated by generating a plurality of main characters extracted from a still image as one main image.
According to a nineteenth aspect of the present invention, in the twelfth aspect of the invention, the main image is synthesized when the main image based on the plurality of main characters is synthesized. If it is one of the original images of the part, only the main image that does not belong to the synthesized still image is synthesized.
In the image reproduction program according to a twentieth aspect, in the twelfth aspect, the composition of the main character image indicates the size and position of the main character image with respect to the screen and synthesizes the main character image.
According to a twenty-first aspect of the invention, in the twelfth aspect of the invention, the main image is generated by separately generating a plurality of main characters extracted from a still image as a plurality of main images. In the synthesis of the still images determined to be suitable for the synthesis, the plurality of main images are sequentially synthesized for each still image.

In order to achieve the above object, an image playback apparatus according to a twenty-second aspect of the present invention is an image playback apparatus that performs slide show playback while sequentially switching target still images at a predetermined time. An extraction unit that extracts a main character part from the extracted image, a main character generation unit that generates a main character image from the extracted main character part, a combining unit that combines the still image with the main character image to create a main character composite image, A scroll display unit that scrolls and reproduces the images synthesized by the unit as a series of still image sequences.
According to a twenty-third aspect of the present invention, in the twenty-second aspect of the present invention, in the twenty-second aspect of the invention, a main character fixing mode for fixing and displaying the main character image and a plurality of the main character images are prepared and displayed while sequentially switching the main character images. A mode setting unit for setting the main character switching mode is provided.
According to a twenty-fourth aspect of the present invention, the image reproduction apparatus according to the twenty-second aspect further includes a composition suitability determining unit that determines whether or not the main image is combined with the still image. The display unit reproduces and displays the still image as it is in the slide show reproduction for the still image determined to be inappropriate by the synthesis suitability determination unit.

According to the present invention, it is possible to provide an image reproduction device, an image reproduction method, and an image reproduction program capable of performing a slide show that attracts the viewer's interest.

A preferred embodiment will be described below using an image reproducing apparatus to which the present invention is applied according to the drawings. An image reproduction apparatus according to an embodiment of the present invention is configured by a personal computer that stores image data of a still image captured by a digital camera or the like, and is scrolled by hardware of the personal computer and installed software. It is possible to execute a slide show.

FIG. 1 is a block diagram showing a configuration of a personal computer 10 according to the present embodiment. The personal computer 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, an operation unit 13, a RAM (Random Access Memory) 14, an image processing unit 15, a person extraction unit 16, a protagonist generation unit 17, an image. The composition unit 18 includes a scroll display unit 19, a storage unit 20, a display control unit 21, and a display unit 22. In the personal computer 10, the CPU 11, ROM 12, RAM 14, image processing unit 15, person extraction unit 16, main character generation unit 17, image composition unit 18, scroll display unit 19, and display control unit 21 are connected to a bus 23. .

A CPU 11 in the personal computer 10 performs overall control of the personal computer 10 in accordance with a control program stored in a ROM 12 composed of a flash memory or the like. A synthesis suitability determination unit 11a is provided inside the CPU 11, and these are substantially executed by a control program, but may be configured by dedicated hardware.

The composite suitability determination unit 11a is a determination unit that determines whether or not it is appropriate to combine a main image with a captured image, as will be described later. The operation unit 13 connected to the CPU 11 includes a user interface such as a keyboard and a mouse, and inputs various operation commands to the personal computer. The RAM 14 connected to the bus 23 is used as a temporary storage area.

The image processing unit 15 processes an image for reproduction display. The person extracting unit 16 detects a face portion from the image data by face detection or the like, and extracts the entire person from the outline of the person. The protagonist generation unit 17 generates the main character extracted by the person extraction unit 16 as a main character image. The image composition unit 18 synthesizes the main character image generated by the main character generation unit 17 with the captured image. As will be described in detail later, the scroll display unit 19 controls scroll display when reproducing and displaying a slide show. In this embodiment, the slide show includes a normal slide show and a composite slide show. Here, the normal slide show reproduces and displays normal images sequentially in a slide show format. A format in which a composite image obtained by combining a main character image with an image is sequentially reproduced.

The storage unit 20 is a storage unit that stores image data of a captured image such as a still image captured by a digital camera or the like, and includes a large-capacity storage device such as a hard disk. The display unit 22 is configured by a display device such as a liquid crystal monitor screen, and also performs playback display of a slide show displayed in a scroll format by the scroll display unit 19. The display unit 22 is connected to the display control unit 21 to perform display control.

As described above, the image reproducing apparatus according to the embodiment of the present invention is configured by a commercially available personal computer. In addition, although the person extraction part 16, the main character production | generation part 17, the image composition part 18, and the scroll display part 19 have shown the structure by hardware, you may be comprised by the software process by CPU11. This image reproducing apparatus stores a program for executing the operation described below in the ROM 12. Before explaining the flowchart relating to this program, a slide show reproduction display operation in the image reproduction apparatus will be described with reference to the drawings.

First, in the present embodiment, information of a captured image given as an example is shown in FIG. In this example, the photographer performs a city tour in P city and Q city on April 6, and in P city, the image numbers 101 to 150 are taken. In Q city, the image numbers 151 to 180 are taken. Taking still images. Further, on April 7, the next day, P city sightseeing is performed and still images with image numbers 181 to 210 are taken. These image data are attached with shooting date / time information and shooting position information by GPS or the like.

Next, FIG. 3 shows images with image numbers 101 to 210 displayed in a normal slide show. Reference numeral 31 shown at the upper left of each image is an image number. Image numbers 101 to 103 are images taken of a landscape, image number 120 is a macro shot, and image number 121 is an image shot at a short distance (specifically, a book on a bookshelf).

The image number 122 is an image in which a person is photographed on the screen, and the image of the leading role A is photographed in the image number 123. The image number 124 includes a plurality of person images. In the image number 125, a person is shown small, and in the image numbers 130, 132, and 133, images of the main characters A, B, and C in the main character switching mode are taken.

In the composite slide show according to the present embodiment, two modes, a main role fixing mode and a main role switching mode, are prepared. The main role fixing mode is a mode in which the extracted fixed main role is combined with the captured image and displayed, and the main role switching mode is a mode in which the main role is switched and displayed for each captured image.

Next, playback display of a normal scroll-style slide show in the present embodiment will be described with reference to FIG. When the scroll display starts, first, the image of the image number 101 within the frame 51 of the screen is displayed on the display unit 22 as shown in FIG. The image with the image number 101 gradually moves to the left, and the image with the image number 102 moves in the frame 51 as shown in FIG.

As the time elapses, as shown in FIG. 4C, the image with the image number 101 is not completely displayed, and the image number 102 and the image number 103 enter the frame 51 and are displayed on the display unit 22. Thus, in a scroll-like slide show, a captured image captured as a still image is displayed by moving within the display screen at a predetermined speed.

Next, the extraction of the main character part and the generation process of the main character image in the main character fixing mode will be described with reference to FIG. FIG. 5A shows an image with an image number 132. In this image, the leading role A41 is photographed with the landscape in the background. In the protagonist selection step, a person is automatically selected using a technique such as face recognition, and when the person is selected, its outline 32 is highlighted.

When the contour line 32 of the main character image of the main character A41 is detected and emphasized, the main character part is extracted as the next step. That is, as shown in FIG. 5B, the leading role A41 is extracted from the image number 132, and the leading role image A43 is pasted on the background 35 that is a blue background. The extracted main character image A43 can be changed in position and size by operating the operation member 13. When a final main character image is determined in the composite slide show, a main character image 44 as shown in FIG. 4C is generated. Note that the main role is not limited to one person, and it is of course possible to extract a plurality of people and combine them as a main role image as shown in FIG.

Next, generation of a composite image displayed in a composite slide show in the main role fixing mode will be described. When the main character image 44 is generated as shown in FIG. 5C, the main character image 44 is combined with the photographed image. At this time, it is determined whether or not the photographed image is suitable for the composition of the main character image. If the image is suitable, the image composition is performed.

For example, the image numbers 101 to 103 are normal landscape photographs, and image synthesis is performed on such photographed images. On the other hand, the image number 120 is macro shooting, the image number 121 is short-distance shooting, and even if a main character image is combined with such an image, it is unnatural and uncomfortable, so image combining is not performed and a combined slide show is performed. The original image is reproduced and displayed as it is. Also, a photographed image in which a person is reflected as in the image number 122 is unnatural even if the image is synthesized, so that no image synthesis is performed.

When one of the main character images is reflected in the original image as in the image number 123 and the image number 124, when the main character image 44 is synthesized, the same person appears in two places, which is unnatural and uncomfortable. . In such a case, the main character image A44 is not displayed. Note that when there are a plurality of main character images as shown in FIG. 5D, only the main characters other than the main character appearing in the original image are synthesized.

A composite image is generated under the above-described conditions. Next, a slide show reproduction display in a scroll format will be described with reference to FIG. When the slide show is started, the image of the image number 101 in the frame 51 is displayed on the display unit 22 as shown in FIG. On the left side of the frame 51, an image of the main character image A44 is synthesized and displayed. Since the scroll display is performed, the image with the image number 101 slowly moves to the left side, but the position of the main character image A44 is fixed.

The image with the image number 101 slowly moves to the left, and as shown in FIG. 6B, the image with the image number 102 appears in the frame 51, and further, as shown in FIG. 6C. In addition, the image with the image number 103 appears in the frame 51. As described above, the captured image moves slowly to the left and the main character image A44 is fixed. However, for the user who is viewing the captured image on the display unit 22, it is as if the main character image A44 is included in the captured image. It looks as if you are walking.

When the photographed image moves and the scene shown in FIG. 6D is reached, the image number 120 acquired by macro photography is scrolled. As described above, since an image obtained by macro shooting is unnatural and uncomfortable even if the main character image is combined with such an image, image combination with the main character image A44 is not performed, and FIG. As shown, only the original image is scrolled. Similarly, for the image of the next short-distance shooting image number 121 and the image of the image number 122 with a large proportion of people, image composition with the main character image A44 is not performed, and only the original image is scroll-displayed.

As shown in FIG. 6F, the leading image A44 is image-synthesized with respect to the image of the image number 123 in which the proportion of the person is small. However, for the image number 124 in which the protagonist image A44 is shown, the image in which the same person is shown in two places is unnatural. Not.

In this way, in the slide show in the main role fixing mode, the main character image A44 is fixed to the lower side in the scroll direction, and is arranged so that the face direction faces the upper side. For this reason, when the photographed image of the back scrolls, that is, moves slowly, it looks as if the leading image A44 is walking in the photographed image and traveling. If the back shot image is an unnatural image such as a macro shot image together with the main character image A44, image composition with the main character image A44 is prohibited.

Next, the leading role switching mode will be described with reference to FIGS. In the main character switching mode, a plurality of main character images are generated, and the main character images are sequentially switched and displayed for each captured image. Since a different protagonist image appears for each captured image, it looks as if the main character is changing, moving from the captured image to the captured image. In the present embodiment, the main character images are the same person, but may be different persons.

The main character image setting process in the main character switching mode will be described with reference to FIG. First, the main part is extracted in the same procedure as described in FIGS. 5 (A) to 5 (C). In the example of FIG. 7A, the main part A is extracted from the image number 130 (see FIG. 3), the main part B is extracted from the image number 132, and the main part C is extracted from the image number 133.

Subsequently, a main character image is sequentially generated from the extracted main character portions shown in FIG. That is, the main character image A′45 (FIG. 7B), the main character image B′46 (FIG. 7C), and the main character image C′47 (FIG. 7D) are generated.

Next, a composite image displayed in the composite slide show in the main role switching mode will be described with reference to FIG. First, the leading image B′46 is synthesized with the first captured image of the image number 101. In this state, the scroll display is started, and the leading image B′46 is fixed in the screen, but the photographed image of the back slowly moves to the left side, and the image of the image number 102 gradually changes from right to left. Moving.

When the left end of the image with the image number 102 is on the main character image B'46, the main character image changes from the main character image B'46 to the main character image A'45 (see FIG. 8B → FIG. 8C). Next, when the left end of the image with the image number 103 is on the main character image C′47, the main character image is changed from the main character image A′45 to the main character image C′47 (FIG. 8D → FIG. 8E). reference).

Thus, when the frame of the captured image is changed, the leading image is also changed. In this embodiment, the images are changed so that one main character image is displayed in one frame, but three main image images may be equally divided in one frame, and several images may be displayed as one main character. An image may appear. Moreover, you may make it change with time. For example, if one frame is changed in 3 seconds as the scrolling speed, the main character image may be changed every second, and conversely, the main character image may be changed at a time interval of 3 seconds or more.

A flowchart of a program for executing the slide show in the present embodiment reproduced and displayed as described above will be described with reference to FIGS. When the program for executing the reproduction display of the photographed image is started, it is first determined whether or not to perform a slide show (S1). Since this is selected from a menu screen of a slide show displayed on the display unit 21 of the personal computer 10, the determination is made based on an input signal from the operation unit 13.

If the result of determination in step S1 is that there is no slide show instruction, normal playback is carried out (S5). Normal playback is a mode in which only a designated captured image is displayed. On the other hand, if a slide show instruction has been given, it is next determined whether or not a composite slide show instruction has been given (S2).

If the result of the determination is that there has not been an instruction for a composite slide show, a normal slide show, that is, playback and display on the display unit 21 at predetermined time intervals in a predetermined order. On the other hand, if the result of the determination is that a composite slide show has been instructed, the composite slide show is executed in a scroll format (S3). In the present embodiment, scrolling is not performed in a normal slide show, but scroll display may be selected, or selection may be performed so that scroll display is not performed in a composite slide show. .

When the composite slide show subroutine is entered, as shown in FIG. 10, it is first determined whether or not an instruction to extract the main part has been issued (S11). As a result of the determination, if the extraction of the main character part is not instructed, the process jumps to step S13. On the other hand, if the extraction of the main character part is instructed, an extraction process of the main character part is executed (S12). As described with reference to FIG. 5B, the extraction of the main part is a process of selecting the main part from the captured image, and will be described later in detail with reference to FIG.

Next, it is determined whether or not a composite main character image generation instruction has been issued (S13). As a result of the determination, if there is no instruction to generate a composite main character image, the process jumps to step S15. On the other hand, if an instruction to generate a composite main character image has been issued, a composite main character image is generated (S14). The generation of the composite main character image is a process of generating a main character image from the main character part as described with reference to FIG. 5C and FIG. Details will be described later with reference to FIG.

Next, it is determined whether or not the reproduction display by the composite slide show is started (S15). As a result of the determination, if the execution instruction has not been given, the process ends. On the other hand, if the instruction has been given, execution of the slide show is started (S16). Details of the slide show will be described later with reference to FIG.

Next, a sub-routine of the leading role extraction process in step S12 will be described with reference to FIG. First, it is determined whether or not an image selection instruction has been issued (S21). In this image selection, the captured image is sequentially displayed on the display unit 22 by frame advance, or an image to be reproduced and displayed by the composite slide show is selected from the captured images displayed as thumbnails. If no instruction is given as a result of the determination, the process jumps to step S31. On the other hand, if an instruction has been given, the selected image is read from the storage unit 20 (S22).

Subsequently, the person extraction unit 16 extracts a person (S23). In this step, the presence of a person is detected from the image selected in the previous step using a technique such as face recognition. Based on the result of the person detection, it is determined whether or not there is a person (S24). As a result of the determination, if the presence of a person is not recognized, the main part cannot be extracted, and the process jumps to step S31.

On the other hand, if the person is recognized as a result of the determination, the person extracting unit 16 creates a person outline image (S25). Subsequently, based on the person contour image, the image composition unit 18 synthesizes a person contour image formed by the contour line 32 shown in FIG. 5A (S26). This person outline image is displayed on the display unit 22 with outline emphasis display as shown in FIG.

Next, it is determined whether or not an OK instruction is given for the person outline image (S27). This is because an error may occur in grasping the shape of a person. If no OK instruction is given, the extraction condition is changed (S30), and the process returns to step S25. By repeating steps S25 to S27 and S30, it is possible to extract the main part desired by the user.

As a result of the determination in step S27, when an OK instruction is issued, extraction of the main character portion is confirmed, and a main character image as shown in FIG. 5B is displayed (S28). Subsequently, the image data of the person portion generated as a composite image is stored (S29). When the storage is completed, it is next determined whether or not the storage is completed (S31). When there are a plurality of persons in the main character image, there are cases where a selection is made from a plurality of photographed images. In this case, the process returns to step S21 to add a main character part. As a result of the determination, if a termination instruction has been given, this subroutine is terminated and the original routine is returned to.

Next, a sub-routine of the leading image generation process in step S14 will be described with reference to FIG. First, all selected main characters and specific main characters are displayed (S41). In this display, for example, a leading image is displayed as shown in FIG. In the example of FIG. 5B, there is only one main character image, but when there are a plurality of main character images, all the main character images are displayed, and the specific main character among them is set as a default value. The main character with the smallest image number is displayed as the specific main character by the cursor 33 (see FIG. 7A) or the like. Thereafter, the specific leading role can be changed as appropriate.

Next, the main role fixing mode and the main role switching mode are set (S42). In step S11, the leading role part is extracted, and it is set whether to display the extracted leading role in the fixed mode or the switching mode. The setting is performed by clicking an icon set on the display unit 22.

Next, it is determined whether or not a switching instruction for a specific leading role has been made (S43). The specific main character is a main character whose position and size are to be changed, and this switching instruction is performed by moving the cursor 33. As a result of the determination, if the switching instruction of the specific leading role has not been made, the process jumps to step S45, whereas if the switching instruction has been made, the specific leading role is displayed (S44). Although the display here is performed with the cursor 33, in addition to this, the display method may be changed as appropriate, such as changing the color of the main character image.

Subsequently, it is determined whether or not an instruction to change the position and size of the specific leading role is made (S45). As a result of the determination, if there is no change instruction, the process jumps to step S48. On the other hand, if the change instruction has been given, display according to the instruction is performed (S46). That is, if the position change instruction is given, the position of the specific leading role is changed, and if the size change instruction is given, the size of the specific leading role is changed and displayed.

Next, it is determined whether or not the change instruction is OK (S47). If the change instruction is not OK, the process returns to step S45 and the above steps are repeated. On the other hand, if the result of the determination is OK, it is next determined whether or not to end (S48). In the multiple person or main character switching mode, a plurality of main character images may be selected even by the same person. If the position or size of these images needs to be changed, the user does not end and proceeds to step S43. Return and repeat the above steps. On the other hand, when the necessary position and size of the specific leading role are over, the end icon is clicked.

Next, it is determined whether or not the main role fixing mode is set (S49). Since the main role fixing mode or the main role switching mode is set in step S42, the mode set there is determined. As a result of the determination, if it is the main role fixing mode, the data (position and size) of the instructed main role is synthesized and stored (S50).

As a result of the determination, if it is the main role switching mode, the data (position and size) of the instructed main role is stored separately, that is, for each main role (S51). When the storage in step S50 or step S51 ends, the process returns to the original flow.

Next, the subroutine for executing the slide show in step S16 will be described with reference to FIG. First, an image is read (S61). In this step, the image data of the first and next frames of the image selected in the storage unit 19 and selected to be reproduced and displayed in the slide show is read. Here, the two frames of image data are read out in order to generate an image for scroll display. Subsequently, the suitability of the composition is determined (S62).

Since the main image is combined only with the image within the same range from the shooting date and time and the shooting location, it is determined whether or not this condition is met in the subroutine for determining whether or not to combine. Further, as described with reference to FIG. 6, since the main role image is not synthesized with respect to a photographed image photographed by macro photography, short-distance photography, or the like, this composition suitability determination is also performed. Details of the subroutine for determining suitability for synthesis will be described later with reference to FIG.

When the composite suitability determination is finished, it is next determined whether or not the composite is OK (S63). Since it is determined whether the composition is OK or not in the composition suitability determination subroutine in step S62 (see S79 and S80), the determination is made according to this determination result. As a result of the determination, if it is not composite OK, the process jumps to step S65, whereas if it is composite OK, image composition is performed (S64). In this image composition subroutine, the main character image 44 and the photographed image are composed, and a composite image is generated in which the photographed image is scrolled. Details of this image composition subroutine will be described later with reference to FIG.

Next, image display is performed (S65). In this step, if the image is combined in step S64, the combined image is displayed. If the combination is not OK in step S63, the original image (photographed in step S61) is displayed. (Image) is displayed as it is. Subsequently, it is determined whether or not an end instruction is given by the user (S66). As a result of the determination, if an end instruction has been given, the process returns to the original flow and an end process is performed.

If the result of determination in step S66 is that there is no end instruction, it is determined whether or not the image is a last image (S67). In this step, it is determined whether or not all of the images selected to be reproduced and displayed in the slide show are finished, that is, whether or not the image currently reproduced and displayed is the last image.

If the result of determination in step S67 is the last image, the process returns to the original flow and finishes. On the other hand, if it is not the last image, image switching is performed (S68). That is, the image is switched to the image next to the image currently being reproduced and displayed, the process returns to step S61, the image data of the image is read, and reproduction and display is performed in the same manner as described above.

While the subroutine for executing the slide show in FIG. 13 has been described above, the subroutine for determining whether or not to combine in step S62 will be described with reference to FIG. First, it is determined whether or not the shooting date and time is within a certain range with respect to the main role original image (S71). If the result of determination is within a certain range, it is then determined whether or not the shooting location is within a certain range with the main role original image (S72). In other words, if the photographed image for synthesizing the main character image is different from the location and date / time at which the main character image was photographed, the clothes and facial expressions may be different, which may be unnatural when viewed in a slide show. Therefore, in the present embodiment, image composition of the main character image is performed only when the shooting date and time and place are within a certain range.

In these two steps, the determination is made for all the leading original images. That is, if there are two main actor original images, it is determined whether or not the shooting date and shooting location of the read shot image are within a certain range with respect to the shooting date and shooting location of these two main actor original images. To be judged. The shooting date / time is determined based on the shooting date / time data attached to the shot image, and the shooting location is determined based on the positional information by GPS or the like attached to the shot image.

If step S71 and S72 are passed by Yes, it will be determined whether it is an image including all the leading roles (S73). This is to prevent an unnatural image from appearing twice when the main role image is synthesized when the main role is reflected in the original image. If all the main characters are not included, it is then determined whether or not close-up shooting is performed (S74). This is to prevent this because it is unnatural to synthesize a leading image with an image taken at a short distance, such as image number 121 (see FIGS. 3 and 6).

If it is not close-up shooting, it is next determined whether or not it is macro shooting (S75). This is to prevent unnaturalness when a leading image is combined with a macro shot image as in image number 120 (see FIGS. 3 and 6). If the result of determination is not macro photography, it is next detected whether a person is present (S76), and based on this detection result, it is determined whether there is a person (S77).

If the result of the determination is that there is a person, it is next determined whether or not the size of the person is large (S78). This is to prevent unnaturalness when a leading image is combined with an image having a large proportion of people, such as image number 122 (see FIGS. 3 and 6).

If it is determined in step S78 that the proportion of the person size is not large, or if it is determined in step S77 that there is no person, it is determined that the composition is OK (S79). This is because the shooting date / time determined in steps S71 and S72 is the same as the shooting date / time of the original image within a certain range (one of the conditions for suiting image synthesis), and steps S73 to S78. It is possible to synthesize the main character image because the determination as to whether or not the image is suitable for image synthesis is cleared.

On the other hand, if it is determined No in steps S71 and S72, and if it is determined Yes in steps S73 to S76 and S78, it is determined to be combined NG (S80). This is because the shooting date / time determined in steps S71 and S72 is not the same within a certain range as the shooting date / time of the original image, and it can be cleared whether the image is suitable for image synthesis in steps S73 to S78. Therefore, it is not suitable for synthesizing a leading image. When the determination in step S79 or S80 ends, the process returns to the original routine.

Next, the image composition subroutine of step S64 will be described with reference to FIG. If this subroutine is entered, it is first determined whether or not the main role fixing mode is set (S91). As described above, since the main role fixing mode or the main role switching mode is set in step S42 (FIG. 12), the set mode is determined.

As a result of the determination, when the main role fixing mode is set, the main character data is read (S92). Here, since the main character data is stored in step S50 (FIG. 12), this data is read out. Next, it is determined whether the original image is an image including a part of the main character (S93). This is because, when the main character is reflected in the original image as in the image number 123, the result of the image composition is that the same person is reflected in the same image because it gives a sense of incongruity. This is so that there is no such thing. As shown in FIG. 5D, when there are a plurality of main character images, whether or not one of the plurality of people is included, and the main character image is the same as in FIG. 5C. In the case of one person, it is whether or not a person in this leading image is included.

If the result of determination in step S93 is that the original image does not contain part of the main character, the main character data is synthesized with the original image (S94). In synthesizing this image, the scroll display unit 19 synthesizes an image in which the main image is displayed at a fixed position with an image in which the original image (captured image) is scroll-displayed, that is, moved in the scroll direction.

On the other hand, as a result of the determination, if a part of the main character is included in the original image, partial data of only the main character not included in the original image is extracted from the main character data (S95), and the main character extracted in the original image is extracted. (S96). As a result, it is possible to prevent a composite image in which two identical persons are shown, and only one main character image appears. Note that, in the image composition in step S96, as in step S94, the image is composed so that the original image is scroll-displayed and the leading image is displayed at a fixed position.

If it is determined in step S91 that the mode is not the main role fixing mode, that is, if it is the main role switching mode, reading of the main role data switching is performed (S97). In step S51 (FIG. 12), since the main character data is stored, the main character data is read out. When reading the main character data, since the main characters are sequentially switched and combined, the main character data to be read is switched in the order of reading.

When the reading of the main character data is completed, the main character data is synthesized with the original image (S98). In synthesizing the image, the original image is scroll-displayed as in steps S94 and S96, and the image is synthesized so that the main character image based on the main character data read in step S97 is displayed at a fixed position.

When step S98, S96 or S94 ends, the process returns to the original routine. By performing the flow of FIG. 9 to FIG. 15, the main character images are combined, the main character image is combined with the original image (captured image), and a slide show that is played back in a scroll format in which the main character image appears is displayed. Can be executed.

According to the present embodiment, display is performed in an atmosphere as if a person appearing as a leading role is moving in a landscape or a town in a captured image. For this reason, even if only a still image is taken at a travel destination, an atmosphere of moving image display can be provided, and a fun and fun image display can be performed. In a series of shootings such as trips, there are many scenes and the like, and even if there are few images of the photographer himself, a small number of his own images can always be displayed, so that a more enjoyable slide show than a conventional slide show can be realized.

In addition, according to the present embodiment, an image that is not suitable for composition, for example, macro photography, short-distance photography, or an image photographed with a large proportion of a person on the screen is not automatically synthesized. Therefore, it is possible to prevent an unnatural composite image from being displayed.

Further, in the present embodiment, whether or not to perform image composition is determined depending on the shooting date / time and location of the leading image. When performing a slide show, it is desirable that the images be in the same theme range, and if the clothes, facial expressions, etc. are different, the composite image will be unnatural. Inconvenience can be prevented.

Furthermore, in this embodiment, since the composition of the person images is limited to only a temporary image at the time of reproduction, the present slide show can be used easily. If a person is imprinted and recorded at the time of shooting, the fact may be lost, and an image becomes noisy.

Furthermore, since the present embodiment only reproduces a slide show, even if the processing of the boundary between the person and the background is simple, it is allowed, and accordingly, the compositing process can be realized in a short time, causing a delay in the slide show. There are few. In addition to displaying the same leading role, it is also possible to change the leading role, thereby realizing a slide show with more changes.

In the present embodiment, the case where the main character is a person has been described. However, the present invention is not limited to this. For example, an animal, an object, or a building may be used. In the present embodiment, either the main role fixing mode or the main role switching mode is set. However, only one mode may be set. Furthermore, in the present embodiment, the fixed position of the main character image is the left end of the screen. However, the position is not limited to this, and the position can be appropriately changed at the center, the right side, the upper side, and the lower side.

Furthermore, in this embodiment, the example is applied to a personal computer. However, the present invention is not limited to this, and an imaging apparatus such as a digital camera may be used. In this case, the display unit 22 is a liquid crystal monitor attached to the imaging apparatus. Or an external display device connected through a cable or the like. Also, a dedicated image playback device may be used.

Further, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

It is a block diagram which shows the structure of the image reproduction apparatus concerning one Embodiment to which this invention is applied. In one Embodiment of this invention, it is a figure which shows the information of the picked-up image mentioned as an example. It is a figure which shows the image reproduced | regenerated and displayed as a normal slide show in the image reproduction apparatus concerning one Embodiment of this invention. FIG. 5 is a diagram for explaining a normal scroll-style slide show in the image reproduction apparatus according to the embodiment of the present invention. It is a figure which shows the extraction process of the main character part in the main character fixed mode, and the production | generation process of a main character image in the image reproduction apparatus concerning one Embodiment of this invention. It is a figure which shows the synthesized image reproduced | regenerated and displayed on the slide show in the scroll format in the main role fixed mode in the image reproduction apparatus concerning one Embodiment of this invention. FIG. 10 is a diagram showing a main character image setting process in a main character switching mode in the image reproduction device according to the embodiment of the present invention. It is a figure which shows the synthesized image reproduced | regenerated and displayed in the slide show in the scroll format in the main character switching mode in the image reproduction apparatus concerning one Embodiment of this invention. 5 is a flowchart showing a main routine in the image reproducing apparatus according to the embodiment of the present invention. 5 is a flowchart of a composite slide show in the image playback apparatus according to the embodiment of the present invention. 5 is a flowchart of main part extraction in an image reproduction apparatus according to an embodiment of the present invention. 5 is a flowchart of a leading image generation process in the image reproduction device according to the embodiment of the present invention. 6 is a flowchart of slide show execution in the image reproduction apparatus according to the embodiment of the present invention. 5 is a flowchart for determining suitability of composition in the image reproducing apparatus according to the embodiment of the present invention. 5 is a flowchart of image composition in the image reproduction apparatus according to the embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Personal computer, 11 ... CPU, 11a ... Composition suitability judgment part, 12 ... ROM, 13 ... Operation part, 14 ... RAM, 15 ... Image processing part, 16 ... person extraction unit, 17 ... main character generation unit, 18 ... image composition unit, 19 ... scroll display unit, 20 ... storage unit, 21 ... display control unit, 22 ... Display unit, 23... Bus, 31... Sign (image number), 32... Contour line (contour emphasis display), 33. 43 ... Main character image A, 44 ... Main character image A, 45 ... Main character image A ', 46 ... Main character image B', 47 ... Main character image C ', 51 ... Frame

Claims (24)

An extraction unit for extracting a main part from an image selected from still images to be reproduced;
A main character generation unit that generates a main character image from the extracted main character part;
A composition suitability determination unit that determines whether each still image in the still image to be reproduced is an image suitable for combining the leading image;
A synthesis unit that synthesizes a leading image with a still image determined to be suitable for synthesis by the synthesis suitability determination unit;
The main composition image is scrolled within the screen as a series of still image sequences for the image determined to be suitable for composition by the composition suitability determination unit, and the uncombined image for the image determined to be unsuitable for composition. A scroll display for playback,
An image reproducing apparatus comprising:   The image reproducing apparatus according to claim 1, wherein the synthesizing unit synthesizes the main character image such that the main character image is displayed at a predetermined position on the screen with respect to the scrolling still image.   The composition suitability determination unit compares the shooting date and time of the still image to be determined with the shooting date and time of the still image to which the main part that is the source of the main character image determines the suitability of the composition. The image reproducing apparatus according to claim 1.   The composition suitability determination unit compares the still image shooting location to be determined with the still image shooting location to which the main part that is the origin of the main image, and determines the suitability of the composition. The image reproducing apparatus according to claim 1.   The image reproduction apparatus according to claim 1, wherein the composition suitability determination unit determines that the composition is inappropriate when the image to be judged is an image in which a person occupies a large area.   The image reproduction apparatus according to claim 1, wherein the composition suitability determination unit determines that the composition is inappropriate when the image to be determined is an image obtained by macro mode photography or short-distance photography.   The image reproducing apparatus according to claim 1, wherein the main character image generation unit generates a plurality of main character parts extracted from a still image as one main character image.   The composition unit, when composing a leading image based on the plurality of main parts, if the composite still image is one of the original images of the main part, The image reproducing apparatus according to claim 1, wherein only the main character images that do not belong are synthesized. An operation unit for instructing the size and position on the screen when the main image is composed;
The synthesizing unit synthesizes a leading image with a size and position based on an instruction from the operation unit;
The image reproducing apparatus according to claim 1, wherein: The main character image generation unit separately generates a plurality of main character parts extracted from a still image as a plurality of main character images,
The combining unit sequentially combines the plurality of leading images for each still image for still images determined to be suitable for the combining;
The image reproducing apparatus according to claim 1, wherein: Extract the main part from the image selected in the still image to play,
Generate a leading role image from the extracted leading role part,
It is determined whether each still image in the still image to be reproduced is an image suitable for combining the leading image,
Combining the leading image with the still image determined to be suitable for the above composition,
For the image determined to be suitable for the above composition, the leading composite image is reproduced, and for the image determined to be unsuitable for composition, the image that is not combined is scrolled and reproduced as a series of still image sequences.
An image reproduction method characterized by the above. Extract the main part from the image selected in the still image to play,
Generate a leading role image from the extracted leading role part,
It is determined whether each still image in the still image to be reproduced is an image suitable for combining the leading image,
Combining the leading image with the still image determined to be suitable for the above composition,
For the image determined to be suitable for the above composition, the leading composite image is reproduced, and for the image determined to be unsuitable for composition, the image that is not combined is scrolled and reproduced as a series of still image sequences.
An image reproduction program characterized by the above.   13. The image reproducing apparatus according to claim 12, wherein in the synthesis, the main character image is synthesized so that the main character image is displayed at a fixed position with respect to the scrolling still image.   The determination as to whether the main character image is suitable for composition is made by comparing the photographing date / time of the still image to be judged with the photographing date / time of the still image to which the main character part that is the source of the main character image belongs. 13. The program for reproducing an image according to claim 12, wherein the image is judged as appropriate.   The determination as to whether or not the main image is suitable for composition is made by comparing the shooting location of the still image to be determined with the shooting location of the still image to which the main portion that is the source of the main image belongs. 13. The program for reproducing an image according to claim 12, wherein the image is judged as appropriate.   13. The determination as to whether or not the main character image is suitable for composition is determined as unsuitable for composition when the image to be judged is an image in which a person occupies a large area. The program for image reproduction as described.   13. The determination as to whether or not the leading image is suitable for composition is determined as unsuitable for composition when the image to be determined is an image obtained by macro mode photography or short-distance photography. Image playback program.   13. The image reproduction program according to claim 12, wherein the generation of the main character image is to generate a plurality of main characters extracted from a still image as one main character image.   In the synthesis of the main character image, when a main character image based on the plurality of main character parts is synthesized, if the still image to be synthesized is one of the original images of the main character part, the still image to be synthesized is synthesized. 13. The program for reproducing an image according to claim 12, wherein only the main image that does not belong to the image is synthesized.   13. The program for reproducing an image according to claim 12, wherein the composition of the main character image indicates the size and position of the main character image on the screen and combines the main character image.   In the generation of the main character image, a plurality of main character parts extracted from the still images are separately generated as a plurality of main character images, and the combination of the main character images is performed for each still image determined to be suitable for the combination. The image reproduction program according to claim 12, wherein the plurality of leading images are sequentially synthesized. In an image reproduction apparatus that performs slide show reproduction while sequentially switching target still images at a predetermined time,
An extraction unit that extracts a leading part from an image selected from still images to be a slide show;
A main character generation unit that generates a main character image from the extracted main character part;
A synthesizing unit that synthesizes a leading role image with the still image to create a leading role synthesized image;
A scroll display unit that scrolls and reproduces the image synthesized by the synthesis unit as a series of still image sequences in the screen;
An image reproducing apparatus comprising:   The main character fixing mode for fixing and displaying the main character image, and a mode setting unit for setting a main character switching mode for preparing a plurality of the main character images and displaying the main character images while sequentially switching the main character images. 22. The image reproduction device according to 22.   A composite suitability determination unit that determines whether or not the main image is combined with the still image is suitable, and the scroll display unit is a still image determined to be inappropriate by the composition suitability determination unit. 23. The image reproducing apparatus according to claim 22, wherein the still image is reproduced and displayed as it is during the slide show reproduction.