JP4682254B2 - Image display method and image processing apparatus - Google Patents

Description

  The present invention relates to an image display method for reproducing digital image data on a television screen.

  With the start of digital high-definition broadcasting and the spread of high-definition television receivers, it has become possible to enjoy powerful images with higher image quality at home. In addition, with the spread of electronic still cameras, an increasing number of users enjoy displaying electronic still images and moving images on a monitor screen of a television as well as a personal computer.

  When viewing still images and videos on a TV receiver, the user generally selects playback data by viewing a list of thumbnail images displayed on the monitor screen and selecting the thumbnail image. is there.

  On the other hand, in recent years, it has become possible to use a memory whose storage capacity has been drastically increased, and the number of images that can be stored has increased dramatically. For example, memory cards have become capable of storing over 500 digital photos.

  It has become difficult for a user to find desired image data from such a large amount of image data.

  As a technique for efficiently arranging a large amount of data objects on the screen and enabling easy selection with the user's own eyes, for example, Patent Document 1 moves while displaying thumbnail images so that desired data is displayed. A technique for making it easy to find the above is disclosed. Patent Document 2 discloses a technique for displaying desired data in a 3D space by displaying an image and moving a virtual viewpoint.

JP 2001-306375 A JP-A-9-259130 JP 2002-358522 A

  However, in the conventional technique, thumbnail images are displayed so as to overlap each other, and an important part for discriminating a photograph may be hidden. For example, when there are a picture A and a picture B as shown in FIG. 14, the picture A is an important part for children who are “kicking” in the left part of the image, and the picture B is on the right side of the image. A person is important. When these photographs are displayed so as to be overlapped with other photographs, an important part is sometimes hidden and displayed as shown in FIG.

  Therefore, in the proposals disclosed in Patent Document 1 and Patent Document 2, an important part of the photograph is seen when the user performs an operation such as scrolling the display image or moving the position of the virtual viewpoint. After that, the only way to identify the photos was.

The present invention has been made in view of the above problems, and an object thereof is to provide an image display method that allows a user to easily find a desired image in a state where a large number of images are efficiently arranged on a screen. Yes.

  In addition, most digital images that have been taken can be divided into a region that the photographer has taken as a subject and other regions. Here, the former area is referred to as an attention area, and the latter area is referred to as a non-attention area.

  Considering the situation where the operator of the image display apparatus searches for a desired image from a large number of digital images, it is considered that the region of interest in each digital image is determined.

  Conventionally, when creating a thumbnail image, the image data is reduced uniformly without distinction between the attention area and the non-attention area. For this reason, when a plurality of images are displayed in the thumbnail image list display, the attention area is displayed small. Therefore, it is difficult to find a desired image using the attention area as a determination criterion from a large number of digital images by this method.

  Therefore, Patent Document 3 proposes a method of detecting a face area of an image and enlarging the detected face area to display a thumbnail image list. With this configuration, it is possible to detect a face area as an attention area and display the enlarged portion of the face area, thereby enabling a system with improved digital image search performance.

  However, in this method, the number of images that can be displayed on one screen is fixed in advance for displaying a thumbnail image list in which the region of interest is enlarged. For this reason, when displaying images exceeding the number of displayable images, a scroll function, a page turning function, and the like are separately required, and the user's operation becomes troublesome.

  When both the enlarged attention area and the original image are displayed as thumbnail images, the number of images that can be displayed on one screen is further reduced.

  Furthermore, there is a problem that it is impossible to deal with a face area that cannot be detected from the target image.

The image display method according to the present invention is an image display method for displaying a plurality of images in an overlapping manner, an acquisition step of acquiring shooting scene mode information given in association with each of the plurality of images, and the acquisition Based on the shooting scene mode information of each image acquired in the process, when the plurality of images are displayed in an overlapped manner, the exposure area determined corresponding to the shooting scene mode of each image is not hidden by other images. And a display step of determining the display position of each image and a display step of displaying the plurality of images in an overlapping manner according to the display position of each image determined in the determination step.

A program according to the present invention is a program for causing a computer to execute the image display method.

An image processing apparatus according to the present invention is an image processing apparatus that generates image data in which a plurality of images are displayed so as to overlap each other, and obtains shooting scene mode information given in association with each of the plurality of images. And an exposure area determined in accordance with the shooting scene mode of each image when the plurality of images are displayed in an overlapping manner based on the shooting scene mode information of each image acquired by the acquisition unit. A determination unit that determines a display position of each image so as not to be hidden by the image, and a generation unit that generates image data in which the plurality of images are displayed according to the display position of each image determined by the determination unit. It is characterized by having.

With the image processing apparatus and image display method of the present invention, when a plurality of images are displayed as a list, a portion that is considered as a region of interest is detected so that the region of interest does not overlap. As a result, the attention area of each image is always exposed, so that it is possible to achieve both image searchability by the user and mass display of images.

  That is, according to the present invention, the user can easily find a desired photograph.

FIG. 1 is a block diagram showing a configuration of a television receiver according to the first embodiment of the present invention. FIG. 2 is a block diagram showing a remote controller of the television receiver of this embodiment. FIG. 3 is a chart diagram showing a flow of operations of the display control unit 110 in the first embodiment. FIG. 4 is a diagram showing the layout information included in the information table used for the overlap control by the display control unit 110 in the first embodiment. FIG. 5 is a view showing the contents of the information table used for the overlap control by the display control unit 110 in the first embodiment. FIG. 6 is a diagram showing the effect of overlap control in the first embodiment. FIG. 7 is a chart showing an operation flow of the display control unit 110 in the second embodiment. FIG. 8 is a diagram showing the types of exposure control masks used by the display control unit 110 in the second embodiment for overlapping control. FIG. 9 is a chart showing an operation flow of the display control unit 110 in the second embodiment. FIG. 10 is a diagram showing an operation example of overlap control in the second embodiment. FIG. 11 is a diagram showing an example of overlap display when the present invention is not used, and an operation of overlap control in the second embodiment of the present invention. FIG. 12 is a chart showing the flow of operations of the display control unit 110 in the third embodiment. FIG. 13 is a diagram showing an operation example of overlap control in the third embodiment. FIG. 14 is a diagram showing a display example of an image list when the present invention is not used and a display example of an image list using the present invention. FIG. 15 shows an example of a screen displayed on the liquid crystal screen of the digital camera used in this embodiment. FIG. 16 is a diagram showing a state of the liquid crystal monitor at the time of photographing in the digital camera capable of recording the focal position used in the first and second embodiments of the present invention. FIG. 17 is a block diagram showing the configuration of the image display apparatus according to the embodiment of the present invention. FIG. 18 is a flowchart showing a processing flow of face detection processing used as one method of attention area detection. FIG. 19A is a diagram illustrating an example of image data, and FIG. 19B is a diagram illustrating an example of a determination result of a face area after face detection processing. FIG. 20A is a diagram illustrating an example of image data, and FIG. 20B is a diagram illustrating an example of range-of-interest determination processing based on shooting information. FIG. 21 is a flowchart illustrating a processing flow of attention area determination processing according to the fourth embodiment. FIG. 22A is a diagram showing an example in which images are randomly displayed in an overlapping manner, and FIG. 22B shows an example in which images in consideration of the size of a region of interest are superimposed and displayed. It is a figure. FIG. 23 is a flowchart showing a process flow of the overlap display process. FIG. 24 is a diagram showing an example of the overlapping list display. FIG. 25 is a flowchart illustrating a processing flow of attention area determination processing according to the fifth embodiment. FIG. 26 is a diagram illustrating a result after the attention area detection according to the sixth embodiment. FIG. 27 is a chart diagram illustrating a flow of operations of the display control unit 110 according to the seventh embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 is a diagram showing a configuration of a digital television receiver 100 as an image display device to which the present invention is applied.

  In FIG. 1, a signal received by an antenna (not shown) is input to the tuner unit 101. The tuner unit 101 performs processing such as demodulation and error correction on the input signal to generate digital data in a format called a transport stream (TS). The tuner unit further performs descrambling processing on the generated TS data and outputs it to the demultiplexer 102.

  The demultiplexer unit 102 receives video, audio data, electronic program guide (EPG) data, data broadcast data, and the like for a plurality of channels input from the tuner unit 101 from TS data in which time division multiplexing is performed. Data and audio data are extracted and output to the video / audio decoder 103. Video data processed by the video / audio decoder is written in the video memory 107 and output to the display device via the display synthesis unit 108. On the other hand, the audio data is output from the audio output unit 104.

  Reference numeral 109 denotes a card slot, which is connected to a removable memory card or the like, and is used to exchange data with the digital camera in this embodiment. An image data input unit 105 reads digital image data from a memory card connected to the card slot 109. An image processing unit 106 performs image processing on the digital image data input from the image data input unit and writes the digital image data in the video memory 107.

  Reference numeral 110 denotes a display control unit that instructs the image data input unit 105 to read an image, instructs the image processing unit 106 to execute processing, writes display data to the video memory 107, and instructs the display composition unit 108 to perform composition processing. An instruction to store information is given to the unit 113. The display control unit 110 also acquires input data from the remote controller 114 via the user input unit 111 and also acquires shooting data attached to the digital image data via the image data input unit 105. In the present embodiment, the display control unit 110 corresponds to an overlap determination unit and an exposure determination unit.

  An example of the remote controller 114 is shown in FIG. However, this figure shows only buttons for performing operations for realizing functions necessary for explaining the present embodiment, and buttons necessary for an actual receiving apparatus are not limited to this.

  2, 201 is a light emitting unit for performing infrared communication between the remote controller and the user input unit light receiving unit 111 of FIG. 1, 202 is a power key for turning on / off the power, and 203 is determined at the middle of the up / down / left / right buttons. Cursor & Enter key with keys, 204 and 205 are Up / Down keys consisting of 2 Up / Down keys, 206 is a numeric key with numbers 1-9 and 0 arranged in a matrix, 207 is blue, red, green, It is a color key with four yellow buttons arranged in a row.

  Reference numeral 208 denotes a monitor mode key for switching the display mode of the monitor of the television receiver. Reference numeral 209 denotes a “viewer” key for calling an image viewer function.

  The digital images used in this embodiment are still images and moving image data taken by a digital camera. The still image data is JPEG compression processed by the digital camera and then recorded on a memory card as a still image data file. Is used. Similarly, moving image data uses a group of images that have been JPEG-compressed in units of frames by a digital camera and recorded on a memory card as a moving image data file. In the following description, “image file” indicates still image data or moving image data.

  Each image file is given information at the time of photographing with a digital camera as related information associated with the image. Information at the time of shooting includes the shooting date / time and camera model name, as well as the shooting scene mode, focus position information indicating the focus position in the viewfinder at the time of shooting, strobe lighting status information, information indicating the distance to the subject, zoom status There is information.

  In the digital camera used in the present embodiment, information in which one of “left”, “middle”, and “right” is recorded is used as focus position information in the viewfinder at the time of shooting.

  FIG. 15 shows an example of a screen displayed on the liquid crystal screen of the digital camera used in this embodiment. FIG. 15A shows a state before touching the shutter button, and FIG. 15B shows a shutter. After the button is pressed lightly, the left rectangular area is in focus.

(Processing operation)
Display of a list of digital images in the digital television receiver 100 according to the present embodiment is started when a user inserts a memory card into the card slot 109 and calls an image viewer function by remote control operation.

  FIG. 3 is a diagram mainly showing the operation of the display control unit 110 for a part of the operation of the image viewer function in the digital television receiver. FIG. 5 shows table data used when the display control unit 110 performs layout control for displaying a list of image data. FIG. 4 is an example showing image display positions associated with the indexes in FIG. Hereinafter, the operation process of the display control unit 110 will be described with reference to these drawings.

  When the user presses the “viewer” key on the remote control shown in FIG. 2, the display control unit 110 starts the process shown in FIG. 3.

  First, the display control unit 110 reads and stores the total number of image files in the memory card inserted into the memory slot 109 via the image data input unit 105 (S301). Next, the Exif header information of the first image file is read (S302, S303).

  Then, the first content of the table shown in FIG. 5 is read (S304). If the image data is not registered (S305), the focal position information in the previously read Exif header information and the “non-exposed portion” of the table are displayed. ”(S307), if they are the same (S308), the contents of the next table are read (S306, S304). If the image data is not registered (S305), the focal position information included in the information of the Exif header is compared with the “non-exposed portion” of the table. If they are not the same, the image file being processed is registered in the table at this position (S309).

  In FIG. 5, IMG001. The image JPG is not registered because the non-exposed part is also “left” at the first index, and is registered in the table because it is different from the non-exposed part at the second index. ing.

  In this way, the display control unit 110 searches for a vacant place from the smaller index of the table, and when the vacant place is found, the non-exposed portion associated with the index, the focal position information of the image file, If they are the same, the same processing is performed for the next table.

  Similarly, the processing of the second, third,... Memory card is continued, and when the processing of the M (= total number of images) -th image data is completed, display processing of each image is performed according to the contents registered in the table. The display control unit 110 reads out the image data from the memory card via the image input unit 105 (S314), sends it to the image processing unit 106, performs decoding processing (S318), and then displays the display data on the video memory 107. The image is displayed by writing (S319).

  Although not shown in particular, when all the images are displayed by the processing of FIG. 3, the user operates the up / down / left / right keys of the remote controller to move the cursor to select an image to be viewed. If you press the Enter key at the image you want to see, only that image can be displayed on the screen.

(Action / Effect)
An operation obtained by the processing of the display control unit 110 described above will be described. FIG. 6A shows an image file IMG001. jpg is “2” instead of the place of “1” in FIG. 6 (the circled numbers in FIGS. 4, 5, 6, 10, 11, and 13 are enclosed in square brackets in the text) by the above processing. It shows a state of being arranged at the position of. As a result, the subject portion on the left side of the image is exposed, and this photo can be easily found compared to the case where it is arranged at the position “1” as shown in FIG. become able to do.

That is, according to the first embodiment of the present invention, it is possible to expose a portion suitable for discriminating a photo based on information unique to the photo.
In the above description, an example using information called focal position information recorded at the time of photographing was introduced. However, for example, a digital camera can set a subject position or a focal point fixed mode as a photographing scene mode. In this case, information on the shooting scene mode may be used. For example, as shown in FIG. 16, when a photograph is taken with a digital camera having shooting scene modes such as “person right” and “person left”, and there is information “person right” in the Exif header information of the image, the right portion of the image is exposed. By controlling the list display position as described above, the same operation as described above can be obtained.

  In the above description, in order to simplify the description, an example in which only 12 photos are displayed on the screen has been shown. However, a larger number of images may be displayed.

(Second Embodiment)
Next, another embodiment of the present invention will be described. Since the configuration of the digital television receiver used in this embodiment is the same as that shown in FIG.

  In this embodiment, a digital image taken with a digital camera similar to that used in the previous embodiment is used. However, in the digital camera used in this embodiment, information on the shooting date and time, and the direction of the camera at the time of shooting are used. Mode settings such as “Auto mode”, “Landscape mode”, “Portrait mode”, “Night (nighttime shooting) mode” as the shooting scene mode when shooting Is recorded. Here, the setting information related to the shooting scene mode corresponds to information related to the shooting mode.

(Processing operation)
Display of a list of digital images in the digital television receiver 100 of this embodiment is started when a user inserts a memory card into the card slot 109 and calls an image viewer function by remote control operation.

  FIG. 7 is a diagram mainly showing the operation of the display control unit 110 for a part of the operation of the image viewer function in the digital television receiver. FIG. 8 shows the types of exposure control masks used in the process of FIG. Exposure control masks with different exposure amounts and / or exposure positions are prepared depending on the focal position and the shooting scene mode. This will be described below with reference to these drawings.

  First, the display control unit 110 acquires the total number of image data recorded on the memory card (S701), and reads the Exif header information of the first image data (S702).

  When the focus position information is included in the Exif header information of this image data (S704), among the three exposure control masks of the left exposure mask, the center exposure mask, and the right exposure mask shown in FIG. Then, the exposure control mask at the focal position corresponding to the focal position information is determined (S705). On the other hand, when there is no focus position information in the image data (S706), if there is shooting scene mode information, the exposure control corresponding to the shooting scene mode information in the exposure control mask shown in FIG. Determine the mask. (S707) If the Exif header information contains neither focus position information nor shooting scene mode information, a standard exposure mask is determined (S708).

The same processing is performed on the subsequent image data to determine an exposure control mask (S709, S710, S703).

  FIG. 9 is a diagram showing the contents of display processing performed by the display control unit 110. FIG. 10 is a diagram showing the image data used in the present embodiment and the state on the screen by the display control of FIG. 9 by the display control unit 110. In the figure, (a) is a diagram showing the image data used in this embodiment. The used images “1” to “10” are assumed to have three types of focus position information arranged in order of left, middle, and right.

  This will be described below with reference to these drawings.

  First, the display control unit 110 reads a thumbnail image of the first image data from the memory card (S901 to S902), performs image decoding processing (S903), and uses the exposure control mask determined in the processing of FIG. Then, mask processing is performed (S904). As this masking process, a process of embedding flag data indicating whether or not the data is an exposed portion is performed on the data of each pixel.

  In this embodiment, the lower 1 bit of the blue component (value of 0 to 255) of each pixel is used as flag data. That is, in the image, a pixel whose lower 1 bit of blue is 0 means a non-exposed part, and 1 means an exposed part.

  Next, the display control unit 110 arranges the image being processed at a predetermined reference position on the screen (S905). In the present embodiment, the reference position is defined as the left end of the area to be displayed as a list on the screen. At this time, it is checked whether there is a pixel including flag data indicating an exposed portion under the region to be arranged (S906). In the case of the first image data, since there is no other image, it is arranged as it is at the reference position.

  Similarly, in the process of processing the second image data, when there is flag data indicating an exposed portion under the arranged image, that is, when it overlaps with an exposed portion of another image (S907). , S908), the position of the arranged image is moved to the right by a certain pixel and rearranged (S909).

  In the figure, (b) is a diagram showing a state in which the display process up to the second sheet is completed. In the second sheet layout process, although it was initially positioned at the reference position, Since the images overlap, the second image is moved to the right, and the arrangement processing is finished when the overlap with the exposed portion of the first image disappears.

  Next, in the process of processing the third image data, the image is moved for rearrangement because it overlaps with the exposed portion of the first image when it is arranged at the reference position. However, as can be seen from (a) in the figure, in this example, the third image is a photograph in which the focal position information is on the right side, and it is determined in the processing of FIG. 7 that the right exposure mask is used. It is an image. In this case, the display control unit moves the image leftward from the reference position (S910), and moves the image to a position where it does not overlap with the exposed portion of the other image, and rearranges the image. At this time, the display control unit slides the arranged image group (here, the first and second images) to the right so that the third image does not protrude from the list display area surrounded by the dotted line in the figure. To do. The reference position is also moved to the right by the same amount (S913). FIG. 10C shows the state at this time.

  Similar processing is performed on the fourth and subsequent image data, and the placement processing proceeds as shown in FIGS.

(Action / Effect)
Next, the operation of the processing of the display control unit 110 in this embodiment will be described. Before that, a display example when the present invention is not used will be described with reference to FIG. FIG. 11A shows a display example when images are simply superimposed and displayed without using the present invention. At this time, when all the images have a focal position at the central portion as shown in FIG. 5B, the portion is exposed as shown in FIG. However, when a list of images in which images whose focal positions are not in the center is mixed as shown in (d) in the figure, the exposed portion of some images is exposed as shown in (e) in the figure. It will be quite hidden.

  In general, when shooting with a digital camera having a plurality of focal positions as used in the present embodiment, it is normal to shoot while confirming that the part that the user considers important is in focus. It is. In other words, the fact that the area near the focal point is hidden and displayed means that an important part for discriminating a subject that the user feels important in many cases, that is, a photograph, is hidden.

  On the other hand, when the processing of the display control unit 110 in the present embodiment described above is performed, an area near the focal position can be displayed without being hidden in any image as shown in FIG.

  Although not described in detail in FIG. 11, in the exposure control mask determination process shown in FIGS. 7 and 8, the focus position is changed by switching the exposure control mask depending on the shooting scene mode such as “landscape” and “person”. Even when there is no information, the exposure part and the exposure amount can be controlled according to the shooting scene of the photograph. For example, in the case of a landscape photograph, the entire landscape often remains as an “impression” for the user, and it is necessary to increase the exposed portion in order to identify and select an image. On the other hand, in the case of a human photograph, it is only necessary to expose the face so that the face can be identified, and there is a tendency that the exposed portion does not have to be so many.

  By performing such control that reflects the shooting scene mode, it is possible to perform efficient image placement in accordance with image characteristics while maintaining ease of discrimination.

  In the above description, only the case where the images are arranged in a horizontal row while being overlaid has been described as a list display method. However, according to the gist of the present invention, the method of overlapping the images is not limited to this. The same applies to the case where the images are stacked vertically and horizontally and vertically.

  In the above description, in the description of the mask process using the exposure control mask, the lower 1 bit of the blue component data is used as a flag. However, the present invention is not limited to this, and is not limited to this for each pixel. A dedicated flag bit may be provided, or a bit of data indicating the alpha blend amount may be used.

  In the above description, only two types of scene modes, “landscape” and “person”, are given as an example. However, the present invention is not limited to this, and the scenes such as “night view” and “macro” can be used. May be.

(Third embodiment)
In the second embodiment, the image arrangement is more efficient and easy to discriminate from the Exif header information such as the focus position information and the shooting scene mode information, but of course, the present invention is limited to this. For example, efficient image placement can be performed based on the shooting date and time information.

  The configuration of the digital television receiver used in this embodiment is the same as that in FIG.

(Processing operation)
FIG. 13A shows image data used in the present embodiment. The images are arranged in time series, and the shooting time is shown below the images.

  First, the display control unit 110 reads the Exif header information of the first image data from the memory card (S1201 to S1202), and further reads the thumbnail image of the first image data (S1203). The read image data is sent to the image processing unit 106, and decoding processing is performed (S1204).

  Next, the shooting time of the (N-1) th image is read (S1205). If the difference between the shooting time of the Nth image and the shooting time of the (N−1) th image is within 10 seconds (“5” and “6” in FIG. 13), the Nth and (N -1) Control is performed so as to increase the amount of overlap with the first image (S1206 to S1207). On the other hand, when the difference between the shooting time of the Nth image and the shooting time of the (N-1) th image exceeds 10 seconds, the overlap amount between the Nth image and the (N-1) th image is set to a predetermined amount. (S1206, S1208).

  Similarly, the same processing is performed on the (N + 1) th and subsequent image data, and the processing is terminated when the same processing is performed on all the image data recorded on the memory card (S1209 to S1210).

  By such control, a list is displayed in an arrangement as shown in FIG.

(Action / Effect)
With the above processing, it is possible to arrange more images than when all the images are overlapped with the same overlap amount ((b) of FIG. 13). With this processing, in the case of an image group with a shooting time of 10 seconds or less, only one image has a large exposure (“6” in FIG. 13C), but the remaining images (“4” and “5”). Is less exposed. Although it seems that it is not desirable for photo discrimination because the exposed part is reduced, in general, in the case of an image with a very short shooting time as described above, the user can use the content of the image with a lot of exposure ("6"). It is known that the contents of the images (“4”, “5”) with low exposure can be estimated, and this is not an obstacle particularly for selection of images.

  That is, according to the processing of the display control unit 110 of the present embodiment, it is possible to perform efficient list display according to the difference in image capturing time.

(Fourth embodiment)
FIG. 17 is a block diagram showing the configuration of the image display apparatus according to the embodiment of the present invention.

Here, reference numeral 1101 denotes the entire image display apparatus. A control CPU 1201 controls the entire image display device 1101. An image display unit 1203 includes a device that can realize a thin screen such as a liquid crystal display or a plasma display. The image display unit 1203 displays an image instructed to the control CPU 1201. In the present embodiment, the control CPU 1201 corresponds to an overlap determination unit and an exposure determination unit, and the image display unit 1203 corresponds to a display.

  The image display device 1101 is also a device that displays an overlapping list display of a plurality of image data, which is a feature of the present invention.

  An image storage unit 1205 includes a nonvolatile storage device such as a hard disk. A large amount of images taken by a digital camera, a digital video camera, etc., and captured by an image display device can be accumulated and stored by a user operation.

  Reference numeral 1207 denotes a memory card. The memory card 1207 is detachable. The memory card 1207 can also be connected to an image capturing device such as a digital camera or a digital video camera, and the captured image data can be taken into the image display device 1101 via the memory card 1207.

  Reference numeral 1209 denotes an external input terminal. Via this terminal, image data of an image capturing device such as a digital camera or a digital video camera can be directly taken into the image display device 1101.

Reference numeral 1211 denotes an image decoder unit. Image data captured from the image storage unit 1205, the memory card 1207, and the external input terminal 1209 is converted into a data format that can be handled inside the image display device 1101 such as the control CPU 1201 here.

  Reference numeral 1213 denotes a face detection processing unit. It is determined whether or not there is a person image in the captured image data. If there is a person image, processing for specifying the area is performed. Details of the processing will be described later.

  Reference numeral 1215 denotes a cache memory. It is used for temporarily holding image data decoded by the image decoder 1211, holding image data displayed on the image display unit 1203, holding temporary data of the face detection processing unit 1213, and the like.

  FIG. 18 is a flowchart showing a processing flow of face detection processing used as one method of attention area detection.

  This process is executed by the face detection processing unit 1213 from the image data to be displayed prior to the execution of the overlay list display. After the start of processing, the process proceeds to step 1101.

  In step 1101, a process of searching for an area containing skin color data from the color data in the image data is executed. After processing, go to step 1103.

  In step 1103, pattern matching processing is performed between the target image and an image pattern having a pattern or mouth shape indicating the facial features. After the processing, the process proceeds to step 1105.

  In step 1105, the processing branches depending on whether or not a face area exists in the target image based on the processing results of steps 1101 and 1103. That is, if a face area exists, the process proceeds to step 1107. If the face area does not exist, this process ends.

In step 1107, the area information determined as the face area based on the processing results in steps 1101 and 1103 is written to the cache memory 215 and the process is returned to the control CPU 1201, thus ending this process.

  FIG. 19A shows an example of image data. FIG. 19B is a diagram illustrating an example of a region-of-interest determination process based on the determination result of the face region after the face detection process.

  Face detection processing is executed on the image data in which a human image is captured as shown in FIG. 19A by the face detection processing unit 1213 in the processing flow described with reference to FIG. As shown in FIG. 19B, the areas determined as face areas are portions 1303 and 1305. As shown in the figure, the face area is recognized as circular graphic data.

  The face area detected by the face detection process is determined as the attention area, and is applied to the control of the overlapping list display.

  The face detection process has a good detection rate when the ratio of the face as the subject to the entire image data is large. However, when the ratio is low, the face may not be detected. Processing in this case will be described with reference to FIGS. 20 (a) and 20 (b).

  FIG. 20A shows an example of image data. FIG. 20B is a diagram illustrating an example of a range-of-interest determination process based on shooting information.

  For image data shot with a digital camera or the like, information at the time of shooting can be held in the image data (Exif information). This Exif information includes position information at the time of shooting. Therefore, in the present invention, when the face detection process fails, the focal position information of the Exif information is read. This is the range indicated by 1403 and 1405 in FIG.

  From this focus position information, a range indicated by a circular figure including the position information is recognized as the attention area. This is the range indicated by 1407 in FIG.

  When the face area cannot be detected from the target image, the attention area is determined from the focal position information of the Exif information.

  FIG. 21 is a flowchart showing a processing flow of the attention area determination processing.

  This process is executed in the pre-processing stage of the control of the overlapping list display. After the start of processing, the process proceeds to step 1201.

  In step 1201, image data is read from the image storage unit 1205, memory card 1207, and external input terminal 1209, decoded by the image decoder 211, and then stored in the cache memo 215. After the processing, the process proceeds to step 1203.

  In step 1203, the face detection processing described in FIG. 18 is called by the face detection processing unit 1203 for the read image. After the processing, the process proceeds to step 1205.

  In step 1205, the process branches depending on whether or not there is a face area based on the face detection process result. That is, if there is a face area, the process proceeds to step 1207. If there is no face area, the process proceeds to step 1209.

  In step 1207, since it is determined that there is a face area in the target image, a process for registering the face area data in the cache memory as attention area data of the target image is performed. After the registration process, this process ends.

  In step 1209, the process branches depending on whether or not the target image has Exif information. That is, if there is Exif information, the process proceeds to step 1211. If not, the process proceeds to Step 1215.

  In step 1211, the Exif information is read from the target image data, and the focal position information is read. After the processing, the process proceeds to step 1213.

In step 1213, processing for converting the focal position information described in FIGS. 20A and 20B to the attention area is executed. This area is registered as the attention area. After the registration process, this process ends.

  In step 1215, since the face area does not exist from the target image and the Exif information cannot be acquired, processing for registering the center position of the image as the attention area is performed. After the registration process, this process ends.

  The process described here is executed for all image data to be displayed in the overlapping list display.

  FIG. 22A is a diagram illustrating an example in which images are randomly displayed in an overlapping manner. FIG. 22B is a diagram showing an example in which images in consideration of the size of the attention area are displayed in an overlapping manner.

  In the multiple image overlapping list display of the present invention, the areas of the attention area portions 1701, 1703, and 1705 of the respective images 1501, 1503, and 1505 are always displayed on the screen as protection areas. This enhances the searchability of the user. In addition, the other area can be efficiently displayed by arranging the other area below the other image as a non-attention area.

  Compared to the case where images are randomly overlaid as shown in FIG. 22 (a), the screen is displayed in a manner such that a large area of interest is overlaid as shown in FIG. 22 (b). Can be used effectively. By incorporating this processing, an image can be displayed more efficiently.

  FIG. 23 is a flowchart showing a process flow of the overlap display process.

  This processing is executed when the overlap list display is selected by the user. After the start of processing, the process proceeds to step 1301.

  In step 1301, a process for executing the attention area determination process described in FIG. 21 is called. This process is executed for all the images to be displayed. After the processing, the process proceeds to step 1303.

  In step 1303, the attention area determination processing in step 1301 is performed, and processing for rearranging the attention areas of each image in descending order is executed. After the processing, the process proceeds to step 1305.

  In step 1305, 1 indicating the head image is set in the variable N indicating the image. In addition, as the number of N increases, it is displayed superimposed on the bottom. After processing, go to step 1307.

In step 1307, processing for determining the position of the image indicated by the variable N is performed. The method for determining the arrangement position is a method for obtaining coordinate values by random numbers. Here, as the determination of the arrangement position, the coordinate value is selected such that the attention area is exposed to the maximum and the non-attention area is hidden by the upper image to the maximum. After processing, go to step 1309.

  In step 1309, it is determined whether or not the arranged image is within the restricted angle range. This is because if the image display angle is too large, it gives an impression that it is difficult for the user to see. If it is determined that it is out of the specified range, the process returns to step 1307. If it is within the specified range, the process proceeds to step 1311.

In step 1311, it is determined whether or not the upper image overlaps the attention area of the arrangement target image. If it is determined that there is an overlap, the process returns to step 1307. If it is determined that they do not overlap, the process proceeds to step 1313.

In step 1313, the process branches depending on whether or not the value stored in the variable N indicates the final image to be displayed. That is, in the case of the final image, the process proceeds to Step 1317. If it is not the final image, the process proceeds to step 1315.

In step 1315, the process of incrementing the value of the variable N and determining the arrangement position of the next image is repeated.

  In step 1317, when the arrangement positions of all the images to be displayed have been determined, the image display unit 1201 is called a process for executing an overlapping list display. After the display is completed, this process ends.

  FIG. 24 is a diagram showing an example of the overlapping list display. It can be seen that the attention area of each image is always displayed on the image display unit 1201.

(Fifth embodiment)
The basic configuration of the image display apparatus according to this embodiment is the same as that of the fourth embodiment. However, the focus area information is prioritized over the face detection process for the attention area detection described in FIG.

  FIG. 25 is a flowchart showing a processing flow of attention area determination processing of the present embodiment.

  This process is executed in the pre-processing stage of the control of the overlapping list display. After the start of processing, the process proceeds to step 1401.

  In step 1401, image data is read from the image storage unit 1205, memory card 1207, and external input terminal 1209, decoded by the image decoder 211, and stored in the cache memo 215. After the processing, the process proceeds to step 1403.

  In step 1403, the process branches depending on whether or not the target image has Exif information. That is, if there is Exif information, the process proceeds to step 1407. If not, the process proceeds to Step 1405.

  In step 1405, the face detection processing described in FIG. 18 is called by the face detection processing unit 1203 for the read image. After processing, go to step 1411.

  In step 1407, the Exif information is read from the target image data, and the focus position information is read. After processing, go to step 1409.

  In step 1409, processing for converting the focal position information described in FIGS. 20A and 20B into a region of interest is executed. This area is registered as the attention area. After the registration process, this process ends.

  In step 1411, the process branches depending on whether or not there is a face area based on the face detection process result. That is, if there is a face area, the process proceeds to step 1413. If there is no face area, the process proceeds to step 1415.

  In step 1413, since it is determined that there is a face area in the target image, processing for registering the face area data in the cache memory as attention area data of the target image is performed. After the registration process, this process ends.

  In step 1415, since the face area does not exist from the target image and the Exif information cannot be acquired, a process of registering the center position of the image as the attention area is performed. After the registration process, this process ends.

(Sixth embodiment)
The basic configuration of the image display apparatus according to this embodiment is the same as that of the fourth embodiment. However, the attention area takes the logical sum of the face detection area and the shooting information.

  FIG. 26 is a diagram illustrating a result after the attention area detection according to the present embodiment. An area 1501 is an attention area by face detection, an area 1503 is a focus area, and an area 1505 is an attention area converted from the focus area 1503. In this embodiment, the attention area is determined by the logical sum of the area 1501 and the area 1505.

(Seventh embodiment)
The basic configuration of the image display apparatus according to this embodiment is the same as that of the second embodiment. However, the operation flow of the display control unit 110 is different. FIG. 27 is a chart showing the flow of operations of the display control unit 110 in this embodiment. The type of exposure control mask used in the process of FIG. 27 is the same as that shown in FIG. Exposure control masks with different exposure amounts and / or exposure positions are prepared depending on the focal position and the shooting scene mode. This will be described below with reference to these drawings.

  First, the display control unit 110 acquires the total number of image data recorded on the memory card (S1501), and executes face detection of the first image data (S1503, S1505).

  When a face is detected in the target image data (S1507), focus position information among the three exposure control masks of the left exposure mask, center exposure mask, and right exposure mask shown in FIG. The exposure control mask at the focal position corresponding to is determined (S1509).

  On the other hand, if the face cannot be detected in the target image data, the Exif header information is read (S1511), and the presence / absence of the focus position information is determined (S1513). If there is focal position information, the exposure mask shown in FIG. 8A is determined based on the focal position information (S1515).

  If there is no focal position information, the presence / absence of shooting scene information is determined (S1517). If there is a shooting scene, an exposure control mask corresponding to the shooting scene mode information is determined. (S1519).

  The same processing is performed on the subsequent image data to determine the exposure control mask (S1523, S1505).

DESCRIPTION OF SYMBOLS 100 Digital television receiver 105 Image data input part 106 Image data processing part 107 Video memory 108 Display composition part 109 Card slot 110 Display control part 113 Storage part 112 Display 1101 Image display apparatus 1201 Control CPU
1203 Image display unit

Claims (9)

An image display method for displaying a plurality of images in an overlapping manner,
An acquisition step of acquiring shooting scene mode information given in association with each of the plurality of images;
Based on the shooting scene mode information of each image acquired in the acquisition step, when the plurality of images are displayed in an overlapped manner, an exposure area defined corresponding to the shooting scene mode of each image is hidden by another image. Determination step for determining the display position of each image,
In accordance with the display position of each image determined in the determination step, a display step for displaying the plurality of images in an overlapping manner,
An image display method comprising: The image display method according to claim 1, wherein the shooting scene mode includes at least one of an auto shooting mode, a landscape shooting mode, a portrait shooting mode, and a night shooting mode. The image display method according to claim 2, wherein an exposure area determined corresponding to the landscape shooting mode is larger than an exposure area determined corresponding to the person shooting mode. In the determination step, the display position of each image is set so that a predetermined exposure area including the center of the image is not hidden by another image for the image in which the shooting scene mode information cannot be acquired in the acquisition step. the image display method according to any one of claims 1 to 3, characterized in that the determining. The program for making a computer perform the image display method as described in any one of Claims 1 thru | or 4 . An image processing apparatus that generates image data in which a plurality of images are overlapped and displayed,
Acquisition means for acquiring shooting scene mode information given in association with each of the plurality of images;
Based on the shooting scene mode information of each image acquired by the acquisition unit, when the plurality of images are displayed in an overlapped manner, an exposure area defined corresponding to the shooting scene mode of each image is hidden by another image. Determination means for determining the display position of each image,
Generating means for generating image data displayed by superimposing the plurality of images according to the display position of each image determined by the determining means;
An image processing apparatus comprising: The image processing apparatus according to claim 6, wherein the shooting scene mode includes at least one of an auto shooting mode, a landscape shooting mode, a person shooting mode, and a night shooting mode. The image processing apparatus according to claim 7, wherein an exposure area determined corresponding to the landscape shooting mode is larger than an exposure area determined corresponding to the person shooting mode. The determination unit determines the display position of each image so that a predetermined exposure area including the center of the image is not hidden by another image for an image for which the shooting scene mode information cannot be acquired by the acquisition unit. The image processing device according to claim 6 , wherein the image processing device is determined.

Images