JP4752409B2 - Image processing apparatus and method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, method, and program, and more particularly, to an image processing apparatus, method, and program that can display an image more effectively with a simpler operation.

  In a television receiver, a broadcast signal transmitted from a transmission station is received, an image as a television broadcast program is displayed, and sound accompanying the image is output. Conventional television receivers are premised on operating alone, and when a user purchases a new television receiver, the television receiver owned by the user becomes unnecessary, Even if it is still usable, it is often discarded.

  Therefore, when a large number of television receivers are connected and higher functions can be realized than when a single television receiver is connected, the disposal of usable television receivers can be prevented, thereby contributing to effective use of resources. it can.

  In view of this, a technique has been proposed in which, when a large number of display devices such as television receivers are connected and used, higher functions can be realized than when used alone (for example, Patent Documents). (See 1)

  According to the technique of Patent Literature 1, for example, an image can be displayed as a single unit on each of nine display devices, and one image is displayed by an aggregate of display devices including nine display devices. Can be done.

JP 2003-198989 A

  By the way, many application examples are conceivable as a method for displaying an image when a large number of display devices such as television receivers are connected and used. For example, if a part of the image displayed on the display device can be enlarged and displayed on another display device connected to the display device, the image can be displayed more effectively. It is possible to view and enjoy the entire image displayed on the display device and an image in which part of the image is enlarged simultaneously.

  However, in the above-described technique, in order to enlarge and display a part of the image, the user must specify each area on the image to be enlarged, and the user can perform the operation while performing the operation. It was not possible to concentrate on viewing the image displayed on the display device.

  The present invention has been made in view of such a situation, and makes it possible to display an image more effectively with a simpler operation.

In the image processing apparatus according to the aspect of the present invention, the predetermined number of the first region is determined based on the number of pixels satisfying the predetermined condition based on the pixel value among the pixels of the first region on the first image. The feature amount indicating the feature of the second region is calculated, and the second region is obtained based on the number of pixels satisfying the condition among the pixels of the second region obtained by removing the first region from the first image. A calculating means for calculating a feature amount indicating the feature of the region; and a feature amount of the first region by calculating a difference between the feature amount of the first region and the feature amount of the second region; The difference between the feature quantity of the first area and the feature quantity of the second area as a result of comparison by the comparison means comparing the feature quantity of the second area and the comparison means is a predetermined threshold value. If it is more, among the pixels of the first area, the area included the condition is satisfied pixels, have the above characteristics That as a characteristic region, extracting means for extracting from said first region, and display control means for extracting said feature regions that control the display of the second image is enlarged is provided.

  The display control means can control display of the second image so that the first image and the second image are displayed simultaneously.

  The calculation unit may calculate a feature amount indicating a hue of the first region or the second region.

  The calculation unit may calculate a feature amount indicating the brightness of the first region or the second region.

  The calculation means may be configured to calculate a feature amount indicating a saturation of the first area or the second area.

  The calculation unit may calculate a feature amount indicating a ratio of pixels constituting an edge in the first region or the second region.

  The extraction means detects a region in which pixels satisfying the condition are continuous in the first region, and extracts a region having the largest area among the detected regions as the feature region. Can do.

  The extracting means detects a region in which pixels satisfying the condition are continuous on the first region, and among the detected regions, a region closest to the center of the first image is used as the feature region. It can be made to extract.

An image processing method or program according to one aspect of the present invention is based on the number of pixels satisfying a predetermined condition based on a pixel value among pixels of a first region on a first image. Based on the number of pixels that satisfy the above condition among the pixels in the second region obtained by calculating a feature amount indicating a predetermined feature and excluding the first region from the first image , the second And calculating a feature amount indicating the feature of the first region, and obtaining a difference between the feature amount of the first region and the feature amount of the second region, and the feature amount of the first region, When the feature amount of the second region is compared, and the difference between the feature amount of the first region and the feature amount of the second region is equal to or greater than a predetermined threshold as a result of the comparison, of the pixels in the region, the region including the pixel satisfying, as the feature region having the feature, the Extracting from the first region, the extracted the feature regions includes the step of controlling the display of the second image that has been enlarged.

In one aspect of the present invention, the predetermined characteristic of the first region is indicated based on the number of pixels satisfying a predetermined condition based on a pixel value among the pixels of the first region on the first image. The feature of the second region is calculated based on the number of pixels that satisfy the condition among the pixels of the second region obtained by calculating the feature amount and excluding the first region from the first image. Is calculated, and the difference between the feature amount of the first region and the feature amount of the second region is obtained, thereby obtaining the feature amount of the first region and the second region. When the difference between the feature amount of the first region and the feature amount of the second region is equal to or greater than a predetermined threshold as a result of the comparison, the feature amount is compared with the pixel of the first region of, a region containing the pixel satisfying there is, as a characteristic region having the feature, extracted from the first region It is, extracted the feature region has been the display of the second image that has been magnified is controlled.

  As described above, according to one aspect of the present invention, an image can be displayed. In particular, according to one aspect of the present invention, an image can be displayed more effectively with a simpler operation.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. Not something to do.

The image processing apparatus according to an aspect of the present invention provides a predetermined number of pixels in the first region based on the number of pixels that satisfy a predetermined condition based on a pixel value among the pixels in the first region on the first image. The second region is calculated based on the number of pixels that satisfy the condition among the pixels of the second region obtained by calculating a feature amount indicating a feature and excluding the first region from the first image. Calculating means (for example, the calculation unit 662 in FIG. 26), a feature amount of the first region, a feature amount of the first region, and a feature amount of the second region By calculating the difference from the feature amount, a comparison unit (for example, the comparison unit 663 in FIG. 26) that compares the feature amount of the second region and a comparison result of the comparison unit, If the feature amount, the difference between the feature quantity of the second region is greater than a predetermined threshold value, said first region Of Motono, a region included the condition is satisfied pixels, as the feature region having the feature, the first extracting means for extracting from the region (e.g., the extraction unit 651 of FIG. 26), extracted the feature Display control means (for example, the zoom processing unit 113 in FIG. 26) that controls the display of the second image in which the area is enlarged is provided.

  The display control means (for example, the zoom processing unit 113 that executes the process of step S66 in FIG. 7) displays the second image so that the first image and the second image are displayed simultaneously. The display can be controlled.

  The calculation unit (for example, the calculation unit 662 that executes the process of step S343 in FIG. 27) may be configured to calculate a feature amount indicating the hue of the first region or the second region.

  The calculation unit (for example, the calculation unit 662 that executes the process of step S343 in FIG. 27) may be configured to calculate a feature amount indicating the brightness of the first region or the second region.

  The calculation unit (for example, the calculation unit 662 that executes the process of step S343 in FIG. 27) may be configured to calculate a feature amount indicating the saturation of the first region or the second region. .

  The calculation means (for example, the calculation unit 662 that executes the process of step S343 in FIG. 27) is configured to calculate a feature amount indicating a ratio of pixels constituting an edge in the first region or the second region. Can be.

  The extraction means (for example, the extraction unit 651 that executes the process of step S345 in FIG. 27) detects a region in which pixels satisfying the condition are continuous on the first region, and among the detected regions The region with the largest area can be extracted as the feature region.

  The extraction means (for example, the extraction unit 651 that executes the process of step S345 in FIG. 27) detects a region in which pixels satisfying the condition are continuous on the first region, and among the detected regions The region closest to the center of the first image can be extracted as the feature region.

An image processing method or program according to one aspect of the present invention is based on the number of pixels satisfying a predetermined condition based on a pixel value among pixels of a first region on a first image. Based on the number of pixels that satisfy the above condition among the pixels in the second region obtained by calculating a feature amount indicating a predetermined feature and excluding the first region from the first image , the second The feature amount indicating the feature of the region is calculated (for example, step S343 in FIG. 27), and the difference between the feature amount of the first region and the feature amount of the second region is obtained . The feature amount of the first region is compared with the feature amount of the second region (for example, step S344 in FIG. 27), and as a result of the comparison, the feature amount of the first region and the second region are compared. If the difference between the feature quantity of not less than the predetermined threshold value, among pixels of the first area, before The region containing the pixel satisfying, as the feature region having the feature, extracted from the first area (e.g., step S345 in FIG. 27), the extracted second image the feature region is magnified (For example, step S66 in FIG. 7) is included.

  Embodiments to which the present invention is applied will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a configuration example of the appearance of a scalable television system to which the present invention is applied. FIG. 1 shows a scalable television system 11 to which nine (3 × 3) display devices 21-1 to 21-9 can be attached. The scalable television system 11 is configured, for example, by connecting a plurality of units capable of mounting a plurality of display devices at a predetermined mounting position, and increasing or decreasing the number of connected units. By doing so, the number of display devices can be freely increased or decreased.

  For example, a unit capable of attaching three units of display device 21-1, display device 21-4, and display device 21-7 in a line in the vertical direction in the figure, display device 21-2, display device 21-5, And three units of display device 21-8 that can be attached in a row in the vertical direction in the figure, and three units of display device 21-3, display device 21-6, and display device 21-9 in the vertical direction in the figure. The scalable television system 11 of FIG. 1 is configured by connecting three units of units that can be attached in a row in the direction and arranged in a row in the horizontal direction in the drawing.

  The scalable television system 11 is provided with nine display devices, display devices 21-1 to 21-9. The display devices 21-1 to 21-9 are each provided with a display unit for displaying an image on the front surface (front side in the figure), and a part or all of the rear surface constitutes the scalable television system 11 described above. It is attached so as to come into contact with the unit. The display devices 21-1 to 21-9 each have a communication function for performing communication with other display devices.

  Further, each of the nine display devices 21-1 to 21-9 has the same shape and function, for example, receives a broadcast signal, a video signal, etc., and displays an image on the display unit. Or output audio from an audio output unit composed of a speaker or the like. The scalable television system 11 allows each display device to function independently, as well as a set of nine display devices to function as a single display device.

  Hereinafter, when it is not necessary to distinguish each of the display devices 21-1 to 21-9, they are simply referred to as the display device 21.

  By the way, in the scalable television system 11, each display apparatus 21 can be functioned independently, and a separate image can be displayed on each of the display apparatuses 21-1 to 21-9. For example, as shown in FIG. 2, among the nine display devices 21, a predetermined image is displayed on the display device 21-5 located in the center in the figure, and at the same time, the display device 21-5 is surrounded. A part of the image displayed on the display device 21-5 is enlarged and displayed on each of the display devices 21-1 to 21-4 and the display devices 21-6 to 21-9. Can be made.

  In FIG. 2, the display device 21-5 shows (displays) people 41 to 43 in the lower right in the figure of the display unit of the display device 21-5, and the background is a mountain 46 and a mountain. 47, the sun 44, the helicopter 45, and the cloud 48 are displayed (displayed). In addition, in each of the display devices 21-1 to 21-3, an enlarged image of each of the people 41 to 43 displayed on the display device 21-5 is displayed, and the display device 21- 4, an image in which the sun 44 is enlarged is displayed, and an image in which the helicopter 45 is enlarged is displayed on the display device 21-6. Further, the display device 21-7 displays an image in which the mountain 46 displayed on the display device 21-5 is enlarged, and the display device 21-8 displays an image in which the mountain 47 is enlarged. Thus, an image in which the cloud 48 is enlarged is displayed on the display device 21-9.

  In this way, a predetermined image is displayed on one of the plurality of display devices 21, and at the same time, a part of the predetermined image is enlarged and displayed on the other eight (remaining) display devices 21. Thus, the user can view and enjoy the displayed predetermined image (that is, the entire image) and simultaneously view and enjoy the image in which a part of the image is enlarged (that is, the details of the predetermined image).

  In FIG. 2, the display device 21-5 displays a predetermined image (hereinafter also referred to as an entire image), and the display devices 21-1 to 21-4 and the display devices 21-6 to 21-21. Although the example in which each of -9 displays an image in which a part of the entire image is enlarged (hereinafter also referred to as a zoom image) has been described, the display device 21 that displays the entire image is not limited to the display device 21-5. Or a plurality of display devices 21 may simultaneously display the entire image. Similarly, any of the plurality of display devices 21 may display the zoom image, and the plurality of display devices 21 may display the same zoom image.

  FIG. 3 is a block diagram illustrating a configuration example of the scalable television system 11 of FIG.

  The scalable television system 11 is configured to include each of the display devices 21-1 to 21-9 (the display devices 21-3 to 21-8 are not shown). Are connected to a distributor 71, a distributor 72, a receiver 73, and a network hub 74.

  The distributor 71 supplies a broadcast signal received by an antenna (not shown) for reproducing a television broadcast program to the display devices 21-1 to 21-9. The distributor 72 supplies a video signal for reproducing a video (moving image) supplied from an external input device (not shown) to the display devices 21-1 to 21-9.

  Further, for example, when the user operates the remote commander (not shown) to instruct the display device 21 to execute a predetermined process, an infrared signal corresponding to the user operation is transmitted from the remote commander. The receiving unit 73 receives (receives) an infrared signal transmitted from the remote commander, performs photoelectric conversion, and supplies a command signal obtained thereby to the display devices 21-1 to 21-9. The network hub 74 supplies various signals supplied from the display devices 21-1 to 21-9 to the other display devices 21.

  The display device 21 reproduces a television broadcast program or video based on the broadcast signal supplied from the distributor 71 or the video signal supplied from the distributor 72. In addition, the respective display devices 21 communicate with each other via the network hub 74 or perform predetermined processing according to a command signal supplied from the receiving unit 73.

  FIG. 4 is a block diagram illustrating a configuration example of functions of the display device 21.

  The display device 21 is configured to include a control information communication unit 91, an input unit 92, a central processing unit 93, a signal acquisition unit 94, an image processing unit 95, a display unit 96, and an audio output unit 97.

  The control information communication unit 91 is connected to the network hub 74 and controls communication between the display device 21 and another display device 21. The control information communication unit 91 receives various control signals transmitted from other display devices via the network hub 74 and supplies them to the central processing unit 93. Further, the control information communication unit 91 supplies various control signals supplied from the central processing unit 93 to the other display device 21 via the network hub 74. Further, the control information communication unit 91 supplies the command signal supplied from the receiving unit 73 to the central processing unit 93.

  The input unit 92 includes, for example, a mouse, a tablet, a button, a switch, and the like, and supplies an input signal corresponding to a user operation to the central processing unit 93. The central processing unit 93 controls the entire display device 21 according to a control signal supplied from the control information communication unit 91 or an input signal supplied from the input unit 92, and performs various processes.

  For example, the central processing unit 93 controls the image processing unit 95 to display an image on the display unit 96. The central processing unit 93 generates a control signal for controlling the other display device 21, supplies the control signal to the control information communication unit 91, and causes the other display device 21 to transmit the control signal.

  Further, when a signal indicating an ID (Identification) for specifying the display device 21 is supplied from the receiving unit 73 or the input unit 92, the central processing unit 93 stores the ID indicated by the signal. For example, the display devices 21 connected to each other are ordered in advance, and each display device 21 is assigned an ID indicating its own order. For example, the display device 21 is ordered in the order of the display devices 21-1 to 21-9, and the ID of the display device 21-1 is “1”, which is the first in its own order. . The ID may be stored in advance by the central processing unit 93 or may be changed by the user.

  The signal acquisition unit 94 is configured to include, for example, a tuner. The signal acquisition unit 94 selects a channel based on the control of the central processing unit 93, and extracts image data for displaying an image and audio data for reproducing sound from the broadcast signal supplied from the distributor 71. To do. The signal acquisition unit 94 extracts image data and audio data from the video signal supplied from the distributor 72. The signal acquisition unit 94 supplies the extracted image data to the image processing unit 95 and supplies the audio data to the audio output unit 97.

  Under the control of the central processing unit 93, the image processing unit 95 extracts a skin color region from an image (entire image) based on the image data supplied from the signal acquisition unit 94, and enlarges and displays the region. Image data is generated and supplied to the display unit 96. The image processing unit 95 is configured to include an area determination unit 111, a parameter determination unit 112, and a zoom processing unit 113.

  The region determination unit 111 is configured to include a skin color region detection unit 121. The skin color region detection unit 121 of the region determination unit 111 detects a skin color region from an image (entire image) based on the image data supplied from the signal acquisition unit 94. The region determination unit 111 determines (selects) a region that is a candidate for a region to be displayed as a zoom image (hereinafter also referred to as a zoom region) of the entire image based on the detection result of the skin color region.

  The parameter determination unit 112 is configured to include the parameter holding unit 122, and calculates the center coordinates of the region that is a candidate for the zoom region determined by the region determination unit 111 and the zoom magnification for enlarging the region. The parameter holding unit 122 of the parameter determining unit 112 holds the calculated center coordinates and zoom magnification. Further, the parameter determination unit 112 calculates the center of gravity (coordinates) of a predetermined number of center coordinates calculated in the past and the average value of the zoom magnification, and supplies the calculated values to the zoom processing unit 113.

  The zoom processing unit 113 displays a zoom image based on the image data supplied from the signal acquisition unit 94, the center of gravity (the coordinates thereof) of the center coordinates supplied from the parameter determination unit 112, and the average value of the zoom magnification. Image data is generated and supplied to the display unit 96.

  In addition, when displaying the entire image without displaying the zoom image, the zoom processing unit 113 supplies the image data supplied from the signal acquisition unit 94 to the display unit 96 as it is.

  The display unit 96 includes, for example, a liquid crystal display, a CRT (Cathode Ray Tube), and the like, and displays an image based on the image data supplied from the image processing unit 95. The audio output unit 97 includes, for example, a speaker and reproduces audio based on the audio data supplied from the signal acquisition unit 94.

  In the scalable television system 11, the display device 21 that displays the entire image and the display device 21 that displays the zoom image may be determined in advance, and the user can display the entire image for each display device 21. You may enable it to set whether to display or to display a zoom image.

  When the user operates the remote commander to instruct the display device 21 to display an image, the receiving unit 73 receives an infrared signal transmitted from the remote commander, performs photoelectric conversion, and displays a command signal obtained as a result. It supplies to the apparatus 21.

  When the central processing unit 93 of the display device 21 receives a command signal from the receiving unit 73 via the control information communication unit 91, the central processing unit 93 displays an image (entire image or zoom image) on the display unit 96 according to the command signal. .

  Hereinafter, with reference to the flowchart of FIG. 5, the display processing of the entire image by the display device 21 that displays the entire image will be described.

  In step S <b> 11, the signal acquisition unit 94 extracts image data and audio data from the broadcast signal supplied from the distributor 71 or the video signal supplied from the distributor 72 under the control of the central processing unit 93. . Then, the signal acquisition unit 94 supplies the extracted image data to the zoom processing unit 113 and supplies the audio data to the audio output unit 97.

  In step S <b> 12, the zoom processing unit 113 supplies the image data supplied from the signal acquisition unit 94 to the display unit 96 as it is, and causes the display unit 96 to display the entire image.

  In step S <b> 13, the audio output unit 97 reproduces audio based on the audio data supplied from the signal acquisition unit 94.

  In step S14, the central processing unit 93 determines whether or not to end the process. For example, the central processing unit 93 determines to end the processing when a command signal instructing the end of the display of the entire image is supplied from the receiving unit 73 via the control information communication unit 91.

  If it is determined in step S14 that the process is not ended, the process proceeds to step S15, the central processing unit 93 performs a process according to the user's operation, and the process returns to step S11. For example, when the user operates the remote commander to instruct the channel change, the central processing unit 93 responds to the command signal supplied from the receiving unit 73 via the control information communication unit 91 and the signal acquisition unit 94. To select a channel.

  If it is determined in step S14 that the process is to be ended, the central processing unit 93 controls the signal acquisition unit 94 and the image processing unit 95 to end the process of displaying the entire image, thereby displaying the entire image. The process ends.

  In this way, the display device 21 that displays the entire image displays the entire image in accordance with a user operation.

  Next, zoom image display processing by the display device 21 that enlarges and displays a part of the entire image displayed on the other display device 21 will be described with reference to the flowchart of FIG. 6. The zoom image display process is performed independently and in parallel with the entire image display process (FIG. 5) by the other display devices 21 connected to each other.

  In step S31, the central processing unit 93 determines whether display of a zoom image has been instructed. For example, when the user operates the remote commander, an infrared signal instructing display of the zoom image is transmitted from the remote commander to the receiving unit 73, and in response thereto, the receiving unit 73 via the control information communication unit 91. When the command signal for instructing the display of the zoom image is supplied to the central processing unit 93, the central processing unit 93 determines that the display of the zoom image has been instructed.

  If it is determined in step S31 that display of a zoom image has been instructed, the process proceeds to step S32, and the image processing unit 95 performs automatic zoom processing under the control of the central processing unit 93. Although details of the automatic zoom process will be described later, in the automatic zoom process, the image processing unit 95 generates image data for displaying a zoom image based on the image data supplied from the signal acquisition unit 94. Then, the image processing unit 95 supplies the generated image data to the display unit 96 and causes the display unit 96 to display the zoom image simultaneously with the display of the entire image on the other display device 21.

  In step S33, the central processing unit 93 determines whether or not to end the display of the zoom image. For example, the remote commander is operated by the user, and an infrared signal instructing the end of the display of the zoom image is transmitted from the remote commander to the receiving unit 73, and the control information communication unit 91 is received from the receiving unit 73 accordingly. When the command signal for instructing the end of the display of the zoom image is supplied to the central processing unit 93 via the central processing unit 93, the central processing unit 93 determines to end the display of the zoom image.

  If it is determined in step S33 that display of the zoom image is not finished, the process returns to step S32, and the zoom image is continuously displayed on the display unit 96.

  On the other hand, if it is determined in step S33 that display of the zoom image is to be terminated, the process proceeds to step S34. In step S34, the central processing unit 93 controls the signal acquisition unit 94 and the image processing unit 95 to end the display of the zoom image on the display unit 96, and the process returns to step S31.

  If it is determined in step S31 that display of a zoom image is not instructed, the process proceeds to step S35, and the central processing unit 93 determines whether or not to end the process. For example, the central processing unit 93 determines to end the processing when a command signal instructing the end of the display of the entire image is supplied from the receiving unit 73 via the control information communication unit 91.

  If it is determined in step S35 that the process is not ended, the process proceeds to step S36, the central processing unit 93 performs a process according to the user's operation, and the process returns to step S31. For example, when the user operates the remote commander to instruct the channel change, the central processing unit 93 responds to the command signal supplied from the receiving unit 73 via the control information communication unit 91 and the signal acquisition unit 94. To select a channel.

  If it is determined in step S35 that the process is to be terminated, the central processing unit 93 terminates the process performed by the display device 21, and the zoom image display process is terminated.

  In this way, the display device 21 causes the display unit 96 to display a zoom image when an instruction to display a zoom image is given by the user while the entire image is displayed on another display device 21.

  Thus, by displaying the zoom image at the same time as the entire image is displayed, the display device 21 can display the image more effectively. Thus, the user can view the entire image and observe the details of the entire image at the same time with a simple operation of operating the remote commander or the input unit 92 and instructing the display of the zoom image, and enjoy viewing the image. be able to. The user can obtain an unprecedented visual effect and make a new discovery by viewing the image at a new angle of view. In addition, in this case, the user does not have to specify the area (zoom area) to be enlarged and displayed, so that the user can concentrate on viewing the image.

  Next, the automatic zoom process corresponding to the process of step S32 of FIG. 6 will be described with reference to the flowchart of FIG.

  In step S61, the image processing unit 95 performs skin color region detection processing. Although details of the skin color area detection process will be described later, in the skin color area detection process, the area determination unit 111 of the image processing unit 95 detects a skin color area from the entire image, and based on the detection result, A candidate for a zoom area to be displayed as a zoom image of the images is determined. Then, the parameter determination unit 112 of the image processing unit 95 calculates the center coordinates and zoom magnification of the determined zoom region candidate.

  In step S 62, the parameter holding unit 122 holds the center coordinates and zoom magnification of the zoom area candidate calculated by the parameter determination unit 112.

  In step S63, the parameter determination unit 112 obtains the centroid (coordinates) of the past N pieces (where N is a natural number) held in the parameter holding unit 122. The parameter holding unit 122 holds the center coordinates of a region that is a candidate for the zoom region on the entire image of the past frame. The parameter determination unit 112 obtains the center of gravity of the past N center coordinates held by the parameter holding unit 122.

  For example, if the center coordinate in the XY space calculated from the last i-th (where i is a natural number) among the center coordinates held in the parameter holding unit 122 is (Xi, Yi), the parameter determination unit 112. Obtains the coordinates (Xg, Yg) of the center of gravity of the first to Nth center coordinates calculated from the last by calculating Expression (1) and Expression (2).

  Xg = (ΣXi) / N (1)

  Yg = (ΣYi) / N (2)

  Here, Σ in Equation (1) represents that the sum of Xi is changed from 1 to N, and similarly, Σ in Equation (2) is the sum of Yi changed from 1 to N. Represents taking. The value of N is assumed to be predetermined.

  In step S <b> 64, the parameter determination unit 112 calculates the average value of the past N zoom magnifications (for N frames) (where N is a natural number) held in the parameter holding unit 122. For example, if the zoom magnification calculated i-th (where i is a natural number) from among the zoom magnifications held in the parameter holding unit 122 is Zi, the parameter determination unit 112 calculates Equation (3). Thus, the average value Zg of the zoom magnifications calculated from the first to the Nth from the end is calculated.

  Zg = (ΣZi) / N (3)

  Here, Σ in Equation (3) represents that the sum of Zi is changed from 1 to N.

  When the parameter determination unit 112 calculates the center of gravity (Xg, Yg) of the center coordinates and the average value Zg of the zoom magnification, the parameter determination unit 112 supplies the calculated center coordinates of the center coordinates (the coordinates thereof) and the average value of the zoom magnification to the zoom processing unit 113. .

  In step S65, the zoom processing unit 113, based on the image data supplied from the signal acquisition unit 94 and the center of gravity coordinates of the center coordinates supplied from the parameter determination unit 112 and the average value of the zoom magnification, Generate data.

  For example, as shown in FIG. 8, in the figure, the right direction is the X-axis direction, the lower direction is the Y-axis direction, and the entire image area in the XY space is defined as 0 ≦ X ≦ Xd and 0 ≦ Y An area 151 surrounded by a range of ≦ Yd. Further, assuming that the center coordinates of the zoom image are the center of gravity (Xg, Yg) of the center coordinates obtained by the expressions (1) and (2) and the average value of the zoom magnification is Zg, the length of the zoom area in the X direction Wx and the length Hy of the zoom area in the Y direction are obtained by Expression (4) and Expression (5), respectively. Here, in order to simplify the description, it is assumed that the number of pixels of the entire image and the number of pixels of the display screen of the display unit 96 in the display device 21 are the same.

  Wx = Xd / Zg (4)

  Hy = Yd / Zg (5)

  Therefore, the zoom region in FIG. 8 has a range of Xg− (Xd / (2Zg)) ≦ X ≦ Xg + (Xd / (2Zg)) and Yg− (Yd / (2Zg)) ≦ Y ≦ Yg + (Yd / (Yd / ( 2Zg)). The zoom processing unit 113 performs, for example, predetermined image conversion processing on the image data supplied from the signal acquisition unit 94, and an image (zoom image) obtained by enlarging the region 152 in the entire image with the zoom magnification Zg times. Image data to be displayed is generated (converted into image data to display a zoom image).

  Returning to the description of the flowchart of FIG. 7, in step S <b> 66, the zoom processing unit 113 supplies image data of the generated zoom image to the display unit 96 and causes the display unit 96 to display the zoom image. Proceed to step S33.

  In this manner, the image processing unit 95 obtains the center of gravity of the center coordinates of the zoom region candidate and the average value of the zoom magnification. Based on these, a zoom area is determined to generate image data, and a zoom image is displayed.

  In this way, by determining the zoom area by calculating the center coordinates of the center coordinates of the areas that are candidates for the zoom area and the average value of the zoom magnification, the position of the zoom area on the entire image varies greatly from frame to frame. This can be suppressed.

  For example, the center coordinates of the center coordinates, the center coordinates used for obtaining the average value of the zoom magnification, and the number of zoom magnifications, that is, the value of N in the equations (1) to (3) are relatively in the order of 1 to 10. If the value is small, the center coordinate of the zoom area may change (move) greatly every frame. In such a case, the zoom image displayed on the display unit 96 (the area on the entire image to be taken) varies greatly from frame to frame. For example, the face of a person displayed as a zoom image is blurred. , It may be difficult for the user to see.

  Therefore, for example, when displaying an image of a content such as sports, the value of N is set to a relatively small value so that it can follow a rapid change of the scene or a fast movement of a displayed athlete, ball, etc. When displaying an image of a content such as a program, the value of N can be set to a relatively large value of about 30 in order to make it easier for a talker or the like with less movement to see an enlarged image.

  The value of N may be determined in advance for each display device 21, and may be set by the user for each display device 21 according to the scene of the image (content) to be displayed and the user's preference. Also good. Thus, when the value of N is changed for each display device 21, the user selectively views a zoom image that meets the viewing condition at that time among the zoom images displayed on the plurality of display devices 21. be able to.

  Further, without obtaining the center of gravity of the center coordinates and the average value of the zoom magnification, the zoom area candidate determined in step S61 may be used as the zoom area as it is (that is, N is set to 1). The zoom region determined only when the center coordinates and the zoom magnification obtained in the above are significantly different from the previous (N-1) center coordinates and zoom magnification obtained before that. These candidates may be directly used as a zoom area.

  Further, for example, the image of the area 152 as the zoom area shown in FIG. 8 may be displayed on a plurality of display devices 21. In this case, for example, one zoom image (area 152) is displayed on the display device 21 of N × N (N horizontal × N vertical) (where N is a natural number) that are installed adjacent to each other. Image).

  Here, among the display devices 21 that display the zoom image with the region 152 of FIG. 8 as the zoom region, the display device 21 that displays the upper left end region of the region 152 in the figure is the first from the left, and The display device 21 that is the first from the top and the display device 21 that displays the upper right end region of the region 152 is the display device 21 that is the Nth from the left and the first from the top. In the figure, the display device 21 that displays the lower left end region of the region 152 is the display device 21 that is first from the left and Nth from the top.

  Xg− (Xd / (2Zg)) = Xb, Xg + (Xd / (2Zg)) = Xe, Yg− (Yd / (2Zg)) = Yb, and Yg + (Yd / (2Zg)) = Ye And At this time, an area displayed as a zoom image by the display device 21 that is i-th (1 ≦ i ≦ N) from the left and j-th (1 ≦ j ≦ N) from the top is defined as a range of Xib ≦ X ≦ Xie. , Yjb ≦ Y ≦ Yje, the values of Xib, Xie, Yjb, and Yje are values represented by the equations (6) to (9), respectively.

  Xib = Xb + ((i−1) × ((Xe−Xb) / N)) (6)

  Xie = Xb + (i × ((Xe−Xb) / N)) (7)

  Yjb = Yb + ((j−1) × ((Ye−Yb) / N)) (8)

  Yje = Yb + (j × ((Ye−Yb) / N)) (9)

  Therefore, the display device 21 that is i-th (1 ≦ i ≦ N) from the left and j-th (1 ≦ j ≦ N) from the top has a range of Xib ≦ X ≦ Xie in the region 152 of FIG. , Yjb.ltoreq.Y.ltoreq.Yje. The image data for displaying an image (zoom image) in which the region surrounded by the range of Yjb.ltoreq.Y.ltoreq.Yje is enlarged at the zoom magnification Zg times is generated, and the image is displayed on the display unit 96. Thereby, an image (zoom image) in which the region 152 is enlarged at the zoom magnification Zg times is displayed by the N × N display devices 21 installed adjacent to each other.

  As described above, one zoom area is divided into a plurality of continuous areas, and one divided area is displayed on one display device 21, whereby one image is displayed on the plurality of display devices 21. Can do. As a result, the zoom image can be displayed with a larger zoom magnification, and the user can observe some details of the entire image.

  The plurality of display devices 21 that display one zoom image may be determined in advance, or may be determined by the user by operating the remote commander or the input unit 92.

  Next, a skin color area detection process corresponding to the process of step S61 of FIG. 7 will be described with reference to the flowchart of FIG. As described above, in the skin color area detection process, the image processing unit 95 detects a skin color area from the entire image, and determines (selects) a candidate area for the zoom area.

  It should be noted that on the whole image, the area that is the target for detecting the skin color area may be the whole whole image or a part of the whole image. Further, the area to be detected for the skin color area may be predetermined for each display device 21 or may be set by the user.

  In step S <b> 91, the skin color area detection unit 121 selects one pixel in an area to be detected as a skin color area on the image (overall image) of the image data supplied from the signal acquisition unit 94.

  In step S92, the skin color area detection unit 121 obtains the value of H (Hue) in the color space of the HSV (Hue Saturation value Value) format based on the pixel value of the selected pixel.

  For example, the red (R), green (G), and blue (B) values of the pixel value of the selected pixel are R, G, and B (however, the values of R, G, and B are 0 to 1). In addition, the maximum value among the R, G, and B values is MAX (R, G, B), and the minimum value among the R, G, and B values is MIN (R, G, B). B). Furthermore, when the values of R, G, and B are 0, the value of H cannot be defined.

  When the value of R is the maximum among the values of R, G, and B, the skin color area detection unit 121 calculates the value of H by calculating Expression (10).

  H = 60 × (GB) / D (10)

  Here, D is a difference between MAX (R, G, B) and MIN (R, G, B), and is expressed by Expression (11).

  D = MAX (R, G, B) −MIN (R, G, B) (11)

  Note that when the value of H obtained by Expression (10) is smaller than 0, that is, when the value of H becomes a negative value, the skin color area detection unit 121 does not use Expression (10) but Expression (12). The value of H is obtained by calculating.

  H = (60 × (GB) / D) +360 (12)

  When the value of G is the maximum among the values of R, G, and B, the skin color area detection unit 121 calculates the value of H by calculating Expression (13), and the value of B is the maximum. In this case, the skin color area detection unit 121 calculates the value of H by calculating Expression (14).

  H = 60 × (2 + ((BR) / D)) (13)

  H = 60 × (4 + ((RG) / D)) (14)

  The value of H obtained in this way is a value of 0 or more and less than 360. Here, FIG. 10 shows an example of the value of H calculated based on the R, G, and B values. For example, when the value of R is 0.39, the value of G is 0.78, and the value of B is 0.39, the value of G is the largest. Therefore, from the equation (13), the value of H is 120 (= 60 × (2 + ((0.39−0.39) /0.39))). In addition, when the value of R is 0.78, the value of G is 0.39, and the value of B is 0.39, the value of R is the largest. Therefore, from the equation (10), the value of H is 0 (= 60 × (0.39−0.39) /0.39).

  Similarly, when the value of R is 0.39, the value of G is 0.39, and the value of B is 0.78, since the value of B is the largest, the value of H is 240 (= 60 × (4 + ((0.39−0.39) /0.39))). Further, when the value of R is 0.78, the value of G is 0.39, and the value of B is 0.78, the value of R is the largest, and the values of R, G, and B are substituted into Expression (10). The value obtained is −60 (= 60 × (0.39−0.78) /0.39), which is a negative value. From the equation (12), the value of H is 300 (= (60 × (0.39−0.78) / 0.39) +360).

  Further, when the value of R is 0.78, the value of G is 0.78, and the value of B is 0.39, the value of R is the largest, so that the value of H is 60 (= 60 from Equation (10). X (0.78−0.39) /0.39)), when the value of R is 0.39, the value of G is 0.78, and the value of B is 0.78, the value of G is the largest. , H is 180 (= 60 × (2 + ((0.78−0.39) /0.39))).

  Returning to the description of the flowchart of FIG. 9, when the value of H is obtained, in step S <b> 93, the skin color area detection unit 121 determines whether or not the obtained value of H is a value within a predetermined range. judge. For example, the skin color area detection unit 121 determines whether the value of H is larger than 0 and smaller than 60. If each of the R, G, and B pixel values of the selected pixel is 0 and the H value is not defined, it is determined in step S93 that the value is not within the predetermined range.

  If it is determined in step S93 that the value is within the predetermined range, the process proceeds to step S94, and the skin color area detection unit 121 sets the selected pixel as a skin color pixel. For example, as shown in FIG. 11, the skin color area detection unit 121 sets a flag as to whether or not the pixel is a skin color pixel for each pixel in the area to be detected as a skin color area.

  In FIG. 11, one square represents one pixel, and “1” or “0” in the square represents a flag indicating whether the pixel is a skin color pixel. A flag “1” in the square indicates that the pixel is a flesh-colored pixel, and a flag “0” in the square indicates that the pixel is not a flesh-colored pixel.

  When the selected pixel is a skin color pixel, the skin color area detection unit 121 sets “1” to the flag of the pixel (sets the flag). In the example of FIG. 11, each of the four pixels included in the region 181, the two pixels included in the region 182, and the three pixels included in the region 183 are skin-colored pixels. Pixels that are not skin-colored.

  Returning to the description of the flowchart of FIG. 9, if the selected pixel is a flesh-colored pixel in step S94, the process proceeds to step S96.

  On the other hand, if it is determined in step S93 that the value is not within the predetermined range, the process proceeds to step S95, and the selected pixel is set as a pixel other than the skin color. For example, as shown in FIG. 11, the skin color area detection unit 121 sets “0” to the flag of the selected pixel (resets the flag).

  In step S94, if the selected pixel is a skin color pixel, or if the selected pixel is a pixel other than the skin color in step S95, the skin color region detection unit 121 detects a skin color region in step S96. It is determined whether or not all the pixels in the target area have been selected.

  If it is determined in step S96 that all the pixels have not yet been selected, the process returns to step S91 to select the next pixel and determine whether or not the pixel is a skin color pixel.

  On the other hand, if it is determined in step S96 that all the pixels have been selected, the process proceeds to step S97, where the area determination unit 111 determines (selects) an area that is a candidate for the zoom area on the entire image. When the region determination unit 111 determines a region that is a candidate for the zoom region, the region determination unit 111 generates information indicating the region and supplies the information to the parameter determination unit 112.

  For example, the region determination unit 111 includes, in the XY space, the maximum value and the minimum value of the X coordinates of the pixels that are the skin color on the entire image (the region that is the target for detecting the skin color region), and the pixels that are the skin color The maximum value and the minimum value of the Y coordinates are calculated, and an area including all skin color pixels is determined as a candidate area for the zoom area. Here, if the maximum value and the minimum value of the X coordinate of the skin color pixel are Xmax and Xmin, respectively, and the maximum value and the minimum value of the Y coordinate of the skin color pixel are Ymax and Ymin, respectively, the region is determined. The unit 111 determines a region including a region surrounded by a range of Xmin ≦ X ≦ Xmax and a range of Ymin ≦ Y ≦ Ymax on the XY space as a candidate for a zoom region. It should be noted that the region that is a candidate for the zoom region is determined to have a similar shape (the same aspect ratio) as the display screen of the display unit 96.

  In addition, the region determination unit 111 detects a region in which skin-colored pixels are continuous on a region that is a target for detecting a skin-colored region, and selects a region having the largest area (number of pixels) among the detected regions. You may make it be the area | region used as the candidate of a zoom area | region. In this case, in the example illustrated in FIG. 11, the regions 181 to 183 are detected as regions where skin color pixels are continuous, and the region including the region 181 having the largest area is determined as a region that is a candidate for the zoom region. .

  Further, the region determination unit 111 detects a region in which skin color pixels are continuous on a region to be detected as a skin color region, and includes a region closest to the center of the entire image among the detected regions. The area may be a candidate area for the zoom area.

  When the zoom region candidate region is determined, in step S98, the parameter determination unit 112 calculates the center coordinates of the zoom region candidate region based on the information indicating the zoom region candidate region. For example, assuming that a candidate region for the zoom region is a region surrounded by a range of Xmin ≦ X ≦ Xmax and a range of Ymin ≦ Y ≦ Ymax in the XY space, the parameter determination unit 112 uses the center coordinates ( X-coordinate Xi and Y-coordinate Yi of Xi, Yi) are calculated by equations (15) and (16).

  Xi = (Xmin + Xmax) / 2 (15)

  Yi = (Ymin + Ymax) / 2 (16)

  In step S99, the parameter determination unit 112 calculates the zoom magnification of the region that is a candidate for the zoom region, and the process proceeds to step S62 in FIG. For example, if the length (number of pixels) in the X direction of the region that is a candidate for the zoom region in the XY space is Xid, and the length (number of pixels) in the X direction of the display screen of the display unit 96 is Xd, The determination unit 112 calculates the zoom magnification Zi using Expression (17).

  Zi = Xd / Xid (17)

  In this way, the image processing unit 95 detects the skin color region, and determines a candidate region for the zoom region based on the detection result.

  In this way, by detecting the skin color area and determining the candidate areas for the zoom area based on the detection result, the user does not need to specify the zoom area one by one, and the entire image can be obtained with a simpler operation. You can enjoy watching images by observing the details.

  In addition, by detecting a skin color region and determining a region that is a candidate for a zoom region, a region that includes a person (that is, a skin color region) that is often noticed by the user on the entire image, It can be enlarged and displayed effectively as a zoom image.

  Note that a method for determining a region that is a candidate for the zoom region based on the detection result of the skin color region may be different for each display device 21. For example, the display device 21-1 sets a region including all skin color pixels as a candidate region, and the display device 21-2 includes a region having the largest area among regions where skin color pixels are continuous. The region can be a candidate region.

  Further, the range of the value of H used for determining whether or not the selected pixel is a skin color pixel may be changed for each display device 21. Accordingly, the display device 21 can detect a relatively light skin color region or a relatively dark skin color region even in the skin color region. Furthermore, in the above description, the skin color region is detected from the entire image. However, a region other than the skin color is detected to be a zoom region, or the S or V value of the HSV format color space is calculated, Of the pixels on the entire image, an area including a pixel that satisfies a predetermined condition based on the value of S or V may be detected and used as a zoom area.

  Incidentally, in the automatic zoom process described with reference to the flowchart of FIG. 7, it has been described that in step S <b> 61 (skin color area detection process), a skin color area is detected to determine a zoom area candidate area. In the process of S61, an object which is a part (or all) of what is called a foreground (for example, an object or a person) is detected from the entire image, and an area including the detected object is selected as a zoom area candidate. It can also be set as an area.

  In such a case, the display device is configured as shown in FIG. 12, for example.

  The display device 211 is configured to include a control information communication unit 91, an input unit 92, a central processing unit 93, a signal acquisition unit 94, a display unit 96, an audio output unit 97, and an image processing unit 231. The control information communication unit 91, the input unit 92, the central processing unit 93, the signal acquisition unit 94, the display unit 96, and the audio output unit 97 of the display device 211 correspond to the case in FIG. The description is omitted.

  Under the control of the central processing unit 93, the image processing unit 231 detects a predetermined object from an image (entire image) based on the image data supplied from the signal acquisition unit 94, and enlarges a region including the object. The image data to be displayed is generated and supplied to the display unit 96. The image processing unit 231 is configured to include an object selection unit 241, a parameter determination unit 112, and a zoom processing unit 113. It should be noted that the parameter determination unit 112 and the zoom processing unit 113 are assigned the same reference numerals in order to correspond to the case in FIG.

  The object selection unit 241 detects an object from an image (entire image) based on the image data supplied from the signal acquisition unit 94. The object selection unit 241 selects an object to be enlarged and displayed from the detected objects, and supplies information indicating the object to the parameter determination unit 112. The object selection unit 241 is configured to include a detection unit 251 and a variable holding unit 252.

  The variable holding unit 252 of the object selection unit 241 holds a predetermined variable necessary for detecting an object. The detection unit 251 of the object selection unit 241 detects an object from the entire image using the variable held in the variable holding unit 252.

  The display device 211 performs the automatic zoom process described with reference to FIG. 7, and determines an area including the object as a candidate zoom area in the process of step S <b> 61. Hereinafter, the object region detection processing by the image processing unit 231 will be described with reference to the flowchart of FIG.

  In step S121, the object selection unit 241 performs object detection processing. Although details of the object detection process will be described later, in the object detection process, the object selection unit 241 detects an object from the entire image.

  It should be noted that the area on which the object is detected on the whole image may be the whole whole image or a part of the whole image. Furthermore, an area that is a target for detecting an object may be predetermined for each display device 211 or may be set by a user.

  In step S122, the object selection unit 241 selects an object candidate to be displayed as a zoom image from the detected objects.

  For example, the object selection unit 241 displays a relatively small object, assuming that an object of a certain size among the detected objects is displayed in a sufficient size without being enlarged and displayed. Select as a candidate for an object. The object selection unit 241 calculates the area of each detected object (the number of pixels of the object), and selects an object whose object area is equal to or less than a predetermined value from among the objects as a candidate.

  In addition, an object having an area of about 10 pixels may be excluded from the candidate objects to be displayed, assuming that it is noise. In this case, the object selection unit 241 selects an object whose area is a value within a predetermined range as a candidate from the detected objects.

  Furthermore, the object selection unit 241 selects a predetermined number of objects from the detected objects in the order of decreasing area, and sets the selected objects as candidate objects for displaying the detected objects. Alternatively, an average value of the areas may be obtained, and an object smaller than 1/10 of the obtained average value may be selected as a candidate for an object to be displayed.

  When an object candidate to be displayed as a zoom image is selected, in step S123, the object selection unit 241 selects an object to be displayed as a zoom image from the selected object candidates. When the object selection unit 241 selects an object, the object selection unit 241 generates information indicating the selected object and supplies the information to the parameter determination unit 112.

  When there is one object selected as a candidate, the object selection unit 241 selects that object as an object to be displayed as a zoom image. Further, when there are a plurality of objects selected as candidates, the object selection unit 241 selects one or a plurality of objects from among the plurality of candidates as objects to be displayed.

  When selecting an object from among a plurality of candidates, for example, the object selection unit 241 selects a predetermined number of objects in ascending order of the area of the object, or the predetermined number of objects in the order in which the object positions are closer to the center of the entire image. Or select the number of objects. Further, the object selection unit 241 may select an object of a predetermined color as an object to be displayed as a zoom image from among a plurality of candidates. In this case, for example, a human image can be displayed as a zoom image by selecting a skin color object from a plurality of candidates.

  Further, for example, the object selection unit 241 selects all objects selected as candidates as objects to be displayed as a zoom image, or selects a plurality of objects adjacent to each other as a zoom image from among a plurality of candidates. You may make it select as one object to display.

  In this manner, an object that satisfies a predetermined condition can be selected by selecting a candidate for an object to be displayed from the detected objects and further selecting an object to be displayed from the candidates.

  When an object to be displayed as a zoom image is selected, in step S124, the parameter determination unit 112, based on the information indicating the object to be displayed as a zoom image supplied from the object selection unit 241, the object indicated by the information The region including the is a region that is a candidate for the zoom region, and the center coordinates of the region are calculated.

  For example, when the object selection unit 241 selects one object as an object to be displayed as a zoom image, the parameter determination unit 112 calculates the coordinates of the center of gravity of the selected object in the XY space as the center coordinate.

  Here, each of N pixels (where N is a natural number) constituting the object selected by the object selection unit 241 is a pixel Gk (1 ≦ k ≦ N), and the coordinates of the pixel Gk in the XY space are ( Xk, Yk), the parameter determination unit 112 calculates the X coordinate Xi and the Y coordinate Yi of the center coordinate (Xi, Yi) by using the equations (18) and (19).

  Xi = (ΣXk) / N (18)

  Yi = (ΣYk) / N (19)

  Here, Σ in the equation (18) represents that the sum of Xk is changed from 1 to N, and Σ in the equation (19) is the sum of Yk changed from 1 to N. Represents.

  For example, when the object selection unit 241 selects a plurality of objects as objects to be displayed as a zoom image, the parameter determination unit 112 sets a region including the selected plurality of objects as a zoom region candidate region. As a center coordinate of the area in the XY space is calculated. Furthermore, even when only one object is selected, the area including the object may be a candidate area for the zoom area, and the coordinates of the center of the area in the XY space may be used as the center coordinates. Even when there are a plurality of objects, the coordinates of the center of gravity of the pixels constituting each selected object in the XY space may be set as the center coordinates.

  In step S125, the parameter determination unit 112 calculates the zoom magnification of the region that is a candidate for the zoom region, and the process proceeds to step S62 in FIG. For example, the parameter determination unit 112 calculates the zoom magnification Zi by calculating Equation (17) described above.

  In this way, the image processing unit 231 detects an object from the entire image, and sets an area including an object that satisfies a predetermined condition among the detected objects as a candidate for a zoom area. As a result, an image in which an object satisfying a predetermined condition is enlarged is displayed on the display unit 96 as a zoom image.

  For example, as shown in FIG. 14A, it is assumed that the entire image 281 is an image in which a person 291 and a mountain as a background are shown (displayed). The display device 211 (the image processing unit 231) detects an object from the entire image 281. For example, among the detected objects, an object that displays an object whose area is equal to or smaller than a predetermined value. Are selected as candidates, and one object is selected from the candidates.

  As illustrated in FIG. 14B, when the portion of the face 292 of the person 291 that is displayed (displayed) in the entire image 281 is selected as an object that the display device 211 displays as a zoom image, the display device 211 A zoom image 301 shown in FIG. 14C is displayed on the display unit 96. In FIG. 14C, the zoom image 301 displays a face 292 as the selected object.

  Further, for example, as shown in FIG. 15A, it is assumed that the entire image 321 is an image in which a person 331, a flying object 332, and a mountain as a background are shown (displayed). The display device 211 detects an object from the entire image 321 and selects, for example, an object whose object area is equal to or smaller than a predetermined value among the detected objects as a candidate for an object to be displayed.

  As a result, for example, as shown in FIG. 15B, the face 341 of the person 331, the body 342 of the person 331, and the flying object 332 are selected from the entire image 321 as object candidates to be displayed as a zoom image.

  When the display device 211 selects all candidates as objects to be displayed as zoom images, the display device 211 displays a zoom image 351 illustrated in FIG. 15C on the display unit 96. In FIG. 15C, the zoom image 351 displays a face 341 and a torso 342 of a person 331 as a selected object, and a flying object 332.

  For example, when the display device 211 selects a plurality of objects adjacent to each other as an object to be displayed as a zoom image, the display device 211 is an object adjacent to each other on the entire image 321 in FIG. 15B. The face 341 and the torso 342 of the person 331 are selected, and a zoom image 352 shown in FIG. In FIG. 15D, the zoom image 352 displays a person 331 including a face 341 and a torso 342 as selected objects.

  Further, for example, when the display device 211 selects a candidate with the smallest area as an object to be displayed as a zoom image, the display device 211 is a flying object 332 that is the object with the smallest area on the entire image 321 in FIG. 15B. And the zoom image 353 shown in FIG. 15E is displayed on the display unit 96. In FIG. 15E, the zoom image 353 displays a flying object 332 as an object selected as the object having the smallest area.

  In this manner, the image processing unit 231 detects an object from the entire image, and among the detected objects, an area including an object that satisfies a predetermined condition is set as a zoom area candidate. Therefore, the user designates the zoom area one by one. This makes it possible to observe the details of the entire image and enjoy viewing the image with a simpler operation.

  In addition, by selecting an object that satisfies a predetermined condition from the detected objects, it is possible to enlarge and effectively display an object that meets the user's needs (needs). As a result, the user can observe the details of the object he / she wants to see on the entire image.

  In particular, in a questionnaire conducted by the present applicant, among the objects displayed in the entire image, there was a result that there were many people who answered that they were small objects other than people as objects to be enlarged and displayed. .

  Therefore, as described above, an object whose area is equal to or less than a predetermined value is selected as a candidate, and among the detected objects, a relatively small (small area) object is selected as an object of the zoom image. By doing so, it is possible to automatically detect a relatively small object among the objects on the entire image and display it in an enlarged manner. Thereby, the object that the user wants to enlarge and see on the entire image can be enlarged and displayed more easily. Since the object that the user wants to see is enlarged and displayed, the user does not need to stare at a small object on the entire image, and the stress on the user when viewing the entire image is reduced.

  Note that the zoom image object is not limited to a small object, and an object larger than a predetermined value or an object close to the center of the entire image may be selected.

  Next, the object detection process corresponding to the process of step S121 of FIG. 13 will be described with reference to the flowchart of FIG. As described above, in this object detection process, the object selection unit 241 detects an object from the entire image.

  In step S151, the detection unit 251 performs initialization processing. For example, the detection unit 251 sets the object numbers of all the pixels included in the region that is the object detection target to 0. Here, the object number refers to a number for specifying an object on the entire image, and the detection unit 251 classifies all the pixels by assigning a predetermined number of 1 or more to the object number. Object number 0 indicates that the pixel has not been classified into any object yet.

  For example, if the region 381 shown in FIG. 17 is a region to be an object detection target, the detection unit 251 attaches the object number 0 to all the pixels included in the region 381 in the initialization process. In the drawing, one square included in the region 381 represents one pixel.

  Also, in the figure, the pixel that is i-th in the right direction and j-th in the downward direction with the pixel at the top left as a reference is the pixel G (i, j), and the object number of the pixel G (i, j) Is Obj (i, j). After performing the initialization process, the detection unit 251 changes all the pixels G (i, j) included in the region 381 by changing the variable i and the variable j for specifying the pixel G (i, j). On the other hand, by assigning an object number Obj (i, j), the pixel G (i, j) is classified into an object indicated by the object number Obj (i, j).

  In the following description, the pixel in the rightmost column in the drawing of the region 381 is a pixel G (w, j), and the pixel in the bottom row in the drawing of the region 381 is a pixel G (i, h). explain. Therefore, in the example of FIG. 17, w = 6 and h = 4.

  Returning to the description of the flowchart of FIG. 16, when initialization processing is performed, in step S152, the detection unit 251 sets the variable i, the variable j, and the variable num held by the variable holding unit 252 to i = 1. , J = 1, and num = 1.

  Here, the variable i and the variable j are variables for specifying the pixel G (i, j). In a state where i = 1 and j = 1, the pixel G (i, j) Let G (1,1) denote. Therefore, in the example of FIG. 17, when i = 1 and j = 1, the pixel G (i, j) indicates the upper leftmost pixel in the drawing. A variable num is a variable indicating an object number assigned to each pixel G (i, j).

  In step S153, the detection unit 251 determines whether or not the object number Obj (i, j) = 0. For example, when initialization processing is performed and the variables i, j, and num are i = 1, j = 1, and num = 1, that is, when the process proceeds from step S152 to step S153, The object numbers of all the pixels are 0 and are not classified as objects. In this case, the pixel G (i, j) is G (1,1) because i = 1 and j = 1, and the object number Obj (1,1) of G (1,1) is also 0. Therefore, the detection unit 251 determines that Obj (i, j) = 0.

  If it is determined in step S153 that Obj (i, j) = 0, the process proceeds to step S154, and the detection unit 251 sets Obj (i, j) = num. For example, when the variable i, variable j, and variable num held by the variable holding unit 252 are i = 1, j = 1, and num = 1, the detection unit 251 uses Obj (1, 1). = 1.

  In step S155, the detection unit 251 performs difference calculation processing. Although details of the difference calculation process will be described later, in the difference calculation process, the detection unit 251 detects the pixel value of the pixel G (i, j) specified by the variable i and variable j held by the variable holding unit 252. And a pixel G (i, j−1), a pixel G (i−1, j), a pixel G (i + 1, j), and a pixel G (i, j + 1) adjacent to the pixel G (i, j), respectively. The difference from the pixel value of is calculated. The detection unit 251 assigns the same number as the object number Obj (i, j) of the pixel G (i, j) to the object number of the pixel whose absolute value of the calculated difference is smaller than a predetermined threshold value. And the same object as the pixel G (i, j).

  In step S156, the detection unit 251 increments the variable num held by the variable holding unit 252 and proceeds to step S157.

  If it is determined in step S153 that Obj (i, j) = 0 is not satisfied, the pixel G (i, j) has already been assigned an object number and is classified as an object. The processes from step S156 to step S156 are skipped, and the process proceeds to step S157.

  If the variable num is incremented in step S156, or if it is determined in step S153 that Obj (i, j) = 0 is not satisfied, the detection unit 251 determines that the variable holding unit 252 holds the variable in step S157. It is determined whether i is i = w. For example, when the region that is the target for detecting the object is the region 381 illustrated in FIG. 17 and the variable i is i = 6, the detection unit 251 determines that i = w.

  If it is determined in step S157 that i = w is not satisfied, the process proceeds to step S158, and the detection unit 251 increments the variable i held by the variable holding unit 252 and returns to step S153. In this way, the detection unit 251 changes the variable i from 1 to w until all the pixels G (i, j) (all the j-th rows are detected) until it is determined in step S157 that i = w. Pixel) is assigned an object number and is classified into an object.

  If it is determined in step S157 that i = w, the process proceeds to step S159, and the detection unit 251 determines whether or not the variable j held by the variable holding unit 252 is j = h. To do. For example, when the region that is the target for detecting the object is the region 381 illustrated in FIG. 17 and the variable j is j = 4, the detection unit 251 determines that j = h.

  If it is determined in step S159 that j = h is not satisfied, since all the pixels have not yet been classified as objects, the process proceeds to step S160, and the detection unit 251 determines the variable i held in the variable holding unit 252 as i = 1.

  In step S161, the detection unit 251 increments the variable j held by the variable holding unit 252 and returns to step S153. As described above, the detection unit 251 changes the variable j from 1 to h and assigns an object number to each pixel G (i, j) until it is determined in step S159 that j = h. Classify into objects.

  If it is determined in step S159 that j = h, all the pixels are classified as objects, and the process proceeds to step S122 in FIG.

  In this way, the detection unit 251 classifies all the pixels included in the object detection target area by assigning the object number to the object. For example, when detecting an object from the area 381 shown in FIG. 17, the detection unit 251 selects each pixel in the top row in the drawing in order from the leftmost pixel to the rightmost pixel. To the next row, select each pixel in the next row in order from the leftmost pixel to the rightmost pixel and classify them as objects, and then from the top row to the bottom row Each pixel is classified into an object in order.

  Here, pixels having the same object number are pixels included in the same object (constitute the object). In other words, one object is an area on the entire image made up of pixels to which an object number specifying the object is attached.

  Next, the difference calculation process corresponding to the process of step S155 of FIG. 16 will be described with reference to the flowcharts of FIGS.

  In step S201, the detection unit 251 determines whether or not the object number of the pixel G (i, j-1) adjacent to the pixel G (i, j) is Obj (i, j-1) = 0. To do. That is, the detection unit 251 determines whether or not the pixel G (i, j−1) adjacent to the pixel G (i, j) has already been classified as an object.

  If it is determined in step S201 that Obj (i, j-1) = 0, the pixel G (i, j-1) is not yet classified as an object, so the process proceeds to step S202, and the detection unit 251 is performed. Calculates the difference between the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i, j-1).

  For example, based on the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i, j−1), the detection unit 251 determines the luminance value of the pixel G (i, j) and the pixel G ( The luminance value of i, j-1) is calculated, and the difference between the luminance values obtained as a result is obtained, whereby the pixel value of the pixel G (i, j) and the pixel of the pixel G (i, j-1) Calculate the difference from the value.

  In step S203, the detection unit 251 determines whether the absolute value of the difference between the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i, j-1) is smaller than a predetermined threshold value. Determine whether or not.

  If it is determined in step S203 that the absolute value of the difference is smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i, j-1) are assumed to be pixels of the same object. In step S204, the detection unit 251 sets the object number of the pixel G (i, j-1) as Obj (i, j-1) = num. For example, when the variable num held in the variable holding unit 252 is num = 1, the detection unit 251 sets Obj (i, j−1) = 1.

  In step S205, the detection unit 251 decrements the variable j held in the variable holding unit 252.

  In step S206, the detection unit 251 performs difference calculation processing. Note that the difference calculation processing in step S206 is the same as the difference calculation processing described with reference to FIGS.

  In step S207, the detection unit 251 increments the variable j held in the variable holding unit 252, and the process proceeds to step S208.

  If it is determined in step S203 that the absolute value of the difference is not smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i, j-1) are pixels of the same object. If not, the process from step S204 to step S207 is skipped, and the process proceeds to step S208.

  Furthermore, if it is determined in step S201 that Obj (i, j-1) = 0, the pixel G (i, j-1) has already been classified as an object, so the process from step S202 to step S207 are performed. This process is skipped and the process proceeds to step S208.

  In step S201, it is determined that Obj (i, j-1) = 0 is not satisfied, in step S203, it is determined that the absolute value of the difference is not smaller than a predetermined threshold value, or in step S207. In step S208, the detection unit 251 determines that the object number of the pixel G (i-1, j) adjacent to the pixel G (i, j) is Obj (i-1, j). ) = 0.

  If it is determined in step S208 that Obj (i−1, j) = 0, the pixel G (i−1, j) has not yet been classified as an object, so the process proceeds to step S209, and the detection unit 251 is performed. Calculates the difference between the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i-1, j).

  In step S210, the detection unit 251 determines whether the absolute value of the difference between the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i-1, j) is smaller than a predetermined threshold value. Determine whether or not.

  If it is determined in step S210 that the absolute value of the difference is smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i-1, j) are determined to be pixels of the same object. In step S211, the detection unit 251 sets the object number of the pixel G (i−1, j) as Obj (i−1, j) = num.

  In step S <b> 212, the detection unit 251 decrements the variable i held in the variable holding unit 252.

  In step S213, the detection unit 251 performs difference calculation processing. Note that the difference calculation processing in step S213 is the same as the difference calculation processing described with reference to FIGS.

  In step S214, the detection unit 251 increments the variable i held in the variable holding unit 252, and the process proceeds to step S215.

  If it is determined in step S210 that the absolute value of the difference is not smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i-1, j) are pixels of the same object. If not, the process from step S211 to step S214 is skipped, and the process proceeds to step S215.

  Further, if it is determined in step S208 that Obj (i-1, j) = 0, the pixel G (i-1, j) has already been classified as an object, so the processing from step S209 to step S214. This process is skipped, and the process proceeds to step S215.

  In step S208, it is determined that Obj (i-1, j) = 0 is not satisfied, in step S210, it is determined that the absolute value of the difference is not smaller than a predetermined threshold value, or in step S214. When the variable i is incremented, in step S215, the detection unit 251 determines that the object number of the pixel G (i + 1, j) adjacent to the pixel G (i, j) is Obj (i + 1, j) = 0. It is determined whether or not there is.

  If it is determined in step S215 that Obj (i + 1, j) = 0, the pixel G (i + 1, j) is not yet classified as an object, so the process proceeds to step S216, and the detection unit 251 The difference between the pixel value of (i, j) and the pixel value of pixel G (i + 1, j) is calculated.

  In step S217, the detection unit 251 determines whether or not the absolute value of the difference between the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i + 1, j) is smaller than a predetermined threshold value. Determine.

  If it is determined in step S217 that the absolute value of the difference is smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i + 1, j) are assumed to be pixels of the same object. In step S218, the detection unit 251 sets the object number of the pixel G (i + 1, j) as Obj (i + 1, j) = num.

  In step S219, the detection unit 251 increments the variable i held in the variable holding unit 252.

  In step S220, the detection unit 251 performs difference calculation processing. Note that the difference calculation processing in step S220 is the same as the difference calculation processing described with reference to FIGS.

  In step S221, the detection unit 251 decrements the variable i held in the variable holding unit 252, and the process proceeds to step S222.

  If it is determined in step S217 that the absolute value of the difference is not smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i + 1, j) are not pixels of the same object. Then, the process from step S218 to step S221 is skipped, and the process proceeds to step S222.

  Furthermore, if it is determined in step S215 that Obj (i + 1, j) = 0, the pixel G (i + 1, j) has already been classified as an object, so the processing from step S216 to step S221 is skipped. Then, the process proceeds to step S222.

  In step S215, it is determined that Obj (i + 1, j) = 0, or in step S217, it is determined that the absolute value of the difference is not smaller than a predetermined threshold value, or in step S221, When the variable i is decremented, in step S222, the detection unit 251 determines whether the object number of the pixel G (i, j + 1) adjacent to the pixel G (i, j) is Obj (i, j + 1) = 0. Determine whether or not.

  If it is determined in step S222 that Obj (i, j + 1) = 0, the pixel G (i, j + 1) has not yet been classified as an object, so the process proceeds to step S223, and the detection unit 251 The difference between the pixel value of (i, j) and the pixel value of pixel G (i, j + 1) is calculated.

  In step S224, the detection unit 251 determines whether or not the absolute value of the difference between the pixel value of the pixel G (i, j) and the pixel value of the pixel G (i, j + 1) is smaller than a predetermined threshold value. Determine.

  If it is determined in step S224 that the absolute value of the difference is smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i, j + 1) are assumed to be pixels of the same object. In step S225, the detection unit 251 sets the object number of the pixel G (i, j + 1) as Obj (i, j + 1) = num.

  In step S226, the detection unit 251 increments the variable j held in the variable holding unit 252.

  In step S227, the detection unit 251 performs difference calculation processing. Note that the difference calculation processing in step S227 is the same as the difference calculation processing described with reference to FIGS.

  In step S228, the detection unit 251 decrements the variable j held in the variable holding unit 252, and the process proceeds to step S156 in FIG.

  If it is determined in step S224 that the absolute value of the difference is not smaller than a predetermined threshold value, the pixel G (i, j) and the pixel G (i, j + 1) are not pixels of the same object. Then, the process from step S225 to step S228 is skipped, and the process proceeds to step S156 in FIG.

  Furthermore, if it is determined in step S222 that Obj (i, j + 1) = 0 is not satisfied, the pixel G (i, j + 1) has already been classified as an object, so the processing from step S223 to step S228 is skipped. Then, the process proceeds to step S156 in FIG.

  Note that, in the process of step S201, the process of step S208, the process of step S215, or the process of step S222, the pixel adjacent to the pixel G (i, j) is not a pixel in a region that is a target for detecting an object. , It is determined that the object number of the pixel adjacent to the pixel G (i, j) is not zero.

  In this way, the detection unit 251 is adjacent to the pixel G (i, j) and the pixel value of the pixel G (i, j) specified by the variable i and variable j held by the variable holding unit 252. The difference between the pixel value and the pixel value is calculated. Then, when the absolute value of the calculated difference is smaller than a predetermined threshold, the detection unit 251 determines that the pixel adjacent to the pixel G (i, j) is the same object as the pixel G (i, j). As the classification, the same number as the object number Obj (i, j) of the pixel G (i, j) is given to the object number of the pixel.

  In this way, an object is detected from the entire image by calculating the difference between the pixel value of a predetermined pixel and the pixel value of a pixel adjacent to that pixel, and assigning an object number to the pixel based on the result. be able to. As a result, the area including the detected object can be a candidate for the zoom area, and the user does not need to specify the zoom area one by one, so the details of the entire image can be observed with a simpler operation, You can enjoy viewing images.

  By the way, in the scalable television system 11 (FIG. 1), each of the nine display devices 21 constituting the scalable television system 11 has a predetermined area (or an area determined by the user) on the entire image. From the above description, it has been described that a skin color region is detected and a region that is a candidate for a zoom region is determined. However, for each display device 21, a region that is a target for detecting a skin color region is dynamically determined. It is also possible.

  As described with reference to FIG. 4, the display devices 21 connected to each other are ordered in advance, and each display device 21 (the central processing unit 93) has an ID (self- ID to be specified) is stored. Therefore, by dynamically determining a region to be detected as a skin color region based on the order of the display devices 21, each display device 21 can detect a skin color region from a different region. It becomes like this.

  For example, if the image (entire image) based on the image data supplied to the display device 21 is the image 411 shown in FIG. 20A, the plurality of display devices 21 use the entire image 411 as a target for detecting a skin color region. From the image 411, a skin color region is detected.

  Then, as shown in FIG. 20B, when the skin color area 421 is detected from the image 411 and is selected as a zoom area candidate, the display device 21 in which the order indicated by the stored ID is first is A zoom image is displayed using the area 421 as a zoom area.

  On the other hand, the display device 21 in which the order indicated by the stored ID is not the first (second and subsequent display devices 21) is an area obtained by removing the area 421 from the image 411 as shown in FIG. 20C. Is a target for detecting a skin color area, and the skin color area is detected again.

  As a result, as shown in FIG. 20D, when the skin-colored area 422 is detected from the area obtained by excluding the area 421 from the image 411 and is determined as a zoom area candidate, the order indicated by the stored ID is 2 The second display device 21 displays a zoom image using the region 422 as a zoom region.

  Further, the display device 21 (the third and subsequent display devices 21) whose order indicated by the stored ID is not the first and second, as shown in FIG. 20E, displays the region 421 and the region 422 from the image 411. The excluded area is a target for detecting a skin color area, and a skin color area is further detected.

  As described above, the display device 21 newly detects a skin color region from a region in which the skin color has already been detected and the region that is a candidate for the zoom region has been removed from the region to be detected as the skin color region. The process of detecting the skin-colored area is repeated until the target area is in the order of itself.

  Next, a process performed by the display device 21 when dynamically determining a target area for detecting a skin color area will be described. Among the display devices 21 constituting the scalable television system 11, the display device 21 that displays a zoom image performs the zoom image display process described with reference to FIG. Then, in step S32 of FIG. 6, the display device 21 dynamically determines a target region for detecting a skin color region, and displays a zoom image.

  Hereinafter, the automatic zoom process corresponding to the process of step S32 of FIG. 6 performed by the display device 21 when dynamically determining an area for detecting the skin color area will be described with reference to the flowchart of FIG. .

  In step S261, the central processing unit 93 determines its own order with reference to the stored ID. For example, when the order indicated by the stored ID is the second, the central processing unit 93 sets its own order to the second.

  In step S <b> 262, the image processing unit 95 performs a skin color area detection process under the control of the central processing unit 93. The skin color area detection process in step S262 is the same as the skin color area detection process described with reference to the flowchart of FIG. Further, in the skin color area detection process, the image processing unit 95 detects a skin color area using the predetermined area or the area set in step S264 as a detection target area, and is a candidate for the zoom area. Define the area to be.

  When the skin color region is detected, in step S263, the central processing unit 93 determines whether or not it is the turn of its own. For example, when the order determined in step S261 is second, when the skin color area is detected twice from the entire image (when the skin color area detection process is performed twice), the central processing unit 93 Judged to be in order.

  If it is determined in step S263 that it is not the order of the player, the skin color area is continuously detected. Therefore, the process proceeds to step S264, and the image processing unit 95 determines areas other than the areas that have been candidates for the zoom area so far. The skin color area is set as a detection target area, and the process returns to step S262 to detect the skin color area from the set area.

  For example, as shown in FIG. 20D, a skin color region 421 (a region that is a candidate for a zoom region) and a region 422 (a region that is a candidate for a zoom region) are detected from an image (entire image) 411, and in step S263, If it is determined that the order is not the order, in step S264, the image processing unit 95 determines that the area obtained by removing the area 421 and the area 422 from the image 411 is the next skin color area, as illustrated in FIG. Is set as an area to be detected.

  On the other hand, if it is determined in step S263 that it is the turn of the player, the process proceeds to step S265. Note that the processing in step S265 and the processing in step S266 are the same as the processing in step S65 and the processing in step S66 in FIG. If an image (zoom image) is displayed in step S266, the process proceeds to step S33 in FIG.

  In FIG. 7, the detected skin color region is set as a candidate for the zoom region, and the center of gravity of the center coordinates and the average value of the zoom magnification are obtained to determine the zoom region. However, the zoom region is described with reference to FIG. 21. In the automatic zoom process, a region that is a candidate for the zoom region determined in step S262, that is, a region including the detected skin color region is directly used as a zoom region, and the region is enlarged and displayed. As in the case of FIG. 7, it is possible to determine the zoom area by determining the area including the detected skin color area as a candidate for the zoom area and obtaining the center of gravity of the center coordinates and the average value of the zoom magnification. Is possible.

  In this manner, the display device 21 dynamically determines a region to be detected as a skin color region, and displays a zoom image using a region including the skin color region detected from the region as a zoom region.

  In this way, for each display device 21, a region for detecting a skin color region is dynamically determined, so that a skin color region from a different region is displayed for each display device 21 connected to each other. As a result, it is possible to display different zoom images for each display device 21 more effectively. Therefore, the user can selectively view the zoom image he / she wants to see from the zoom images displayed on each of the plurality of display devices 21.

  Although it has been described that the zoom region is determined by detecting the skin color region in step S262, the display device is configured as described with reference to FIG. 12, and in step S262, the description is made with reference to FIG. The detected object area may be detected. In this case, for each display device 211, an area for detecting an object is dynamically determined, and an area including the object detected from the area is set as a zoom area, and a zoom image is displayed.

  Further, the function configuration of the display device is configured to have the function described with reference to FIG. 4 and the function described with reference to FIG. 12, and in step S262, a skin color area detection process (FIG. 9); The object area detection process (FIG. 13) may be alternately performed. In this case, for example, when the zoom area is defined for the first time, a skin color area detection process is performed in step S262, and when the zoom area is determined next (when the process of step S262 is performed for the second time), the step is performed. In S262, an object area detection process is performed.

  Furthermore, in step S262, the user can set whether to perform the skin color area detection process or the object area detection process, or the display apparatus can perform the skin color area detection process or the object area detection process. The detection process may be performed differently.

  Furthermore, the configuration of the function of the display device is the configuration having the function described with reference to FIG. 4 and the function described with reference to FIG. 12, and among the plurality of display devices constituting the scalable television system. The number of display devices that display the zoom image by performing the skin color region detection process (FIG. 9) and the number of display devices that display the zoom image by performing the object region detection process (FIG. 13) are dynamically determined. It may be changed.

  For example, as shown in FIG. 22A, when the skin color region is only the face portion of the person 471 and the person 472 on the entire image 461, the two display devices are the face parts of the person 471 and the person 472, respectively. Is displayed as a zoomed image, the skin color region on the image 461 is all enlarged and displayed.

  On the other hand, as shown in FIG. 22B, when the skin-colored area is the face portion of the person 491 to the person 497 on the entire image 481, the face part of the person 491 to the person 497 is respectively replaced by one unit. When the display device displays, seven display devices that detect and display the skin color area are required.

  As described above, the number of display devices required to display the skin color region differs depending on whether the skin color region is large or the skin color region is small on the entire screen. Therefore, zooming is performed by dynamically changing the number of display devices that detect and display the skin color area and the number of display devices that detect and display the object according to the area of the skin color region on the entire screen. Images can be displayed more effectively.

  For example, as shown in FIG. 22A, the skin color area is only the face portion of the person 471 and the person 472 on the whole image 461, and the area of the skin color area on the whole image 461 is larger than a predetermined value. If it is small, there are four display devices that perform skin color region detection processing and four display devices that perform object region detection processing. As shown in FIG. 22B, the skin color region is displayed on the entire image 481. If the area of the skin color area on the entire image 481 is the face portion of the person 491 to 497 and is equal to or greater than a predetermined value, seven display devices that perform the skin color area detection process and the object area One display device that performs detection processing can be provided.

  When the number of display devices that perform skin color region detection processing and the number of display devices that perform object region detection processing are dynamically changed depending on the area of the skin color region, the display device is, for example, illustrated in FIG. Configured as follows.

  The display device 531 is configured to include a control information communication unit 91, an input unit 92, a central processing unit 93, a signal acquisition unit 94, a display unit 96, an audio output unit 97, and an image processing unit 541. Note that the control information communication unit 91, the input unit 92, the central processing unit 93, the signal acquisition unit 94, the display unit 96, and the audio output unit 97 of the display device 531 correspond to the case in FIG. The description is omitted.

  Under the control of the central processing unit 93, the image processing unit 541 detects a skin color region or object from an image (overall image) based on the image data supplied from the signal acquisition unit 94, and detects the detected skin color. Image data for enlarging and displaying the region or the region including the object is generated and supplied to the display unit 96. The image processing unit 541 is configured to include an area determination unit 111, an object selection unit 241, a parameter determination unit 112, and a zoom processing unit 113.

  Note that the region determination unit 111, the parameter determination unit 112, and the zoom processing unit 113 are denoted by the same reference numerals in order to correspond to the case in FIG. Further, the object selection unit 241 is assigned the same reference numeral to correspond to the case in FIG. 12, and the description thereof is omitted.

  In the display device 531 shown in FIG. 23, the image data extracted by the signal acquisition unit 94 is supplied to the region determination unit 111, the object selection unit 241, and the zoom processing unit 113. The parameter determination unit 112 is connected to the region determination unit 111 and the object selection unit 241.

  When the user operates the remote commander to instruct the display device 531 to display an image, the display device 531 performs the zoom image display processing described with reference to FIG. In step S32 of FIG. 6, the display device 531 dynamically determines whether the area including the skin color area is set as the zoom area or the area including the object is set as the zoom area, and displays the zoom image. .

  Hereinafter, referring to the flowchart of FIG. 24, the display device 531 performs when dynamically determining whether the region including the skin color region is the zoom region or the region including the object is the zoom region. The automatic zoom process corresponding to the process in step S32 will be described.

  In step S <b> 301, the skin color region detection unit 121 calculates the area of the skin color region on the image (entire image) based on the image data supplied from the signal acquisition unit 94 based on the control of the central processing unit 93. For example, as described with reference to the flowchart of FIG. 9, the skin color area detection unit 121 appropriately uses the above formulas (10) to (14) based on the pixel value of each pixel, in the HSV format. The value of H in the color space is obtained, and the area of the skin color area on the entire image is calculated.

  In step S302, the central processing unit 93 determines whether or not the area of the skin color area on the entire image calculated by the skin color area detection unit 121 is equal to or larger than a predetermined value.

  If it is determined in step S302 that the value is equal to or greater than the predetermined value, the process advances to step S303, and the central processing unit 93 selects a skin color region from among the eight display devices 531 that are connected to each other and display a zoom image. It is assumed that there are five display devices 531 serving as zoom areas and three display devices 531 including an object-containing area as zoom regions.

  On the other hand, if it is determined in step S302 that the value is not equal to or greater than the predetermined value, the process proceeds to step S304, and the central processing unit 93 includes, among the eight display devices 531 that display the zoom images connected to each other. Three display devices 531 having a skin color region as a zoom region and three display devices 531 having a region including an object as a zoom region are provided.

  In step S303 or step S304, when the number of display devices 531 whose skin color area is a zoom area and the number of display devices 531 whose area includes an object are determined, the process proceeds to step S305, and the central processing unit 93 Referring to the stored ID, determine the order of itself.

  As in the case of the display device 21, the display devices 531 connected to each other are ordered in advance, and each display device 531 (the central processing unit 93) stores an ID indicating its own order. . For example, when the order indicated by the stored ID is the second, the central processing unit 93 sets its own order to the second.

  In step S306, the central processing unit 93 determines whether or not the skin color region is set as the zoom region. For example, each of the eight display devices 531 that display a zoom image stores an ID that indicates one of the first to eighth orders. In step S303, when there are five display devices 531 having the skin-colored region as the zoom region and three display devices 531 having the zoom region as the region including the object, the order is Nos. 1 to 5. The display device 531 determines that the skin color region is the zoom region, and the display device 531 whose order is Nos. 6 to 8 determines that the skin color region is not the zoom region.

  Further, in step S304, if there are three display devices 531 having the skin-colored region as the zoom region and five display devices 531 having the region including the object as the zoom region, the order is Nos. 1 to 3. The display device 531 determines that the skin color region is the zoom region, and the display device 531 whose order is Nos. 4 to 8 determines that the skin color region is not the zoom region.

  If it is determined in step S306 that the skin color area is the zoom area, the central processing unit 93 instructs the image processing unit 541 to execute the skin color area detection process, and the process proceeds to step S307.

  In step S307, the image processing unit 541 (the region determining unit 111 and the parameter determining unit 112) performs a skin color region detection process, and the process proceeds to step S309. The skin color area detection process is the same as the process described with reference to the flowchart of FIG.

  On the other hand, if it is determined in step S306 that the skin color area is not the zoom area, the area including the object is the zoom area, and the central processing unit 93 causes the image processing unit 541 to execute the object area detection process. The process proceeds to step S308.

  In step S308, the image processing unit 541 (the object selection unit 241 and the parameter determination unit 112) performs an object area detection process, and the process proceeds to step S309. The object area detection process is the same as the process described with reference to the flowchart of FIG.

  If the skin color area detection process is performed in step S307 or the object area detection process is performed in step S308, the process proceeds to step S309. Note that the processing in step S309 and the processing in step S310 are the same as the processing in step S65 and the processing in step S66 in FIG. When an image (zoom image) is displayed in step S310, the process proceeds to step S33 in FIG.

  In FIG. 7, the detected skin color region is used as a candidate for the zoom region, and the center of gravity of the center coordinates and the average value of the zoom magnification are obtained to determine the zoom region. However, the zoom region is described with reference to FIG. 24. In the automatic zoom process, a region that is a candidate for the zoom region determined in step S307 or step S308 is directly used as a zoom region, and the region is enlarged and displayed. As in the case of FIG. 7, a region including a detected skin color region or a region including an object is set as a candidate for a zoom region, and the center of gravity of the center coordinates and the average value of the zoom magnification are obtained to determine the zoom region. Of course, it is also possible to do so.

  In this way, the display device 531 dynamically determines whether the area including the skin color area is set as the zoom area or the area including the object is set as the zoom area, and displays the zoom image.

  As described above, each display device 531 is connected to each other by dynamically determining whether the area including the skin color area is set as the zoom area or the area including the object is set as the zoom area. Among the display devices 531, the number of display devices 531 necessary for displaying a zoom image using the skin-colored region as a zoom region can be determined dynamically, and the zoom image can be displayed more effectively. .

  Although it has been described that the number of display devices 531 having the skin color region as a zoom region is determined based on the determination whether the area of the skin color region is equal to or greater than a predetermined value, Depending on the area, the number of display devices 531 having a skin-colored region as a zoom region may be continuously changed from 1 to 7 units.

  By the way, in a questionnaire conducted by the present applicant, there were many people who responded as a conspicuous region (for example, a region including a conspicuous object) as a region to be enlarged and displayed on the entire image. The result is obtained.

  Here, the conspicuous area on the entire image is an area in which the color, brightness, vividness of the area, the amount of edges in the area, etc. are significantly different from other areas on the entire image, that is, the entire area. On the image, it can be said that other predetermined areas are concentrated and exist in other areas.

  Therefore, if it is possible to detect a region that differs greatly from other regions in color, brightness, vividness, and the amount of edges in that region, the conspicuous region on the entire image is enlarged and displayed as a zoom image. can do.

  For example, as shown in FIG. 25A, a yellow star graphic 591 is shown (displayed) in the center of the overall image 581, and the area other than the star graphic 591 in the overall image 581 is white. Suppose there is. At this time, since the color of the star figure 591 (yellow) is different from the color of the other area (white), the star figure 591 (the area) is compared with other areas of the entire image 581. In other words, it can be called a characteristic region. That is, it can be said that the star figure 591 (region) is a conspicuous region.

  In order to detect the area of the star graphic 591 from the entire image 581, for example, as shown in FIG. 25B, a central area 601 in the figure of the entire image 581, and an area obtained by removing the area 601 from the entire image 581 Compare The region 601 includes a part of the star graphic 591, and the region obtained by removing the region 601 from the entire image 581 includes a part of the star graphic 591. The yellow region and the edge are detected from the region 601 and the region obtained by removing the region 601 from the entire image 581, and the region 601 does not become a characteristic region in the entire image 581.

  Further, as shown in FIG. 25C, a case is considered in which the upper left region 602 in the entire image 581 is compared with the region obtained by removing the region 602 from the entire image 581. The region 602 includes a part of the star graphic 591, and the region obtained by removing the region 602 from the entire image 581 includes a part of the star graphic 591. Accordingly, in this case as well, as in the case of FIG. 25B, yellow areas and edges are detected from the area 602 and the area obtained by removing the area 602 from the entire image 581, and the area 602 is a characteristic area in the entire image 581. It will not be.

  On the other hand, as shown in FIG. 25D, consider a case where a central area 603 in the figure of the entire image 581 is compared with an area obtained by removing the area 603 from the entire image 581. The region 603 includes a star graphic 591, and the region obtained by removing the region 603 from the entire image 581 does not include the star graphic 591. Accordingly, a yellow region and an edge are detected from the region 603, but a yellow region and an edge are not detected from the region obtained by removing the region 603 from the entire image 581, so that the region 603 is a feature of the entire image 581. It can be a certain area.

  In this way, a conspicuous area (characteristic area) on the entire image is detected by comparing a part of the area on the entire image with an area on the entire image excluding the area. Can be displayed.

  When detecting a conspicuous region on the entire image and displaying it as a zoom image, the display device constituting the scalable television system is configured as shown in FIG. 26, for example.

  The display device 631 is configured to include a control information communication unit 91, an input unit 92, a central processing unit 93, a signal acquisition unit 94, a display unit 96, an audio output unit 97, and an image processing unit 641. Note that each of the control information communication unit 91, the input unit 92, the central processing unit 93, the signal acquisition unit 94, the display unit 96, and the audio output unit 97 of the display device 631 corresponds to the case in FIG. The description is omitted.

  The image processing unit 641 extracts a conspicuous region from an image (overall image) based on the image data supplied from the signal acquisition unit 94 under the control of the central processing unit 93, and enlarges the region including the region. The image data to be displayed is generated and supplied to the display unit 96. The image processing unit 641 is configured to include an extraction unit 651, a parameter determination unit 112, and a zoom processing unit 113. It should be noted that the parameter determination unit 112 and the zoom processing unit 113 are assigned the same reference numerals in order to correspond to the case in FIG.

  The extraction unit 651 extracts a conspicuous region from an image (entire image) based on the image data supplied from the signal acquisition unit 94. The extraction unit 651 supplies information indicating the extracted conspicuous area to the parameter determination unit 112. The extraction unit 651 includes a generation unit 661, a calculation unit 662, and a comparison unit 663, and the generation unit 661, the calculation unit 662, and the comparison unit 663 can exchange data with each other. .

  The generation unit 661 of the extraction unit 651 generates image data of an image of a partial region of the entire image (hereinafter also referred to as a partial image) based on the image data supplied from the signal acquisition unit 94.

  Based on the image data supplied from the signal acquisition unit 94, the calculation unit 662 of the extraction unit 651 calculates a feature amount indicating the characteristics of an image of an area (hereinafter, also referred to as a comparative image) from which the partial image is removed from the entire image. calculate. Further, the calculation unit 662 of the extraction unit 651 calculates a feature amount indicating the feature of the partial image based on the image data generated by the generation unit 661.

  Here, the feature amount refers to a value indicating the feature of the image, such as the color, brightness, and vividness of the image, and the ratio of pixels constituting the edge in the image.

  The comparison unit 663 of the extraction unit 651 compares the partial image and the comparison image based on the feature amount calculated by the calculation unit 662, and detects a conspicuous region (a region having a feature) from the entire image.

  When the user operates the remote commander to instruct the display device 631 to display an image, the display device 631 performs the zoom image display processing described with reference to FIG. Then, the display device 631 performs an automatic zoom process (FIG. 7) corresponding to the process of step S32, and selects a conspicuous area as a candidate for a zoom area in the process of step S61 of FIG. Hereinafter, with reference to the flowchart of FIG.

  In step S341, the generation unit 661 generates image data of the partial image based on the image data supplied from the signal acquisition unit 94. Here, the area to be a partial image may be predetermined for each display device 631 or may be set by the user for each display device 631.

  For example, as shown in FIG. 28, each of predetermined regions 702-1 to 702-8 on the entire image 701 is a partial image. The eight display devices 631 constituting the scalable television system may use different regions among the regions 702-1 to 702-8 as partial images, and one display device 631 may include the region 702. -1 to 702-8 may be partial images, and a conspicuous region may be detected from each of the regions 702-1 to 702-8.

  When the generation unit 661 generates image data of the partial image, in step 342, the calculation unit 662 calculates the partial image and the comparison based on the image data supplied from the signal acquisition unit 94 and the image data generated by the generation unit 661. A value indicating the feature of each pixel of the image is calculated.

  Here, the values indicating the characteristics of each pixel are, for example, the H value of the HSV format color space indicating the color (hue) of each pixel, and the HSV format color space indicating the vividness (saturation) of each pixel. S (Saturation value) value, V (Value) value of color space in HSV format indicating the brightness (brightness) of each pixel, the pixel value of each pixel, and the pixel value of the pixel adjacent to that pixel It can be a difference.

  For example, when the region 702-1 illustrated in FIG. 28 is a partial image, and the value of H is calculated as a value indicating the feature of each pixel, the calculation unit 662 displays the pixel value of each pixel in the region 702-1. Based on the above, the value of H of each pixel is calculated using the above-described equations (10) to (14) (see the flowchart of FIG. 9) as appropriate. Further, the calculation unit 662 uses the image of the region composed of the regions 702-2 to 702-8 as a comparison image, and based on the pixel value of each pixel of the comparison image, the above equations (10) to (14) are obtained. The value of H for each pixel is calculated as appropriate.

  For example, when the difference between the pixel value of a pixel and the pixel value of a pixel adjacent to the pixel is calculated as a value indicating the feature of each pixel, the calculation unit 662 displays the pixel value of each pixel of the partial image. Is used to calculate the luminance value of each pixel, and the difference between the luminance value of each pixel and the luminance value of one pixel adjacent to the pixel is calculated, and this is used as a value indicating the characteristic of each pixel. Similarly, the calculation unit 662 calculates the luminance value of each pixel based on the pixel value of each pixel of the comparison image, and calculates the difference between the luminance value of each pixel and the luminance value of one pixel adjacent to that pixel. A value indicating the feature of each pixel is calculated.

  Further, the calculation unit 662 calculates the average value of the difference between the luminance value of the predetermined pixel and the luminance values of the four pixels adjacent to the pixel, and the calculated average value indicates the feature of the pixel. It may be a value.

  When the value indicating the feature of each pixel is calculated, in step S343, the calculation unit 662 based on the value indicating the feature of each pixel, the feature amount indicating the feature of the partial image and the feature indicating the feature of the comparison image. Calculate the amount.

  For example, when the value of H is calculated as a value indicating the feature of each pixel, the calculation unit 662 generates the histogram shown in FIG. 29 based on the value of H of each pixel of the partial image. In FIG. 29, the horizontal direction indicates the value of H, and the vertical direction indicates the number of pixels in the partial image whose H value is within a predetermined range.

  For example, among the pixels of the partial image, there are 1400 pixels having an H value of 0 or more and less than 30, 500 pixels having an H value of 30 or more and less than 60, and an H value of 60 or more and 90 The number of pixels that are less than 600 is 600, and the number of pixels whose H value is 90 or more and less than 120 is 500.

  Further, among the pixels of the partial image, there are 600 pixels having an H value of 120 or more and less than 150, 700 pixels having an H value of 150 or more and less than 180, and an H value of 180 or more and 210. The number of pixels that are less than 800 is 800, and the number of pixels whose H value is 210 or more and less than 240 is 700. Furthermore, 600 pixels have an H value of 240 or more and less than 270, 500 pixels have an H value of 270 or more and less than 300, and 600 pixels have an H value of 300 or more and less than 330. And the number of pixels having an H value of 330 or more and less than 360 is 800.

  In the example of FIG. 29, it can be seen that the number of pixels having an H value of 0 or more and less than 30 is larger than the pixels in other ranges (for example, pixels having an H value of 30 or more and less than 60). The calculation unit 662 selects the H value range having the largest number of pixels from each H value range, and calculates the feature amount of the partial image based on the number of pixels in the H value range. calculate.

  In the example of FIG. 29, since the number of pixels having an H value of 0 or more and less than 30 is the largest in the range of each H value, the calculation unit 662 has an H value of 0 or more and less than 30. A feature amount of the partial image is calculated based on the number of pixels. For example, the calculation unit 662 calculates, as a feature amount, a value obtained by dividing the number of pixels having an H value of 0 or more and less than 30 by the number of pixels of the partial image. In this case, the feature amount of the partial image indicates a ratio of the number of pixels in which the value of H is 0 or more and less than 30 to the number of pixels of the partial image.

  As in the case of the partial image, the calculation unit 662 generates a histogram similar to the histogram shown in FIG. 29 based on the H value of each pixel of the comparison image. Then, the calculation unit 662 selects the H value range having the largest number of pixels in the histogram of the partial image from among the H value ranges, and the number of pixels in the selected H value range ( The feature amount of the comparison image is calculated based on the number of pixels in the comparison image.

  In the example of FIG. 29, since the number of pixels having the H value of 0 or more and less than 30 is the largest in the range of H values of the pixels of the partial image, the calculation unit 662 displays the pixel values of the comparison image. The feature amount of the comparative image is calculated based on the number of pixels in which the value of H is 0 or more and less than 30. For example, the calculation unit 662 calculates, as a feature amount, a value obtained by dividing the number of pixels having a H value of 0 or more and less than 30 among the pixels of the comparison image by the number of pixels of the comparison image.

  A histogram is generated based on the H value of each pixel, and the H value range having the largest number of pixels is selected from the respective H value ranges, and pixels in the H value range are selected. It has been described that the feature amount of the partial image is calculated based on the number of pixels. However, based on the H value of each pixel, a distribution diagram indicating the number of pixels having each H value is generated to calculate the feature amount of the partial image. You may make it calculate.

  In this case, for example, a predetermined H value range centering on the H value having the largest number of pixels is determined. Then, a value obtained by dividing the number of pixels in the determined H value range by the number of pixels of the partial image is calculated as a feature amount. Further, as in the case of the partial image, for the comparative image, a distribution diagram indicating the number of pixels having each H value is generated and is determined when calculating the feature amount of the partial image among the pixels of the comparative image. A value obtained by dividing the number of pixels in the H value range by the number of pixels in the comparison image is calculated as the feature amount.

  In addition, even when the S value or V value of each pixel is calculated as a value indicating the feature of each pixel, the H value of each pixel is calculated as a value indicating the feature of each pixel. In the same manner as above, a histogram is generated, and a range of S values (or V values) having the largest number of pixels is selected from a range of each S value (or V value), and a partial image and The feature amount of the comparative image can be calculated.

  Further, for example, when the difference between the luminance value of each pixel and the luminance value of one pixel adjacent to the pixel is calculated as a value indicating the characteristic of each pixel, the calculation unit 662 displays the pixel value of the partial image. Among them, a value obtained by dividing the number of pixels in which the absolute value of the difference between the pixel values of each pixel is equal to or greater than a predetermined threshold by the number of pixels of the partial image is calculated as a feature amount of the partial image. In this case, the feature amount of the partial image indicates the ratio of the number of pixels constituting the edge (the number of pixels in the edge portion) to the number of pixels in the partial image.

  Similarly, the calculation unit 662 calculates the number of pixels in the comparison image where the absolute value of the difference between the pixel values of each pixel is equal to or greater than a predetermined threshold (the same threshold as in the partial image). Then, a value divided by the number of pixels of the comparison image is calculated as a feature amount of the comparison image.

  Thus, the calculation unit 662 calculates the feature amount of the partial image (or comparison image) based on the number of pixels that satisfy the predetermined condition among the pixels of the partial image (or comparison image).

  Returning to the description of the flowchart of FIG. 27, when the feature amounts of the partial image and the comparison image are calculated, in step S344, the comparison unit 663 calculates the partial image and the comparison image based on the calculated feature amounts of the partial image and the comparison image. Compare with the comparison image.

  For example, the comparison unit 663 obtains the difference between the feature amount of the partial image and the feature amount of the comparison image (subtracts the feature amount of the comparison image from the feature amount of the partial image), and the obtained difference is When the value is equal to or greater than a predetermined value, it is assumed that the partial image includes a characteristic region (a conspicuous region).

  If it is determined that the partial image does not include a conspicuous area, for example, the feature amounts of the partial image and the comparison image are obtained by another method (feature amounts indicating other features) and again. The partial image and the comparative image may be compared, and the new partial image and the comparative image are compared with each other as a new partial image in the entire region that is different from the partial image. May be.

  In step S345, the extraction unit 651 extracts a conspicuous region from the partial image by using, as a conspicuous region, a region including pixels that satisfy a predetermined condition in the partial image. The extraction unit 651 generates information indicating the extracted conspicuous region and supplies the information to the parameter determination unit 112.

  For example, as described with reference to FIG. 29, when the feature amount of the partial image is calculated based on the number of pixels whose H value is 0 or more and less than 30, the extraction unit 651 determines that the value of H is Pixels that are 0 or more and less than 30 are extracted from the partial image, with pixels satisfying a predetermined condition, and regions including pixels with H values of 0 or more and less than 30 being conspicuous.

  Further, for example, the extraction unit 651 may extract a conspicuous region from the partial image, with a region in which pixels satisfying a predetermined condition are contiguous as a conspicuous region. As described above, when a region where pixels satisfying a predetermined condition are continuous is a conspicuous region, a plurality of conspicuous regions may be detected from the partial image. In such a case, the extraction unit 651 selects and extracts a predetermined number of conspicuous regions in ascending order of the area of the conspicuous region, or a predetermined number of conspicuous in the order that the position of the conspicuous region is closer to the center of the entire image. Select and extract areas.

  Further, the extraction unit 651 may select a predetermined color region (a conspicuous region) from a plurality of conspicuous regions. In this case, for example, a human image can be displayed as a zoom image by selecting a skin color region from a plurality of conspicuous regions.

  When the conspicuous region is extracted from the partial image, in step S346, the parameter determination unit 112 includes the conspicuous region indicated by the information based on the information indicating the extracted conspicuous region supplied from the extraction unit 651. The area is a candidate for the zoom area, and the center coordinates of the area are calculated.

  For example, the parameter determination unit 112 calculates the coordinates of the center of gravity in the XY space of the pixels satisfying a predetermined condition that form a conspicuous region as the center coordinates. Further, for example, when a plurality of conspicuous regions are extracted as conspicuous regions to be displayed as a zoom image, the parameter determination unit 112 sets a region including a plurality of conspicuous regions as a candidate for a zoom region, and The center coordinate in the XY space is calculated as the center coordinate.

  In step S347, the parameter determination unit 112 calculates the zoom magnification of the region that is a candidate for the zoom region, and the process proceeds to step S62 in FIG. For example, the parameter determination unit 112 calculates the zoom magnification Zi by calculating Equation (17) described above.

  In this manner, the image processing unit 641 extracts (detects) a conspicuous region from the entire image, and sets a region including the extracted conspicuous region as a candidate for a zoom region. As a result, an image in which the extracted conspicuous area is enlarged is displayed on the display unit 96 as a zoom image.

  For example, as shown in FIG. 30A, the red sun 741 is shown (displayed) in the upper left in the entire image 731, and the red area in the entire image 731 is only the area of the sun 741. . As shown in FIG. 30B, the display device 631 sets a region 751 including the sun 741 in the entire image 731 (FIG. 30A) as a partial image, and the number of red pixels relative to the number of pixels in the region 751 (partial image 751). When the feature amount indicating the ratio of the number is calculated and a conspicuous region is detected, since there is no red region in the comparison image (the region obtained by removing the region 751 from the entire image 731), the sun 741 is detected as a conspicuous region. .

  Then, the display device 631 extracts a region 752 including the sun 741 from the partial image 751, sets the region 752 as a candidate for the zoom region, and displays a zoom image 761 including the sun 741 as illustrated in FIG. 30C. To do.

  In addition, as shown in FIG. 31A, the partial image 801 includes the sun 802, the human face 803, the human torso 804, and the flying object 805 (displayed), and an area in which the display device 631 is conspicuous is detected. As a result, it is assumed that a region 806 including the sun 802, a region 807 including a human face 803, a region 808 including a human torso 804, and a region 809 including a flying object 805 are detected as prominent regions.

  As described above, when a plurality of conspicuous regions are detected from the partial image 801, the display device 631 sets a region including all the conspicuous regions as a candidate for a zoom region, Among the conspicuous regions, a region including one conspicuous region can be set as a candidate region for the zoom region.

  For example, if a region including regions 806 to 809 detected as conspicuous regions from the partial image 801 is a region as a candidate for a zoom region, the display device 631 is detected as a conspicuous region as illustrated in FIG. 31B. The zoom image 811 including the sun 802, the human face 803, the human torso 804, and the flying object 805 included in the region is displayed.

  Further, for example, if the region including the region 809 among the regions 806 to 809 detected as the conspicuous regions illustrated in FIG. 31A is a region that is a candidate for the zoom region, the display device 631 is illustrated in FIG. 31C. As described above, the zoom image 812 including the flying object 805 included in the region 809 detected as the conspicuous region is displayed.

  As described above, the display device 631 (the image processing unit 641) compares the partial image and the comparative image, extracts a conspicuous region from the entire image, and selects the region including the extracted conspicuous region as a candidate for a zoom region. Therefore, a conspicuous area on the entire image can be enlarged and displayed more easily. This eliminates the need for the user to designate the zoom area one by one, and allows the user to view the details of the entire image and enjoy viewing the image with a simpler operation.

  In addition, since the user sees the conspicuous area in an enlarged manner, the user does not need to stare at the area of interest (the conspicuous area) on the entire image, and the user's stress when viewing the entire image is reduced. The

  In the above description, it has been described that the display device configuring the scalable television system defines the zoom region and displays the zoom image. However, the user can specify the zoom region and display the zoom image on the display device. is there.

  The zoom area is determined by, for example, the user operating the remote commander to display a frame indicating the zoom area on the display unit 96 and changing the size of the frame or moving the position thereof. Can do. Further, a zoom image may be displayed based on a zoom area designated by the user, and the user may be able to adjust the position and size of the zoom area while viewing the zoom image actually displayed.

  When the user designates a zoom area to display a zoom image, the display device 21 (FIG. 4) constituting the scalable television system 11 performs the zoom image display process described with reference to FIG. When the zoom area is designated by the user, the display device 21 displays a zoom image using the designated area as the zoom area in step S32.

  Hereinafter, with reference to the flowchart of FIG. 32, the automatic zoom process when the zoom area is designated by the user will be described.

  In step S381, the central processing unit 93 (FIG. 4) determines a zoom area. For example, when the user operates the remote commander to specify the zoom area, an infrared signal corresponding to the user operation is transmitted from the remote commander. The receiving unit 73 (FIG. 3) receives (receives) an infrared signal transmitted from the remote commander, performs photoelectric conversion, and supplies a command signal to the central processing unit 93 via the control information communication unit 91.

  The central processing unit 93 determines an area designated by the user as a zoom area based on the command signal supplied from the control information communication unit 91. The central processing unit 93 generates information indicating the zoom area and supplies the information to the image processing unit 95.

  In step S <b> 382, the parameter determination unit 112 calculates the center coordinates of the zoom region indicated by the information indicating the zoom region supplied from the central processing unit 93 based on the control of the central processing unit 93.

  In step S383, the parameter determination unit 112 calculates the zoom magnification Zi of the zoom region by calculating the above-described equation (17). In this case, in Expression (17), Xid is the length (number of pixels) in the X direction of the zoom area in the XY space.

  When the zoom magnification of the zoom area is calculated, the process of step S384 and the process of step S385 are performed thereafter, and the process proceeds to step S33 of FIG. 6, but the process of step S384 and the process of step S385 are illustrated in FIG. 7 is the same as the process in step S65 and the process in step S66, and the description thereof is omitted.

  When the zoom area is designated by the user, the area designated by the user is continuously used as the zoom area until a new zoom area is designated by the user. In this case, since the center coordinates and zoom magnification of the zoom area do not change, when returning from step S33 in FIG. 6 to step S32, the processing from step S381 to step S383 in FIG. 32 is omitted.

  In this way, the display device 21 enlarges the image in the zoom area designated by the user and displays the zoom image.

  As described above, in the present invention, the skin color region is detected from the entire image, and the zoom image is displayed based on the detection result. Therefore, the user can perform more effectively with a simpler operation. An image can be displayed.

  In the present invention, since an object is detected from the entire image and a zoom image is displayed based on the detection result, the user can display the image more effectively with a simpler operation. Can do.

  Furthermore, in the present invention, a conspicuous region is detected from the entire image, and the zoom image is displayed based on the detection result, so that the user can display the image more effectively with a simpler operation. be able to.

  In the scalable television system described above, it has been described that each display device independently performs a process of determining a zoom region and displaying a zoom image, but each display device communicates with each other to perform a zoom region. May be determined. In such a case, the display devices communicate with each other, and for each display device, detect a skin color region and display a zoom image, detect an object and display a zoom image, or display a conspicuous region. Whether to detect and display a zoom image is determined.

  In the above description, an example in which a scalable television system is configured by nine display devices has been described. However, the number of display devices is not limited to nine, for example, four or sixteen. May be.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 33 is a block diagram showing an example of the configuration of a personal computer 901 that executes the above-described series of processing by a program. A CPU (Central Processing Unit) 911 executes various processes according to a program recorded in a ROM (Read Only Memory) 912 or a recording unit 918. A RAM (Random Access Memory) 913 appropriately stores programs executed by the CPU 911 and data. The CPU 911, the ROM 912, and the RAM 913 are connected to each other by a bus 914.

  An input / output interface 915 is also connected to the CPU 911 via a bus 914. The input / output interface 915 is connected to an input unit 916 including a keyboard, a mouse, and a microphone, and an output unit 917 including a display and a speaker. The CPU 911 executes various processes in response to commands input from the input unit 916. Then, the CPU 911 outputs the processing result to the output unit 917.

  The recording unit 918 connected to the input / output interface 915 includes, for example, a hard disk, and records programs executed by the CPU 911 and various data. The communication unit 919 communicates with an external device via a network such as the Internet or a local area network.

  Further, a program may be acquired via the communication unit 919 and recorded in the recording unit 918.

  The drive 920 connected to the input / output interface 915 drives a removable medium 931 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the program or data recorded therein. Get etc. The acquired program and data are transferred to the recording unit 918 and recorded as necessary.

  As shown in FIG. 33, a program recording medium that stores a program that is installed in a computer and is ready to be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory), DVD (including Digital Versatile Disc), magneto-optical disk), or removable media 931 which is a package medium made of semiconductor memory, or ROM 912 in which a program is temporarily or permanently stored, The recording unit 918 is configured by a hard disk or the like. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 919 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the described order, but is not necessarily performed in time series. Or the process performed separately is also included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure which shows the structural example of the external appearance of the scalable television system to which this invention is applied. It is a figure which shows an example of the image displayed on a display apparatus. It is a block diagram which shows the structural example of a scalable television system. It is a block diagram which shows the structural example of the function of a display apparatus. It is a flowchart explaining the display process of a whole image. It is a flowchart explaining the display process of a zoom image. It is a flowchart explaining an automatic zoom process. It is a figure for demonstrating a zoom area | region. It is a flowchart explaining the detection process of a skin color area | region. It is a figure which shows an example of the value of H calculated based on each value of R, G, and B. It is a figure for demonstrating the area | region of a skin color. It is a block diagram which shows the structural example of the other function of a display apparatus. It is a flowchart explaining the detection process of an object area | region. It is a figure which shows an example of the image displayed as a zoom image. It is a figure which shows an example of the image displayed as a zoom image. It is a flowchart explaining an object detection process. It is a figure explaining the area | region used as the object which detects an object. It is a flowchart explaining a difference calculation process. It is a flowchart explaining a difference calculation process. It is a figure explaining the area | region used as the object which detects the area | region of a skin color. It is a flowchart explaining an automatic zoom process. It is a figure which shows an example of a whole image. It is a block diagram which shows the structural example of the other function of a display apparatus. It is a flowchart explaining an automatic zoom process. It is a figure for demonstrating the detection of a conspicuous area | region. It is a block diagram which shows the structural example of the other function of a display apparatus. It is a flowchart explaining the detection process of a conspicuous area | region. It is a figure for demonstrating a partial image. It is a figure which shows an example of the histogram which shows the characteristic of each pixel of a partial image. It is a figure which shows an example of the conspicuous area | region displayed as a zoom image. It is a figure which shows an example of the conspicuous area | region displayed as a zoom image. It is a flowchart explaining an automatic zoom process. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 Scalable television system, 21-1 thru | or 21-9, 21 Display apparatus, 93 Central processing part, 95 Image processing part, 96 Display part, 111 Area | region determination part, 112 Parameter determination part, 113 Zoom processing part, 121 Skin color area | region Detection unit, 122 parameter holding unit, 211 display device, 231 image processing unit, 241 object selection unit, 251 detection unit, 252 variable holding unit, 531 display device, 541 image processing unit, 631 display device, 641 image processing unit, 651 Extraction unit, 661 generation unit, 662 calculation unit, 663 comparison unit, 901 personal computer, 911 CPU, 912 ROM, 913 RAM, 918 recording unit, 931 removable media

Claims (10)

A feature amount indicating a predetermined feature of the first region is calculated based on the number of pixels satisfying a predetermined condition based on a pixel value among the pixels of the first region on the first image, and the first Calculation for calculating a feature amount indicating the feature of the second region based on the number of pixels satisfying the condition among the pixels of the second region obtained by removing the first region from one image Means,
Comparing means for comparing the feature quantity of the first area and the feature quantity of the second area by obtaining a difference between the feature quantity of the first area and the feature quantity of the second area. When,
Result of comparison by the comparing means, the feature amount of the first region, when the difference between the feature quantity of the second region is greater than a predetermined threshold value, among pixels of the first area, the condition Extraction means for extracting, from the first region, a region including pixels that satisfy the condition as a feature region having the feature;
An image processing apparatus comprising: display control means for controlling display of a second image in which the extracted feature region is enlarged. The image processing apparatus according to claim 1, wherein the display control unit controls display of the second image so that the first image and the second image are displayed simultaneously. The image processing apparatus according to claim 1, wherein the calculation unit calculates a feature amount indicating a hue of the first region or the second region. The image processing apparatus according to claim 1, wherein the calculation unit calculates a feature amount indicating brightness of the first region or the second region. The image processing apparatus according to claim 1, wherein the calculation unit calculates a feature amount indicating a saturation of the first region or the second region. The image processing apparatus according to claim 1, wherein the calculating unit calculates a feature amount indicating a ratio of pixels constituting an edge in the first region or the second region. The extraction means detects a region in which pixels satisfying the condition continue in the first region, and extracts a region having the largest area among the detected regions as the feature region. Image processing apparatus. The extraction unit detects a region in which pixels satisfying the condition are continuous on the first region, and extracts a region closest to the center of the first image among the detected regions as the feature region. The image processing apparatus according to claim 1. A feature amount indicating a predetermined feature of the first region is calculated based on the number of pixels satisfying a predetermined condition based on a pixel value among the pixels of the first region on the first image, and the first Based on the number of pixels satisfying the condition among the pixels of the second region obtained by removing the first region from one image, the feature amount indicating the feature of the second region is calculated,
The feature amount of the first region is compared with the feature amount of the second region by obtaining a difference between the feature amount of the first region and the feature amount of the second region,
Result of the comparison, the feature amount of the first region, when the difference between the feature quantity of the second region is greater than a predetermined threshold value, among pixels of the first area, the pixel satisfying Is extracted from the first region as a feature region having the feature,
An image processing method including a step of controlling display of a second image in which the extracted feature region is enlarged. A feature amount indicating a predetermined feature of the first region is calculated based on the number of pixels satisfying a predetermined condition based on a pixel value among the pixels of the first region on the first image, and the first Calculation for calculating a feature amount indicating the feature of the second region based on the number of pixels satisfying the condition among the pixels of the second region obtained by removing the first region from one image Steps,
A comparison step of comparing the feature amount of the first region with the feature amount of the second region by obtaining a difference between the feature amount of the first region and the feature amount of the second region. When,
Result of comparison in the comparison step, the feature amount of the first region, when the difference between the feature quantity of the second region is greater than a predetermined threshold value, among pixels of the first area, the condition An extraction step of extracting, from the first region, a region including pixels that satisfy the condition as a feature region having the feature;
And a display control step for controlling display of the second image in which the extracted feature region is enlarged.

Images