JP5192940B2 - Image conversion apparatus, image conversion program, and image conversion method - Google Patents

Description

  The present invention relates to an image conversion apparatus, an image conversion program, and an image conversion method for performing image conversion so as to selectively exaggerate a specific video object in an image.

  Conventionally, when a high-resolution image is displayed on a low-resolution display or a small display, a method of converting the resolution and display size at a constant magnification so that the entire high-resolution image fits within the display area of the display has been the mainstream.

For example, Patent Document 1 proposes a trimming technique in which an important region of an image is estimated and cut out, and the cut out region is converted so as to be within the display region of the display.
JP 2007-6111 (paragraph 0020 to paragraph 0023, FIG. 1)

  However, in the method of converting the entire high-resolution image so that it fits within the display area of the display, especially when the main subject is photographed small, the visibility of the subject deteriorates due to the reduction of the image. There is a problem that the person is tired and it is difficult to understand the image contents.

  For example, in the case where a high-resolution image taken with a high-definition camera is viewed on a relatively wide area near the ball in a video of a soccer game, when using a high-definition display that is a high-resolution display, Even if the player's face can be easily identified, for example, if the entire image is reduced and displayed for display on a low-resolution small display such as a television for 1-segment broadcasting, the face of each player It was difficult to identify who was playing small.

  The technique of Patent Document 1 is an effective technique when the main subject is unevenly distributed in a part of the image. However, when the main subject is dispersed throughout the image, an effective clipping range is set. Since setting is difficult and a wide range is reduced and displayed, the same problem as when the entire image is reduced and displayed occurs.

  Accordingly, in the present invention, in view of such a problem, an image to be converted into an exaggerated composite image by changing the magnification ratio of the main subject region and the other regions and relatively expanding the main subject. It is an object to provide a conversion device, an image conversion program, and an image conversion method.

In order to achieve the above-described object, the image conversion device according to claim 1 is an image conversion device that converts an input image into an image in which a specific video object in the input image is relatively enlarged, A video object detection unit, a video object selection unit, a partial image cutout unit, an image scaling unit, and an image synthesis unit; and the video object selection unit includes a plurality of decision logic units, a target selection unit, and a structure in which Ru equipped with.

According to such a configuration, the image conversion device detects one or more types of video objects of a predetermined type from the input image by the video object detection unit, and includes a type including the type of the detected video object for each detected video object. Information and position information indicating the detected position are output. Next, based on the type information and position information of the video object detected by the video object detection unit, the video object selection unit selects a video object that meets a predetermined condition from these detected video objects. Select as a specific video object. Subsequently, the partial image cutout means cuts out the partial image including the selected specific video object from the input image, and the image scaling means scales at least one of the input image or the partial image and inputs it. An image pair including an entire image and a partial image in which the partial image is relatively enlarged with respect to the entire area of the image is generated. Then, the whole image and the partial image forming the image pair are synthesized by the image synthesis means. At this time, a plurality of decision logic means of the video object selection means are respectively predetermined from the detected video objects based on the type information and position information of the video objects detected by the video object detection means. Video objects that meet the specified conditions are determined as specific video object candidates. Subsequently, a specific video object is selected from these specific video object candidates by the target selection means of the video object selection means. Then, the image scaling means enlarges the partial image of the selected specific video object relative to the input image or, if the entire image scaling means is provided, to the scaled whole image. A variable-magnification partial image is created by scaling the image so that the resulting image is obtained. Then, an entire image (an input image or a zoomed whole image) forming an image pair and the zoomed partial image are synthesized by the image synthesizing unit.

Thus, the image conversion device selects an input image from video objects that meet a plurality of conditions when converting an input image into an exaggerated image by selectively enlarging an image of a specific video object. The specific video object can be converted into an exaggerated image.

  According to a second aspect of the present invention, in the image conversion device according to the first aspect, the image scaling unit includes an entire image scaling unit.

According to such a configuration, the image conversion apparatus creates a scaled whole image obtained by reducing the entire input image by the whole image scaling unit of the image scaling unit. Then, the image composition unit synthesizes the entire zoom image forming a pair with the partial image of the specific video object cut out by the video object partial image cut-out unit.
Accordingly, the image conversion apparatus can display the input image on an image display apparatus having a resolution lower than the resolution of the input image, and can convert the input image into an image in which a specific video object is exaggerated.

  According to a third aspect of the present invention, in the image conversion device according to the first or second aspect, the image scaling unit includes a partial image scaling unit.

According to such a configuration, the image conversion apparatus changes the partial image of the specific video object with respect to the input image or the entire image scaling unit by the partial image scaling unit of the image scaling unit. A zoomed partial image is created by scaling the image of the entire doubled image to a relatively enlarged image. Then, an entire image (an input image or a zoomed whole image) forming an image pair and the zoomed partial image are synthesized by the image synthesizing unit.
Thus, the image conversion apparatus can convert the input image into an image in which a specific video object is exaggerated. Further, when the entire image scaling unit is provided, the input image can be converted into an image in which a specific video object is exaggerated by conforming to an image display device having a desired resolution.

  The image conversion device according to claim 4 is the image conversion device according to claim 3, wherein the partial image scaling means performs nonlinear scaling, and the image composition means sets the joint between video objects as a reference point. Was configured to synthesize.

According to such a configuration, the image conversion apparatus allows the partial image scaling unit to select another image that is not the specific video object in the input image synthesized by the image synthesizing unit. At the junction with the object, the magnification is the same as that of the other video object, and the magnification partial image is created by nonlinear scaling so that the magnification increases as the distance from the junction increases. Then, the image synthesizing means synthesizes the non-linearly scaled scaled partial image, the whole image forming an image pair, and the scaled partial image with the joint as a reference point.
Thus, the image conversion apparatus can convert the input image into an image in which a specific video object is exaggerated by nonlinear scaling.

Image conversion program according to claim 5, a computer, and a function as an image conversion apparatus according to any one of claims 1 to 4.

7. The image conversion method according to claim 6 , wherein the input image is converted into an image obtained by relatively enlarging a specific video object in the input image, and includes a video object detection step and a video object selection. a step, a video object partial image cutting-out step, viewed including the image scaling step, and an image synthesizing step, the video object selection step, characterized a plurality of decision logic step, the object selecting step, the-containing Mukoto And

According to this method, in the video object detection step, one or more predetermined types of video objects are detected from the input image by the video object detection means, and each detected video object includes the type of the detected video object. Information and position information indicating the detected position are output, and in the video object selection step, based on the type information and position information of the video object detected in the video object detection step, the detected video object is selected from the detected video objects. A video object that meets a predetermined condition is selected as a specific video object by the video object selection means. Subsequently, in the partial image cutout step, a partial image including the selected specific video object is cut out from the input image by the partial image cutout means, and in the image scaling step, at least one of the input image or the partial image is imaged. The image is scaled by the scaling means to generate an image pair composed of the entire image and the partial image in which the partial image is relatively enlarged with respect to the entire area of the input image. Then, in the image composition step, the whole image and the partial image forming the image pair are synthesized by the image composition means. At this time, based on the type information and position information of the video object detected by the video object detection means by the plurality of decision logic steps of the video object selection step, each of the detected video objects is determined in advance. Video objects that meet the specified conditions are determined as specific video object candidates. Subsequently, a specific video object is selected from these specific video object candidates in the target selection step of the video object selection step. Then, by the image scaling step, the partial image of the selected specific video object is relatively enlarged with respect to the input image or, when the entire image scaling step is provided, with respect to the entire scaled image. A variable-magnification partial image is created by scaling the image so that the resulting image is obtained. Then, in the image synthesis step, the entire image (input image or zoomed whole image) forming an image pair and the zoomed partial image are synthesized.
Thus, when the input image is converted into an exaggerated image by selectively enlarging the image of the specific video object, the specific video object selected from the video objects that meet a plurality of conditions Can be converted into an exaggerated image.

According to the present invention, specific portions that meet a plurality of conditions are selectively exaggerated, so that when there are a plurality of particularly notable portions, the portions are exaggerated without omission and an image with excellent visibility is obtained. Can be converted.

Na us, in the appended claims, and the video object is that of the object image in the input image obtained by photographing the subject (object), such as a person or a ball.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. Here, a case where the present invention is applied to an image of a soccer game will be described as an example.

<First Embodiment>
[Configuration of Image Conversion Device]
First, the configuration of the image conversion apparatus 1 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the image conversion apparatus according to the first embodiment of the present invention.

The image conversion apparatus 1 shown in FIG. 1 includes an image scaling unit 100 including an entire image scaling unit 10 and a partial image scaling unit 50, a video object detection unit 20, an exaggeration target video object selection unit 30, The image object area cutout means 40 and the image composition means 60 are included.
The image conversion apparatus 1 receives the input image H supplied from the image input apparatus 2, creates a composite image G, and outputs it to the image display apparatus 3.

Next, the configuration of each unit of the image conversion apparatus 1 will be described.
The image scaling unit 100 includes an entire image scaling unit 10 and a partial image scaling unit 50, and the entire image scaling unit 10 and the partial image scaling unit 50 cooperate with each other. An input image H that is an image and a partial image T of a specific video object are composed of a whole image (magnification whole image L) in which the partial image T is relatively enlarged and a partial image (magnification partial image U). An image pair is generated.

  The overall image scaling unit 10 of the image scaling unit 100 inputs an input image H that is an entire image supplied from the image input device 2, and scales the input image H to suit the image display device 3. Is generated, and the generated entire enlarged image L is output to the image composition means 60.

  As the image input device 2 that supplies the input image H, a photographing device such as a high-vision camera or an image storage device such as a hard disk or an optical disk can be used. The whole image scaling unit 10 inputs an image taken by such a photographing device or an image read from the image storage device as an input image H directly or via a communication line or a broadcast receiving device. To do.

Here, the input image H will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of an input image.
In the input image H, the upper left corner of the input image H is the origin (0, 0), the first element of the two-dimensional coordinate is the horizontal coordinate X (the right direction is positive), the second element is the vertical coordinate Y (the lower direction is It is assumed that each pixel is counted in units of the number of pixels. The same applies to the image coordinates in the scaled whole image L described later.

The input image H shown in FIG. 2 shows a scene of a soccer game. A video object M _{1 of} a person who holds a video object O _{3 of a} ball on the right side of the screen and a video object M of a person on the left side of the screen. ₂ is projected. As will be described later, in this embodiment, when “human face (head)” and “ball” are detected as the types of video objects, the human face video object O ₁ , the human face video, The object O ₂ and the video object O _{3 of the} ball are detected. If the center of gravity is used as the representative point of each video object, the image coordinates are expressed as (f ₁ , g ₁ ), (f ₂ , g ₂ ) and (f ₃ , g ₃ ), respectively. Can do.

Returning to FIG. 1, the description will be continued.
The whole image scaling unit 10 performs a scaling process (a reduction process or an enlargement process) on the entire area of the input image H with a scaling factor suitable for the resolution of the image display device 3 to create a scaled whole image L. Preferably, the scaling process is performed at a scaling factor that matches the resolution of the image display device 3. The created entire zoom image L is output to the image composition means 60.

  In the image scaling process, interpolation is performed using the pixel values of the input image H, and re-sampling is performed to obtain each pixel value constituting the scaled entire image L having a desired resolution. The interpolation method is arbitrary, but for example, nearest neighbor interpolation method (nearest neighbor interpolation method), bilinear interpolation method (bilinear interpolation method) or cubic interpolation method (cubic convolution interpolation method or bicubic interpolation method) is used. be able to.

For example, when a high-definition image (high-definition image) having a size of 1920 pixels in the horizontal direction and 1080 pixels in the vertical direction is converted into a low-resolution image having 320 pixels in the horizontal direction and 180 pixels in the vertical direction, 1/6 each in the vertical and horizontal directions. It may be reduced to the number of pixels. The input image H is set to H (X, Y), and the entire zoom image L is set to L (x, y). Here, X is the horizontal image coordinate in the input image H, Y is the vertical image coordinate in the input image H, x is the horizontal image coordinate in the entire zoom image L, and y is vertical in the zoom entire image L. The image coordinates of the direction. H (X, Y) and L (x, y) are pixel values such as the luminance value and chromaticity value of the input image H at the image coordinates (X, Y), respectively, and the entire zooming at the image coordinates (x, y). Pixel values such as the luminance value and chromaticity value of the image L are shown.

  When the image size of the input image H is 1920 pixels wide and 1080 pixels high, Xε {0, 1,..., 1919} and Yε {0, 1,. Further, in the case where the image size of the entire zoom image L is a low resolution image having horizontal 320 pixels and vertical 180 pixels, x∈ {0, 1,..., 319}, y∈ {0, 1,. It is.

  Here, in order to obtain a scaled whole image L obtained by scaling the input image H to 1/6, the whole image scaling unit 10 converts the input image H into the vertical 6 as shown in the equation (1), for example. By dividing the block into pixels each having a size of 6 pixels in the horizontal direction, and using the average pixel value in the block as the pixel value of one pixel in the entire magnification-changed image L, scaling processing (reduction of 1/6 in the vertical and horizontal directions) is performed. Processing).

  The video object detection means 20 receives an input image H that is an entire image supplied from the image input device 2, detects a predetermined type of video object from the input image H, and detects the detected video object. Type information Q and position information P are output to the exaggeration target video object selecting means 30.

  Here, the type information Q is an identifier that identifies the type, individual, and individual of the video object. The type information Q and the position information P are output by the number (K) of video objects detected from the input image H.

  Further, an upper limit may be set for the number of type information Q and position information P to be output. When the applied video object detection method can calculate the reliability of whether or not the video object has been correctly detected for each detected video object, for example, the method of Viola described later, The type information Q and the position information P related to video objects up to a predetermined upper limit number may be output to the exaggeration target video object selection means 30 in preference to those having a high reliability.

Here, if the type information regarding the k-th video object included in the type information Q output from the video object detection means 20 is Q _k , the type information Q can be expressed as in Expression (2).

The type information Q _k is, for example, an identifier for identifying an individual or an individual such as “person's face”, “person's leg”, “ball”, or the like, or a person's name or number, and is expressed as a character string. Or may be expressed numerically.
Further, if the position information regarding the kth object included in the position information P output from the video object detection means 20 is P _k , the position information P can be expressed as Equation (3).

The position information P may be the image coordinates (f _k , g _k ) of the representative point corresponding to the existence range of the video object, or may be a numerical value expressing the position and the area spread. For example, the center of gravity of the video object area can be used as the representative point.

For example, when the existence area of the kth video object corresponds to the area _Dk in the input image H, the barycentric point can be calculated by Expression (4). That is, the average value in the horizontal direction of the image coordinate values of all the pixels belonging to the region D _k and the average value in the vertical method are calculated and set as an element f _k and an element g _{k of the} position information P _k , respectively.

In addition, the bounding box of the area D _k described above (a rectangular area inscribed in the area D _k ) can be used as the position information P _k . If the upper left point of the bounding box is (p _k , q _k ) and the lower right point is (r _k , s _k ), the position information P _k can be obtained as shown in equation (5). That is, the minimum value in the horizontal direction and the vertical direction and the maximum value in the horizontal direction and the vertical direction of the image coordinates in all the pixels belonging to the region D _k are obtained, and the elements p _k and q _{k of} the position information P _k are _obtained . Let element r _k and element s _k .

The expression of the bounding box is arbitrary, such as the expression using the upper left point and the lower right point, the expression using the upper left point, the width and the height, and the expression using the center point, the width and the height, as in Expression (5). Can be used.
Further, for example, the bitmap information of the region D _k itself may be used as the position information P _k as shown in Expression (6).

  As a technique for realizing video object detection by the video object detection means 20, for example, the document “Paul Viola and Michael Jones:“ Rapid Object Detection using a Boosted Cascade of Simple Features ”, Proceedings of the 2001 IEEE Computer Society Conference on Vision and Pattern Recognition, pp.511-518, (2001) ”can be used.

  This method uses multiple simple classifiers called weak classifiers created using statistical learning in accordance with the features that appear in the target object to be detected. The sum obtained by weighting the discrimination result is calculated, and whether or not the object is a desired object is determined based on the sum. In addition, according to the present method, since the reliability of whether or not a desired object is detected can be defined by the magnitude of the sum calculated by the strong classifier, priority is given to the detected video object as described above. Suitable for cases.

The classifier used in this method is learned in advance using learning data of a human face image and a non-face image as a target object to be detected, thereby detecting a face from the input image H, Appearance position and size can be output.
Similarly, this method is pre-learned using, for example, learning data of a ball image and a non-ball image, thereby detecting a ball from the input image H and outputting the appearance position and size of the ball. can do.

Further, the chroma key method can be used as a method for detecting a video object. In the chroma key method, pixels belonging to a color range designated in advance can be extracted as a non-video object area, and other areas can be extracted as a video object area, and a silhouette image of the video object can be obtained. The silhouette image obtained here is divided for each connected region, each connected region is set as the above-described region _Dk, and for example, the position information is output by the expression method of the above-described equations (4) to (6). it can.

  Further, for example, an average color is obtained for each video object region obtained from a silhouette image, and the average color belongs to the video object region by determining which of several representative colors is preset. The color of the clothes of the person can be color-coded, and this can be used as the type information Q.

  The exaggeration target video object selecting means (video object selecting means) 30 inputs the type information Q and position information P of the K video objects detected by the video object detecting means 20, and the inputted K videos. Based on the object type information Q and the position information P, a video object that meets a predetermined condition is selected from these video objects as a video object to be exaggerated, and the selected number (W). Are sequentially output to the video object area cutout means 40 and the image composition means 60.

  For example, the exaggeration target video object selecting unit 30 may determine that a certain specific type of video object satisfies a predetermined positional relationship with respect to another video object (group). Outputs position information about the object. Specifically, for example, when a video object having the type “person” comes close to a video object having the type “ball” within a predetermined distance (for example, when the distance between the centers of gravity is within 10 pixels). The position information の of the video object whose type is “person” is output.

  Next, the detailed configuration of the video object detection means 20 and the exaggeration target video object selection means 30 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the video object detection means and the exaggeration target video object selection means of the image conversion apparatus according to the first embodiment of the present invention.

Video object detecting means 20, the particular M number in total up to a first specific image object detecting means 21 ₁ and the second specific image object detecting unit 21 ₂ to the M particular video object detecting means 21 _M (where, M is a natural number) The video object detection means is included.

Each m-th specific video object detection means 21 _m (m∈ {1, 2,..., M}) inputs the input image H supplied from the image input device 2 and from among the input images H that are input, A predetermined specific type of video object is detected, respectively, and the type information Q ^(m) and position information P ^(m) of the detected video object are used as the first determination logic 31 ₁ to 31 of the exaggeration target video object selection unit 30. N decision logic 31 outputs to _N. Here, Q ^(m) and P ^(m) indicate type information and position information output by the m-th specific video object detection means 21 _m , respectively.

The m-th specific video object detection means 21 _m sequentially detects video objects of a specific type Q ^(m) determined in advance from the input image H, and outputs the position information P ^(m) . Here, the number of video objects detected by the m-th specific video object detection means 21 _m is set to K ^(m) . When K ^(m) is 1 or more, the position information of the kth (kε {0, 1,..., K ^(m) −1}) video object included in the position information P ^(m) is represented by P _k ^{(m )} , And when K ^(m) = 0, if P ^(m) = φ (φ is an empty set), the position information P ^(m) can be expressed as in Equation (7).

Examples of the predetermined video object type include “person's face”, “ball”, “person's foot”, “goal”, “corner”, “number”, and “uniform”. Specifically, for example, the first specific image object detecting means 21 _1, type detects the video object "human face", and outputs the type of the "human face" as the type information Q ^(1), The position information P ⁽¹⁾ is output. The second specific image object detecting means 21 ₂ detects an image object of type "ball", and outputs the type of the "ball" as the type information Q ^(2), and outputs the position information P ⁽²⁾ . When a plurality of video objects belonging to one type, for example, “person's face”, are detected, the position information P ⁽¹⁾ is set to each detected video object as shown in the equation (7). The information is composed of a plurality of elements corresponding to the position information.

Exaggerated target video object selecting unit 30, N pieces in total up to the first decision logic 31 ₁ and the second decision logic 31 ₂ to N-th decision logic 31 _N (where, N is a natural number) and decision logic, object selecting means 32.

Each n-th decision logic (determination logic _{means) 31 n (n∈ {1,2,} ..., N}) , respectively the first specific image object 21 _first through M particular video object detecting means 21 of the video object detecting means 20 and input type information Q ⁽¹⁾ to type information Q a ^(M) and the position information P ⁽¹⁾ to the position information P ^(M) output from the _M, predetermined, the exaggeration process in accordance with the logic of each decision The target video object is determined, and the position information Π ⁽ⁿ⁾ of the determined video object is output to the target selection unit 32.

The nth decision logic 31 _n is, for example, whether or not there is one or more specific types of video objects, and if there are specific types of video objects, the position of the video object, the video object Based on the positional relationship between each other, it is determined whether there is a video object to be output as a candidate for the exaggeration process among the video objects corresponding to the input positional information P ^{(1) to} P ^(M). When there is a video object that should be a candidate for the exaggeration process, the position information Π ⁽ⁿ⁾ corresponding to the video object is output to the target selection unit 32.

The number of video objects determined by the nth decision logic 31 _n as a candidate for the exaggeration process is an integer greater than or equal to 0, and the number is J ⁽ⁿ⁾ . When J ⁽ⁿ⁾ is 1 or more, the position information of the _jth video object (j∈ {0, 1,..., J ⁽ⁿ⁾ −1}) included in the output position information Π ⁽ⁿ⁾ is represented as ｊ _j ^{Let (n)} . Further, when J ⁽ⁿ⁾ = 0, if Π ⁽ⁿ⁾ = φ (φ is an empty set), the position information Π ⁽ⁿ⁾ can be expressed as in equation (8).

The first decision logic 31 ₁ to N decision logic 31 _N, which are N decision logics, include type information Q ⁽¹⁾ to type information Q ^(M) and position information P ⁽¹⁾ to input video object. Based on the position information P ^(M) , logic for determining a video object to be a candidate for each exaggeration process is predetermined. For example, the first decision logic 31 ₁ includes the position information P ^{(1) of} the video object detected by the first specific video object detection means 21 ₁ and whose type information Q ⁽¹⁾ is “person's face”, and 2 Referring to the position information P ⁽²⁾ of the video object whose type information Q ⁽²⁾ detected by the specific video object detection means 21 ₂ is “ball”, the video object of “person's face” is subject to exaggeration processing If it is determined whether or not to be a candidate, and if it is determined to be a target candidate, the position information Π ⁽¹⁾ of the video object of the “person's face” is output to the target selecting means 32.

More specifically, first, the first decision logic 311 _first calculates the k-th position information P _k ^{(1 )} included in the position information P ⁽¹⁾ of the video object whose type information Q ⁽¹⁾ is “person's face”. ⁾ And κ-th position information P _κ ⁽²⁾ included in the position information P ⁽²⁾ of the video object whose type information Q ⁽²⁾ is “ball”, for all combinations of k and κ The distance D (k, κ) between the face and the ball is obtained by the equation (9). ^However, if ^{P (1)} to the number of elements ^{K (1)} or ^contained, either one or both of the ^{P (2)} to contain the number of elements ^{K (2)} is 0, the first decision logic 31 ₁ does not determine the distance D (k, κ), and shall not perform subsequent operations and output. In a ball game with one ball like soccer, the number K ⁽²⁾ of the balls is preferably 0 or 1, but K ⁽²⁾ takes an integer larger than 1 due to erroneous detection or the like. A case may be assumed.

  Here, the function dist (P, P ′) represents the distance between two video objects having position information P and position information P ′. For example, it is possible to output the Euclidean distance between the gravity center positions of two video objects.

Subsequently, for example, as shown in Expression (10), the first determination logic 31 _{1 1} relates to each index κ indicating the element of the position information P ⁽²⁾ of the video object whose type is “Ball”. The index h (κ) indicating the element of the position information P ⁽¹⁾ of the video object whose type is “person's face” that minimizes the distance D (k, κ) shown in 9) is obtained.
In this way, the first decision logic 31 ₁ may determine a video object of a face of a person in closest proximity to each ball as candidates for exaggeration process.

Finally, the first decision logic 31 _1, obtained by the equation (10), the type of video objects "human face" position information _{P h} a ^{_(kappa) (1),} shown in equation (11) Thus, the information is output to the object selection means 32 as each element of the position information Π ⁽¹⁾ .

Further, for example, the second decision logic 31 _2, detected by the first specific image object detecting unit 21 _1, the position information P of the video object that is type information Q ⁽¹⁾ is "human face" a ⁽¹⁾ With reference to this, it is determined as a candidate for the exaggeration process, and the determined position information Π ⁽²⁾ of the video object of “person's face” is output to the target selection means 32.

In detail, first, second determination logic 31 2, type information Q (1) k-th position information is included in the position information P (1) of the video object that is "human face" P k (1 ) And a video object having the κ-th position information P κ (1) are obtained by Expression (12). However, if the number of elements K included in the position information P (1) (1) is 0, the second decision logic 31 2 does not seek distance E (k, kappa), also the subsequent operation and output Shall not be performed.

  Here, the function dist (P, P ′) is the distance between the two video objects having the position information P and the position information P ′ as in the equation (9). For example, the center of gravity of each of the two video objects The Euclidean distance between positions shall be output.

Subsequently, the second decision logic 31 _2, for example, as shown in Equation (13), for each index κ of classification points to elements of the position information of the video object that is "human face" P ^(1), An index f indicating the element of the position information P ⁽¹⁾ of the video object whose type is “person's face” that minimizes the sum of the distances E (k, κ) relating to k shown in Expression (12) is obtained.
In this manner, it is possible to select a video object located at a place where the video objects are most crowded among the detected video objects of the “person's face”.

Finally, a second decision logic 31 _2, obtained by the equation (13), type is the positional information P _{f ⁽¹⁾} of the video object "human face", positioned as shown in Equation (14) It outputs to the object selection means 32 as an element of information Π ⁽²⁾ .

In this manner, each nth decision logic 31 _n performs exaggeration processing on a specific type of video object detected in accordance with the positional relationship between different types or types of video objects such as “person's face” and “ball”. Whether or not to be a target candidate is selected and determined, and the position information 位置⁽ⁿ⁾ of the determined video object is output to the target selecting means 32.

The object selection means 32 inputs the position information Π ⁽¹⁾ to the position information Π ^(N) output by the first decision logic 31 ₁ to the Nth decision logic 31 _N , and the inputted position information Π ⁽¹⁾ to The number of elements of the position information of the video object determined as a candidate for the exaggeration process included in the position information Π ^(N) is narrowed down, and the video object region cutout means 40 and the image composition means 60 are set using the narrowed elements as the position information Π. (See FIG. 1).

For example, the object selection unit 32 sets the priority order for each of the first determination logic 31 ₁ to the Nth determination logic 31 _N and sets the upper limit value W of the number of position information pieces to be output, thereby determining the priority. The n-th decision logic 31 having a higher value is selected and output in order from the elements included in the position information Π ⁽ⁿ⁾ input by the _n , and the total number of output elements reaches W or all elements of the input position information Until position information is output.

For example, in the case than the first decision logic 31 ₁ was set high second decision logic 31 ₂ priority, position information [pi ⁽¹⁾ the target selecting means shown from the first decision logic 31 ₁ in the equation (11) 32 are input to, when the position information shown from the second decision logic 31 ₂ in the equation (14) [pi ^{that (2)} is input, the processing as follows.

When the upper limit value W of the position information す る to be output is one, if K ⁽²⁾ ≧ 1, only the element P _f ^{(1) of} the position information Π ⁽²⁾ shown in Expression (14) is output. If K ⁽²⁾ = 0, the empty set φ is output. In this way, if even one video object of “person's face” is detected, among the detected video objects of “person's face” as the video objects to be exaggerated most, A video object located at a place where video objects are most dense can be selected.

When the upper limit value W of the position information す る to be output is 2 or more and K ⁽¹⁾ or less, the element P _f ^{(1) of} the position information Π ⁽²⁾ shown in the equation (14 ⁾ and the equation (11) Among the elements of the position information Π ⁽¹⁾ indicated by ^(1), a maximum of W−1 is output. As a method of selecting W-1 elements, for example, among the elements of the position information Π ⁽¹⁾ shown in Expression (11), the first W-1 elements (K ⁽¹⁾ satisfy W-1 ^). If not, all the elements shown in Expression (11) are sequentially output as position information Π. In this way, in addition to the video object located in the most crowded place of the video objects among the “person's face” selected and detected, each ball can be applied within a range not exceeding the upper limit value W. It is possible to select a video object of the face of the person closest to you. For example, when the number of balls is one, such as soccer, K ⁽²⁾ = 1 or less. Therefore, when a ball is detected, the video object of the person's face located at the most crowded place of the person's face Then, the video object of the face of the person closest to the ball is selected as an exaggeration processing target, and the corresponding position information is output as a bag.

Returning to FIG. 1, the description of the image conversion apparatus 1 will be continued.
The video object area cutout means (partial image cutout means) 40 inputs the input image H supplied from the image input device 2 and W position information Π sequentially output from the exaggeration target video object selection means 30. , W partial images T corresponding to each position information Π are sequentially output to the partial image scaling means 50.

  The video object region cutout means 40 cuts out a local region in the vicinity including the video object specified by the position information の of the video object selected as the exaggeration processing target from the input image H, and uses it as a partial image T of the video object. Output.

  As a method of cutting out the partial image T, for example, when the position information Π represents the position of the video object by a representative point (for example, the center of gravity), a region near the representative point of the input image H is cut out and A partial image T of the video object can be obtained. Such a nearby region can be, for example, an inner region of a circle having a radius R (R is the number of pixels) centered on the representative point.

Here, referring to FIG. 4 (refer to FIG. 1 as appropriate), an example of cutting out a partial image is shown. FIG. 4 is a diagram for explaining a state of image conversion by the image conversion apparatus according to the first embodiment of the present invention.
In the example shown in FIG. 4, in the input image H, as an output of the exaggeration target video object selecting means 30, the video object O ₁ whose type information Q is “person's face” is selected as an exaggeration processing target, and the representative point It is detected at a position indicated by (f ₁ , g ₁ ). At this time, the video object region cutout means 40 cuts out an image in the vicinity of the point (f ₁ , g ₁ ) in the input image H (circular in this example), and changes the partial image as a partial image T ₁ of the video object O _1. Output to the multiplier 50.

  Note that the shape of the partial image T of the video object can be any of a rectangle, a polygon, and an arbitrary shape in addition to a circle as in this example. For example, when the position information P is expressed by the bounding box shown in Expression (5), the position information P may be cut out in accordance with the position and size of the bounding box. Further, for example, when the position information P is expressed by a region having an arbitrary shape represented by Expression (6), the position information P can be cut out according to the region shape.

Returning to FIG. 1, the description will be continued.
The partial image scaling unit 50 sequentially inputs the W partial images T output from the video object region cutout unit 40, performs a scaling process on the input partial image T, and creates a zoomed partial image U. Then, the generated zoomed partial images U are sequentially output to the image synthesizing means 60.

  The partial image scaling unit 50 performs a scaling process on the partial image T of the video object cut out by the video object area cutout unit 40 to convert the resolution. In this case, the partial image scaling unit 50 applies a scaling factor larger than the scaling factor by which the entire image scaling unit 10 scales the input image H. That is, the scaled partial image U obtained by the partial image scaling unit 50 is relatively larger than the scaled whole image L obtained by the whole image scaling unit 10.

  The scaling process for the partial image T at this time is not limited to the enlargement process, and when the reduction ratio of the entire scaled image L is small, it is reduced at the same scale or a reduction ratio larger than the reduction ratio of the entire scaled image L. Processing may be performed. For example, when the reduction ratio of the entire zoomed image L is 1/6, the partial image T can be reduced by a factor of 2 and the zoomed partial image U can be output. In this case, the zoomed partial image U is an image that is enlarged three times relative to the zoomed whole image L.

  As described above, the entire image (magnification entire image L) and the partial image (magnification) are partially enlarged by the image magnification unit 100 including the entire image magnification unit 10 and the partial image magnification unit 50. An image pair consisting of the double partial image U) can be generated.

  The input image H is not necessarily subjected to the scaling process, and the input image H may be output to the image composition unit 60 as the entire scaled image L with the same resolution. When the input image H is not scaled by the whole image scaling unit 10, the partial image T is enlarged by the partial image scaling unit 50 to create a scaled partial image U, thereby substantially adding the input image H. An image pair consisting of the (entire image) and the zoomed partial image U can be generated.

Furthermore, in the case where the input image H is reduced by the entire image scaling unit 10 to create the entire enlarged image L, the partial image T is not subjected to the scaling process by the partial image scaling unit 50. The image T may be output to the image synthesizing unit 60 as a zoomed partial image U with the same resolution. In this case, it is possible to generate an image pair composed of the entire zoom image L and the partial image T.

  As described above, the scaling factor of the entire image scaling unit 10 and the scaling factor of the partial image scaling unit 50 are arbitrary depending on the relationship between the resolution of the input image H and the resolution of the image display device 3 that is desired to display an image. You may comprise so that it can set to.

  The partial image scaling unit 50 can obtain the scaled partial image U by interpolating and resampling the partial image T of the video object. When there are a plurality of partial images T of the video object, a magnification partial image U is created for each of them. The scaling factor when there are a plurality of partial images T of the video object may be the same scaling factor for all the video objects, or a plurality of different scaling factors may be mixed. In the case where the variable magnifications are mixed, for example, the partial image T is set so that the size (for example, width, height, or area) of the variable partial image U obtained as a result of scaling the partial image T is the same. It is also possible to change the scaling factor every time.

  Any interpolation method can be used. For example, nearest neighbor interpolation method (nearest neighbor interpolation method), bilinear interpolation method (bilinear interpolation method) or cubic interpolation method (cubic convolution interpolation method or bicubic interpolation method). Can be used.

  The image synthesizing unit 60 includes the entire scaled image L output from the entire image scaling unit 10, W position information boxes sequentially output from the exaggeration target video object selecting unit 30, and the partial image scaling unit 50. Are sequentially input from W, and a composite image G is generated and output to the image display device 3.

  The image synthesizing unit 60 corresponds to the scaled partial image U for the scaled partial image U output from the partial image scaling unit 50 for the scaled whole image L output from the whole image scaling unit 10. Then, the composite image G is created by overwriting the corresponding position on the entire zoom image L determined by the position information Π output from the exaggeration target video object selecting means 30. When a plurality of scaled partial images U are input, overwriting synthesis is sequentially performed on the scaled whole image L based on the position information 対 応 corresponding to each scaled partial image U.

Next, a method of overwriting and composing the zoomed partial image U on the zoomed entire image L will be described.
First, the positional information of the k-th video object output from the exaggeration target video object selecting means 30 is Π _k , the partial image corresponding to the positional information Π _k is T _k , and the scaling obtained by scaling the partial image T _k. Let U _k be a partial image. When the kth scaling partial image U _k is synthesized at a position on the entire scaling image L corresponding to the location indicated by the positional information _{ｋ k} on the input image H, for example, a point representing the positional information _{ｋ k} As the center of gravity is obtained, the image coordinates are scale-converted based on the scaling ratio of the image scaling processing by the overall image scaling means 10, and the center of gravity of the scaled partial image U _k matches the scale-converted image coordinates. The zoomed partial image U _k is placed in

Here, with reference to FIG. 4 (refer to FIG. 1 as appropriate), a state in which the zoomed partial image U is overwritten and combined with the zoomed entire image L is described. In the example shown in FIG. 4, it is assumed that the video object O ₁ of “person's face” is selected as an exaggeration target by the exaggeration target video object selection means 30. Selecting has been the image coordinates on the input image H of the center of gravity of the partial image T ₁ obtained from the position information [pi ₁ video objects O ₁ and (f _{1, g 1).}

Next, the image coordinates (f ₁ , g ₁ ) are scaled based on the scaling factor of the entire image scaling unit 10. For example, magnification is M _x in the horizontal _direction, when the magnification in the vertical direction is M _y are image coordinates scale image coordinates after conversion, i.e., on the zooming whole image L of (f _{1, g 1)} The image coordinates are (M _x f ₁ , M _y g ₁ ).
Then, the image coordinates of the zooming whole image _{L (M x f 1, M} y g 1), the scaled partial image _{U 1,} by arranging such that the center of gravity coincides, to create a composite image G be able to.

In this example, the case where there is one video object to be exaggerated has been described, but there may be a plurality of video objects to be exaggerated. When a plurality of position information 位置 of the video object to be exaggerated is output from the exaggeration target video object selecting means 30, the position information に つ い て is scale-converted for each of them, and the scaled whole image is corresponding to each position. The corresponding scaled partial image U is overwritten on L. On the other hand, when the position information Π is not output from the exaggeration target video object selection means 30, that is, when the number of video objects to be exaggerated is zero, the overwriting composition operation is not performed. However, even when the overwriting composition operation is not performed, the entire zoom image L output from the image composition means 60 is formally called a composite image G.

  In the example shown in FIG. 4, the scaling process of the partial image T creates a scaled partial image U with a constant magnification for the entire area of one partial image T, and the corresponding scaled whole image L The composite image G is created by arranging the upper partial images so that the centers of gravity coincide with each other.

  For example, the variable-magnification partial image U is arranged with the boundary with the image of another video object in the variable-magnification whole image L to be joined to the variable-magnification partial image U as a reference point, and In order to smoothly join the double partial image U, the partial image T may be subjected to a non-linear scaling process to create the zoom partial image U.

  With reference to FIGS. 5 and 6 (refer to FIG. 4 as appropriate), an example in which the partial image T is subjected to non-linear scaling and combined with the scaled whole image L will be described. FIG. 5 is a diagram for explaining an example of non-linear scaling of a partial image, where (a) is a partial image T and (b) is a diagram illustrating a state of a non-linear scaling variable partial image U. FIG. 6 is a diagram illustrating an example of a composite image, where (a) illustrates a composite image obtained by combining a zoomed entire image and a linearly zoomed partial image, and (b) illustrates a zoomed entire image. FIG. 3 is a diagram illustrating a composite image obtained by combining a zoomed partial image that has been subjected to nonlinear scaling.

  First, FIG. 5A shows a state of the partial image T cut out from the input image H. Here, it is assumed that the entire zoom image L is reduced. In addition, the reference point of the partial image T when the images are combined is a point indicated by a circle in the center of the bottom portion.

At this time, as shown in FIG. 5B, the zoom partial image U has a horizontal scaling factor at the base including the reference point indicated by a circle, and a scaling factor of the scaling whole image L with respect to the input image H. To match. A trapezoidal scaling partial image U can be created by performing nonlinear scaling processing that increases the scaling ratio in the horizontal direction as the distance from the reference point in the vertical direction increases. By arranging such a zoomed partial image U on the entire zoomed image L with a junction with another video object as a reference point, on the synthesized image G, the apparent continuity at the bottom portion of the zoomed partial image U The partial image T of the video object can be enlarged and exaggerated presentation can be performed while maintaining the characteristics.

6 shows a composite image G created by exaggerating the video object O _{2 of the} head of the human image object M ₂ of the input image H shown in FIG. It is an example.

In the synthesized image G shown in FIG. 6A, a scaled partial image U ₂ created by linear scaling of the partial image of the head video object O ₂ is obtained in the same manner as in the example shown in FIG. , And an image synthesized so that the center of gravity coincides with the corresponding center of gravity position (M _x f ₂ , M _y g ₂ ) of the entire zoomed image.

On the other hand, the synthesized image G shown in FIG. 6 (b) is obtained by applying the present technique to a zoomed partial image U ₂ ′ created by nonlinear scaling of the partial image of the head video object O ₂ . The original joint between the video object O _{2 of the} head and the video object M _{2 of the} person, that is, the neck position (image coordinates (M _x f ₂ ′, M _y g ₂ ′)) as a reference point. Note that f ₂ ′ and g ₂ ′ are the horizontal coordinate and the vertical coordinate of the neck of the person's video object M ₂ in the input image H, respectively.

As shown in FIG. 6B, in the exaggeration process of the video object O ₂ of the head, the neck of the person corresponding to the bottom portion of the trapezoidal scaling partial image U shown in FIG. By making the horizontal scaling factor of the zoomed partial image U at the reference point the same as the scaling factor of the overall zooming image L, the cervical continuity on the composite image G is ensured and a more natural impression The composite image G can be created.

  In the first embodiment, “person's face (head)” is described as an example of the video object to be exaggerated. However, the “number” and “name” drawn on the uniform are exaggerated. In addition, the input image is not limited to a soccer game, but can be applied to other sports or video from a surveillance camera.

As described above, the configuration of the image conversion apparatus 1 according to the first embodiment has been described. The image conversion apparatus 1 can execute a part of or all of the dedicated hardware by creating a computer program, It can also be realized by operating an arithmetic device, a storage device, an input device, an image display device and the like in the computer. This program (image conversion program) can be provided via a communication line, or can be written on a recording medium such as a CD-ROM and distributed .

[Operation of image converter]
Next, the operation of the image conversion apparatus 1 in the first embodiment of the present invention will be described with reference to FIG. 7 (refer to FIG. 1 as appropriate). FIG. 7 is a flowchart showing the flow of processing of the image conversion apparatus in the first embodiment of the present invention shown in FIG.

  First, the image conversion apparatus 1 detects a predetermined specific type of video object from the input image H supplied from the image input apparatus 2 by the video object detection means 20, and the type information Q of the detected video object. And the position information P are output to the exaggeration target video object selecting means 30 (step S11).

  Next, the image conversion apparatus 1 uses the exaggeration target video object selection unit 30 to select a video object to be exaggerated from the video objects detected by the video object detection unit 20 in step S11, and selects the selected video. The object position information Π is output to the video object region cutout means 40 and the image composition means 60 (step S12).

  Next, the image conversion device 1 is supplied from the image input device 2 by referring to the position information の of the video object selected by the exaggeration target video object selection unit 30 in step S12 by the video object region cutout unit 40. A region including the video object is cut out from the input image H as a partial image T, and is output to the partial image scaling unit 50 of the image scaling unit 100 (step S13).

  Next, the image conversion apparatus 1 performs the scaling process on the partial image T cut out by the video object area cutout unit 40 in step S13 by the partial image scaling unit 50 of the image scaling unit 100, and the scaling described later. A zoomed partial image U that is enlarged relative to the entire image L is created and output to the image composition means 60 (step S14).

  While the image conversion apparatus 1 performs the processing from step S11 to step S14 described above, the entire input image H supplied from the image input apparatus 2 is changed by the whole image scaling means 10 of the image scaling means 100. A magnification process is performed to create a zoomed whole image L at a zoom ratio suitable for the resolution of the image display device 3 and output it to the image synthesizing means 60 (step S15).

  When both the partial image scaling process in step S14 and the entire image scaling process in step S15 are completed, the image conversion device 1 is selected by the image composition means 60 by the exaggeration target video object selection means 30 in step S12. Referring to the position information た, the scaled whole image L created by the whole image scaling unit 10 in step S15 is overwritten with the scaled partial image U created by the partial image scaling unit 50 in step S14. Then, a composite image G is created and output to the image display device 3 (step S16).

  Here, the processing from step S11 to step S14 and the processing of step S15 are performed in parallel, but the image composition processing of step S16 is the partial image scaling processing of step S14 and the entire image of step S15. It starts after the scaling process is completed. Therefore, the processing from step S11 to step S14 and the processing of step S15 may be executed sequentially. At this time, either the process from step S11 to step S14 or the process of step S15 may be executed first. The process of step S15 is inserted and executed in the middle of the process from step S11 to step S14. Also good.

  When there are a plurality of video objects to be exaggerated, every time the position information 対 応 corresponding to the video object to be exaggerated is output by the exaggeration target video object selecting unit 30 in step S12, Steps S14 and S16 are repeatedly executed. Here, since it is not necessary to create the entire zoom image L twice, the execution of step S15 is limited to once. In step S <b> 16, when the zoomed partial image U of the second and subsequent exaggerated video objects is input to the image composition unit 60, the image conversion device 1 sequentially applies the composite image G to the composite image G. Overwrite and synthesize the rescaled partial image U repeatedly.

  In this way, the image conversion device 1 converts the input image H supplied from the image input device 2 into a composite image G having a resolution suitable for the image display device 3 and outputs it. The image can be converted into an exaggerated image and displayed on the image display device 3 so as not to impair the performance.

Next, referring to FIG. 8 (refer to FIGS. 3, 1 and 7 as appropriate), details of the video object detection means 20 and the exaggeration target video object selection means 30 of the image conversion apparatus 1 in the first embodiment of the present invention. The operation will be described. FIG. 8 is a flowchart showing the processing flow of the video object detection means and the exaggeration target video object selection means of the image conversion apparatus in the first embodiment of the present invention shown in FIG.
Note that the flowchart shown in FIG. 8 corresponds to step S11 and step S12 of the flowchart shown in FIG.

Video video object detecting means 20, the first specific image object detecting means 21 _1, from among the input image H supplied from the image input device 2, for detecting a video object of a specific type predetermined detected and outputs to the type information ^{Q (1)} and the position information ^{P (1)} the first decision logic 31 of the exaggerated object image object selecting means 30 the _first through N decision logic 31 _N objects (step _S21 1).

Similarly, the video object detection means 20 is supplied from the image input device 2 by the other m-th specific video object detection means 21 _m (m∈ {1, 2,..., M}) in parallel with step S21 _1. A predetermined specific type of video object is detected from each of the input images H, and the type information Q ^(m) and position information P ^{(m) of} the detected video object are exaggerated target video object selection means. It outputs to 30 first decision logic 31 ₁ thru | or Nth decision logic 31 _N (step S21 _m (step S21 ₂ -step S21 _M )).

Here, steps S21 ₁ to S21 _M are processed in parallel, but each step may be executed sequentially in any order.

When all the processes of step S21 ₁ to step S21 _M are completed, the exaggeration target video object selecting unit 30 performs the first specific video object detecting unit 21 ₁ to 21 in step S21 ₁ to step S21 _{M according} to the first determination logic 31 ₁ . with reference to the first specific image object detecting means 21 _M of the video object that is output by the type information ^{Q (1)} ~ type information ^{Q (M)} and the position information ^{P (1)} ~ position information ^{P (M),} is detected A video object to be a candidate for the exaggeration process is determined from the video objects, and the position information Π ⁽¹⁾ of the determined video object is output to the target selection unit 32 (step S22 ₁ ).

Similarly, exaggerated target video object selecting means 30, in parallel with Step S22 _1, the n decision other logic _{31 n (n∈ {1,2, ...} , N}) by steps S21 ₁ ~ step S21 _M The video object type information Q ⁽¹⁾ to the type information Q ^(M) and the position information P ⁽¹⁾ to the position information output by the first specific video object detection unit 21 ₁ to the M-th specific video object detection unit 21 _M Referring to P ^(M) , a video object to be exaggerated from the detected video objects is determined, and position information Π ⁽ⁿ⁾ of the determined video object is output to target selection means 32. (Step S22 ₂ to Step S22 _N ).

Here, steps S22 1 _~ step S22 _N has been to perform parallel processing, each of the steps may be performed sequentially in any order.

Step S22 ₁ to exaggerated target video object selection unit 30 and all of the processing in step S22 _N is completed, the object selecting means 32, exaggerated by the first specific image object detecting means 21 ₁ through M particular video object detecting means 21 _M A video object to be exaggerated within a range of not more than the upper limit value W is selected from the video objects determined as processing target candidates, and the position information 切 り 出 し of the selected video object is sequentially extracted as a video object area extracting means. 40 and the image composition means 60.

  In this way, the video object detection unit 20 and the exaggeration target video object selection unit 30 cooperate to select a video object that meets a specific condition to be subjected to the exaggeration process from many types of video objects. can do.

  As described above, according to the image conversion device 1 in the first embodiment of the present invention, many types of video objects are detected from the input image H and detected even if they are of the same type or the same type. Based on the positional relationship between the selected video objects, the video object to be exaggerated is selected, and the selected video object is subjected to the exaggeration process such as image enlargement. Therefore, it can be automatically converted into the composite image G that is exaggerated.

  In particular, when the input image H, which is a high-resolution whole image, is converted into a low-resolution variable whole image L suitable for the low-resolution image display device 3 by performing a reduction process, the low-resolution image display device 3 Thus, while displaying the entire image in the same manner as the high-resolution original image, it is possible to convert into a composite image G with excellent visibility in which a specific video object to be noticed is selectively exaggerated.

  Further, as a method of exaggerating the video object, a non-linear change in which the magnification rate of the partial image T of the video object to be exaggerated is increased with increasing the distance from the reference point with a joint portion with another video object as a reference point. By doubling, the video object can be converted into an image that can be exaggerated and presented with a more natural impression.

Second Embodiment
[Configuration of Image Conversion Device]
Next, the configuration of the image conversion apparatus 1a according to the second embodiment of the present invention will be described with reference to FIG. FIG. 9 is a block diagram showing a configuration of an image conversion apparatus according to the second embodiment of the present invention.

  The image conversion apparatus 1a shown in FIG. 9 includes a video object detection unit 20, an exaggeration target video object selection unit 30, a video object area cutout unit 40, an image scaling unit 100a including a partial image scaling unit 50, The image composition means 60 is included.

  The image conversion device 1 a receives the input image H supplied from the image input device 2, creates a composite image G, and outputs it to the image display device 3.

Next, the configuration of each unit of the image conversion apparatus 1a will be described.
In the image conversion apparatus 1a shown in FIG. 9, the components having the same reference numerals as those of the image conversion apparatus 1 shown in FIG. 1 perform the same functions, and thus detailed description thereof will be omitted as appropriate.
The image conversion apparatus 1a has a configuration in which the entire image scaling unit 10 is removed from the image scaling unit 100 of the image conversion apparatus 1 in the first embodiment shown in FIG.

  In the image conversion apparatus 1a according to the second embodiment, the entire image of the input image H is not subjected to the scaling process, but is synthesized without changing the input image H. Therefore, the partial image scaling unit 50 of the image scaling unit 100a includes: The enlargement process is performed exclusively as an exaggeration process for the partial image T.

  As a result, the scaled partial image U becomes an image that is relatively enlarged with respect to the input image H, and the image scaling unit 100a is an entire image in which the partial image is relatively enlarged by the partial image scaling unit 50. An image pair composed of (input image H) and a partial image (magnification partial image U) can be generated.

  The image composition unit 60 sequentially outputs the input image H supplied from the image input device 2, W position information boxes sequentially output from the exaggeration target video object selection unit 30, and the partial image scaling unit 50. The W scaled partial images U to be input are input, a composite image G is created, and is output to the image display device 3.

  The image synthesizing means 60 uses the enlarged partial image U output from the partial image scaling means 50 of the image scaling means 100a for the input image H supplied from the image input device 2, as an exaggeration target video object selection means. The composite image G is obtained by overwriting the position of the input image H corresponding to the position information 出力 output from 30. When a plurality of zoomed partial images U are input, overwriting is sequentially performed on the input image H based on the position information Π corresponding to each zoomed partial image U.

[Operation of image converter]
Next, the operation of the image conversion apparatus 1a in the second embodiment of the present invention will be described with reference to FIG. FIG. 10 is a flowchart showing the flow of processing of the image conversion apparatus in the second embodiment of the present invention shown in FIG.

  First, the image conversion device 1a detects a predetermined specific type of video object from the input image H supplied from the image input device 2 by the video object detection means 20, and the type information Q of the detected video object. And the position information P are output to the exaggeration target video object selecting means 30 (step S31).

  Next, the image conversion apparatus 1a uses the exaggeration target video object selection unit 30 to select a video object to be exaggerated from the video objects detected by the video object detection unit 20 in step S31, and selects the selected video. The object position information Π is output to the video object area cutout means 40 and the image composition means 60 (step S32).

  Next, the image conversion device 1a is supplied from the image input device 2 by referring to the position information の of the video object selected by the exaggeration target video object selection unit 30 in step S32 by the video object region cutout unit 40. A region including the video object is cut out from the input image H as a partial image T, and is output to the partial image scaling unit 50 of the image scaling unit 100a (step S33).

  Next, the image conversion apparatus 1a performs the scaling process (enlargement process) on the partial image T cut out by the video object area cutout unit 40 in step S33 by the partial image scaling unit 50 of the image scaling unit 100a. A zoomed partial image U is created and output to the image composition means 60 (step S34).

  Finally, the image conversion device 1a refers to the position information 選定 selected by the exaggeration target video object selection unit 30 in step S32 by the image synthesis unit 60, and adds the input image H supplied from the image input device 2 to the input image H. In step S34, the scaled partial image U created by the partial image scaling unit 50 is overwritten and synthesized to create a composite image G, which is output to the image display device 3 (step S35).

  When there are a plurality of video objects to be exaggerated, every time the position information 対 応 corresponding to the video object to be exaggerated is output by the exaggeration target video object selecting unit 30 in step S32, the steps S33 to S33 are performed. Step S35 is repeatedly executed. In step S <b> 35, when the zoomed partial image U of the second and subsequent exaggerated video objects is input to the image composition unit 60, the image conversion device 1 a sequentially applies the composite image G to the composite image G. Overwrite and synthesize the rescaled partial image U repeatedly.

  As described above, according to the image conversion device 1a of the second embodiment of the present invention, when the resolution conversion is not performed on the entire input image H, for example, the input image H is captured over a wide range. Even when the video object appearing therein is small and the visibility is low, a specific video object to be noticed can be automatically converted into an image that is selectively exaggerated.

It is particularly suitable for applications that do not require conversion of the overall resolution of the input image H, and is economically advantageous because the means for scaling the entire image can be omitted. Since the entire scaling process of the image H can be omitted, the processing load on the computer can be reduced.
Further, as a method of exaggerating the video object, the nonlinear scaling method shown in FIG. 5 may be used.

<Third Embodiment>
[Configuration of Image Conversion Device]
Next, the configuration of the image conversion apparatus 1b according to the third embodiment of the present invention will be described with reference to FIG. FIG. 11 is a block diagram showing a configuration of an image conversion apparatus according to the third embodiment of the present invention.

An image conversion apparatus 1b shown in FIG. 11 includes an image scaling unit 100b including an entire image scaling unit 10, a video object detection unit 20, an exaggeration target video object selection unit 30, a video object area cutout unit 40, The image composition means 60 is included.
The image conversion device 1 b receives the input image H supplied from the image input device 2, creates a composite image G, and outputs it to the image display device 3.

Next, the configuration of each unit of the image conversion apparatus 1b will be described.
In the image conversion apparatus 1b shown in FIG. 11, the components having the same reference numerals as those of the image conversion apparatus 1 shown in FIG. 1 perform the same functions, and thus detailed description thereof will be omitted as appropriate.
The image conversion apparatus 1b has a configuration in which the partial image scaling unit 50 is removed from the image scaling unit 100 of the image conversion apparatus 1 in the first embodiment shown in FIG.

  The entire image scaling unit 10 of the image scaling unit 100 b inputs the entire image supplied from the image input device 2 as the input image H, and outputs the scaled entire image L to the image composition unit 60. Here, the entire image scaling unit 10 exclusively performs a reduction process so that the input image H is adapted to the low-resolution image display device 3.

  In the image conversion apparatus 1b according to the third embodiment, as described above, the entire image scaling unit 10 exclusively reduces the input image H to obtain a scaled entire image L suitable for the low-resolution image display device 3. create. For this reason, the image conversion apparatus 1b does not perform the scaling process (enlargement process) on the partial image T by the partial image scaling unit 50 like the image conversion apparatus 1 in the first embodiment shown in FIG. The image T is an image that is enlarged relative to the entire zoom image L.

  Therefore, the image scaling unit 100b generates an image pair composed of the whole image (magnification whole image L) and the partial image (partial image T) in which the partial image is relatively enlarged by the whole image scaling unit 10. be able to.

  The video object area cutout means (partial image cutout means) 40 inputs the input image H supplied from the image input device 2 and W position information Π sequentially output from the exaggeration target video object selection means 30. The W partial images T corresponding to each position information Π are sequentially output to the image composition means 60.

  The image synthesizing unit 60 includes a scaled whole image L output from the whole image scaling unit 10, W position information boxes sequentially output from the exaggeration target video object selecting unit 30, and a video object area clipping unit 40. The W partial images T that are sequentially output are input, a composite image G is created and output to the image display device 3.

  The image synthesizing unit 60 outputs the partial image T output from the video object area clipping unit 40 to the exaggerated target video object selecting unit 30 for the scaled whole image L output from the whole image scaling unit 10. By overwriting the position of the entire scaled image L corresponding to the position information る, a composite image G is obtained. When a plurality of partial images T are input, overwriting synthesis is sequentially performed on the entire scaled image L based on the position information 対 応 corresponding to each partial image T.

  Next, the operation of the image conversion apparatus 1b in the third embodiment will be described with reference to FIG. FIG. 12 is a flowchart showing the flow of processing of the image conversion apparatus in the third embodiment of the present invention shown in FIG.

[Operation of image converter]
The image conversion device 1b first detects a predetermined specific type of video object from the input image H supplied from the image input device 2 by the video object detection means 20, and the type information Q of the detected video object. And the position information P are output to the exaggeration target video object selecting means 30 (step S41).

  Next, the image conversion apparatus 1b selects the video object to be exaggerated from the video objects detected by the video object detection unit 20 in step S41 by the exaggeration target video object selection unit 30, and selects the selected video. The object position information Π is sequentially output to the video object region cutout means 40 and the image composition means 60 (step S42).

  Next, the image conversion apparatus 1b is supplied from the image input apparatus 2 with reference to the position information の of the video object selected by the exaggeration target video object selection means 30 in step S42 by the video object area cutout means 40. A region including the video object is cut out as a partial image T from the input image H and output to the image composition means 60 (step S43).

  The image conversion apparatus 1b performs the processing from step S41 to step S43, while changing the entire input image H supplied from the image input apparatus 2 by the overall image scaling means 10 of the image scaling means 100b. A magnification process (reduction process) is performed to create a zoomed whole image L with a zoom ratio suitable for the resolution of the low-resolution image display device 3, and is output to the image composition means 60 (step S44).

  When both the video object region cutout process in step S43 and the entire image scaling process in step S44 are completed, the image conversion device 1b is selected by the exaggeration target video object selection means 30 in step S42 by the image composition means 60. Referring to the position information た, the partial image T cut out by the video object area cutout unit 60 in step S43 is overwritten and synthesized on the full-scale image L created by the whole image enlargement unit 10 in step S44. A composite image G is created and output to the image display device 3 (step S45).

  Here, the processing from step S41 to step S43 and the processing of step S44 are performed in parallel. However, the image composition processing of step S45 is the video object region cutout processing of step S43 and the entire image of step S44. It starts after the scaling process is completed. Therefore, the processing from step S41 to step S43 and the processing of step S44 may be executed sequentially. At this time, either the process from step S41 to step S43 or the process of step S44 may be executed first, and the process of step S44 is inserted and executed in the middle of the process from step S41 to step S43. Also good.

  If there are a plurality of video objects to be exaggerated, step S43 and step S43 each time position information corresponding to the video object to be exaggerated is output by the exaggeration target video object selecting unit 30. Step S45 is repeatedly executed. Here, since it is not necessary to create the entire zoom image L twice, the execution of step S44 is limited to once. In step S <b> 45, when the partial image T of the second and subsequent exaggerated video objects is input to the image composition unit 60, the image conversion apparatus 1 sequentially repeats the partial image with respect to the composite image G by the image composition unit 60. The image T is overwritten and synthesized.

  As described above, according to the image conversion device 1b of the third embodiment of the present invention, the high-resolution input image H is subjected to the reduction process exclusively and is subjected to the reduction processing, and the entire scaled image adapted to the low-resolution image display device 3 is used. In the case of conversion to L, the partial image T cut out from the input image H is an image that is enlarged relative to the entire variable magnification image L without performing an enlargement process. By synthesizing the partial image T with the image L, the video object corresponding to the partial image T can be converted into a synthetic image G obtained by exaggerating the video object.

  In addition, it is economically advantageous because the means for scaling the partial image T can be omitted, and when the program is executed by a computer, the scaling process for the partial image T can be omitted. Can be reduced.

1 is a block diagram illustrating a configuration of an image conversion apparatus according to a first embodiment of the present invention. It is a figure showing the example of the input image in this invention. It is a block diagram which shows the structure of the video object detection means and the exaggeration object video object selection means of the image conversion apparatus in 1st Embodiment of this invention. It is a figure for demonstrating the mode of the image conversion by the image conversion apparatus in 1st Embodiment of this invention. It is a figure for demonstrating the example of the nonlinear scaling of a partial image, (a) is a partial image, (b) is a figure which shows the mode of the scaling partial image which carried out nonlinear scaling. It is a figure which shows the example of a synthetic | combination image, (a) shows the synthetic | combination image which synthesize | combined the magnification-change whole image and the linear magnification change partial image, (b) shows the magnification change whole image and the non-linear magnification change. It is a figure which shows the synthesized image which synthesize | combined the double partial image. It is a flowchart which shows the flow of a process of the image converter in 1st Embodiment of this invention. It is a flowchart which shows the flow of a process of the video object detection means and the exaggeration object video object selection means of the image conversion apparatus in 1st Embodiment of this invention. It is a block diagram which shows the structure of the image converter in 2nd Embodiment of this invention. It is a flowchart which shows the flow of a process of the image converter in 2nd Embodiment of this invention. It is a block diagram which shows the structure of the image converter in 3rd Embodiment of this invention. It is a flowchart which shows the flow of a process of the image converter in 3rd Embodiment of this invention.

Explanation of symbols

1, 1a, 1b Image conversion device 2 Image input device 3 Image display device 10 Whole image scaling unit 20 Video object detection unit 21 ₁ First specific video object detection unit 21 ₂ Second specific video object detection unit 21 _Mth M specific Video object detection means 30 Exaggeration target video object selection means (video object selection means)
31 ₁ First decision logic (decision logic means)
31 ₂ 2nd decision logic (decision logic means)
31 _Nth Nth decision logic (decision logic means)
32 Target selection means 40 Video object area cutout means (partial image cutout means)
50 partial image scaling unit 60 the image synthesizing unit 100, 100a, 100b image scaling unit G composite image H input image L zooming whole image M _{1, M 2} video objects O _{1, O} 2, _{O 3} video object P, P ⁽¹⁾ ,..., P ^(M) Position information Q, Q ⁽¹⁾ ,..., Q ^(M) Type information T, T ₁ , T ₂ Partial images U, U ₁ , U ₂ , U ₂ ′ Image Π, Π ⁽¹⁾ , ..., Π ^(N) Location information

Claims (6)

An image conversion device that converts an input image into an image obtained by relatively enlarging a specific video object in the input image,
Video object detection for detecting one or more types of video objects of a predetermined type from the input image and outputting type information including the type of the detected video object and position information indicating the detected position for each detected video object Means,
A video that selects a video object that meets a predetermined condition from the detected video objects as the specific video object based on the type information and position information of the video object detected by the video object detection means. Object selection means,
Partial image cutout means for cutting out a partial image including the specific video object from the input image;
Scaling at least one of the input image or the partial image to generate an image pair composed of the entire image and the partial image in which the partial image is relatively enlarged with respect to the entire area of the input image Image scaling means to
Image combining means for combining the whole image and the partial image forming the image pair;
Equipped with a,
The video object selecting means selects a video object that meets a predetermined condition from the detected video objects based on the type information and position information of the video object detected by the video object detecting means. A plurality of decision logic means for determining as candidates for the specific video object; and a target selection means for selecting the specific video object from the candidates for the specific video object determined by the plurality of decision logic means; an image conversion apparatus characterized by comprising a.   The image conversion apparatus according to claim 1, wherein the image scaling unit includes an entire image scaling unit that reduces the entire input image.   The image conversion apparatus according to claim 1, wherein the image scaling unit includes a partial image scaling unit that scales the partial image.   The partial image scaling means sets the partial image to the same scaling factor as the other video object at a junction with the other video object that is not the specific video object in the input image synthesized by the image synthesis means, While creating a zoomed partial image that is nonlinearly scaled so that the zooming factor increases as the distance from the joint increases, the image composition means uses the joint as a reference point to form an image pair with the zoomed partial image. The image conversion apparatus according to claim 3, wherein the image and the zoomed partial image are synthesized. Computer
The image conversion program for functioning as an image conversion apparatus as described in any one of Claims 1-4 . An image conversion method for converting an input image into an image obtained by relatively enlarging a specific video object in the input image,
One or more types of video objects of a predetermined type are detected from the input image by video object detection means, and for each detected video object, type information including the type of the detected video object and position information indicating the detected position are obtained. A video object detection step to output;
Based on the type information and position information of the video object detected in the video object detection step, a video object that meets a predetermined condition is selected from the detected video objects by the video object selection means. Video object selection step to select as an object,
A partial image cutout step of cutting out a partial image including the specific video object from the input image by a partial image cutout means;
An image pair composed of an entire image and a partial image in which the partial image is relatively enlarged with respect to the entire area of the input image by scaling at least one of the input image or the partial image. An image scaling step generated by the scaling means;
An image combining step of combining the input image and the partial image forming the image pair by an image combining means;
Only including,
In the video object selection step, based on the type information and position information of the video object detected in the video object detection step, a video object that meets a predetermined condition is selected from the detected video objects. A plurality of decision logic steps for determining as candidates for the specific video object; and a target selection step for selecting the specific video object from the candidates for the specific video object determined by the plurality of decision logic steps; The image conversion method characterized by including these.

Images