JP2004178368A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor for acquiring an image which is produced by extracting a space part (part of a frame) from noticed temporal frame intervals of data on serially photographed motion pictures with a noticed photographic subject at the center thereof and by enlarging the noticed photographic subject to facilitate viewing. <P>SOLUTION: A motion picture file composed of two or more serial still pictures is read, a motion picture frame thereof is decomposed to extract a frame within a designated time range, and thereafter the frame is subjected to three processes. The first process is a characteristic color extraction process 30 which performs processing for extracting the characteristic color of the photographic subject. In this process, a photographic subject is designated by pointing several points in a part of the photographic subject, and a characteristic color list at each point is produced. The second process is an area extraction processing process 32 which performs image area extraction processing. In this process, a color analogous to the characteristic color produced in the first process is extracted, a photographic subject is identified by superimposing extracted parts, a moving area of the image is acquired by superimposing photographic subject areas of entire frames, and detects the moving range of the photographic subject. The third process is an area cutout process 34, which on the basis of the moving range of the photographic subject which has been detected in the second process, cuts out an area thereof and synthesizes still pictures. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that processes moving image (including “continuous still images”) data, and more particularly to an image processing apparatus that obtains an image centered on a subject of interest.
[0002]
[Prior art]
In recent years, video cameras and digital cameras can be used to shoot and record videos. However, when printing recorded videos, still images can be taken from the recorded videos at regular time intervals and printed. Many. In particular, printing is often performed in the state of a divided photograph in which a plurality of image frames are displayed on one sheet. In this case, the subject to be noticed is often small in each image frame. It was difficult for the person to see.
[0003]
The following Patent Document 1 captures a moving picture with a shooting start switch, extracts a moving picture from a recorded moving picture at a predetermined time interval, and prints it. In particular, the moving picture is extracted from the beginning of the moving picture at a fixed time interval. An over-the-counter printing machine (so-called photo booth machine) is disclosed. This is related to printing when the shooting conditions of the photo booth are met, and if the frame of interest is extracted from an image shot with a normal video camera, the moving subject is too small or off the center of the screen. It was.
[0004]
Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique that allows a subject of interest to be always displayed at a predetermined position on a screen during reproduction from moving image data recorded including a camera shake state with a video camera or the like. In this method, a part of a subject is specified, the amount of movement is detected by pattern matching, and blur correction is performed, and data lost by blur correction is brought from other frames.
[0005]
However, when shooting a child's figure in a kindergarten athletic meet, etc., the user divides the subject of interest (my child) into a predetermined position on the screen, especially the center, and enlarges the subject of interest (my child). I have a strong desire to see prints of photographs.
[0006]
[Patent Document 1]
Japanese Patent Laid-Open No. 10-115866
[Patent Document 2]
Japanese Patent Application Laid-Open No. 11-187303
[Problems to be solved by the invention]
In consideration of the above circumstances, the present invention extracts a spatial portion (a part of a frame) centered on a subject of interest from a temporally noticeable frame section of continuously captured moving image data to make the subject of interest larger and easy to see. An object of the present invention is to provide an image processing apparatus for obtaining a processed image.
[0009]
[Means for Solving the Problems]
The present invention of claim 1 is an image processing apparatus for processing moving image data composed of a plurality of continuous images,
Means for reading out the video data and taking out the frames constituting the video in the specified time range from the video data;
A specifying means for specifying the subject from the frame of the extracted video data;
Detecting means for detecting a range in which the subject has moved in a frame constituting the moving image data;
An image processing apparatus including: a range in which the subject has moved; and a cutout unit that cuts out each frame of the moving image data of the image including the periphery of the movement range.
[0010]
According to the present invention of claim 1, the take-out means reads the moving picture data, takes out a frame constituting a moving picture in a specified time range from the moving picture data, that is, takes a moving picture constituent frame from a certain time point to another certain time point, The specifying unit specifies a subject existing in the frame from the extracted moving image constituting frame. The detecting means detects a range in which the subject has moved in a plurality of frames constituting the moving image data. The cutout means cuts out each frame of the moving image data of the image including the range in which the subject has moved between a plurality of frames and the periphery of the movement range. The cut-out image is an image including a space portion centered on the subject, and is an image that is easier to see than a large subject. The “moving image” means “continuous still image”.
[0011]
According to a second aspect of the present invention, there is provided the image processing apparatus according to the first aspect, further comprising a synthesizing unit that synthesizes each frame of the extracted moving image data.
[0012]
According to the second aspect of the present invention, for example, a multi-exposure image or an index image can be obtained by the synthesizing unit synthesizing each frame of the clipped moving image data.
[0013]
According to a third aspect of the present invention, in the image processing apparatus according to the first aspect, the specifying unit specifies a subject color by specifying a feature color of the subject and extracting a color approximate to the feature color from the constituent frame. It consists of.
[0014]
According to the third aspect of the present invention, the specifying means specifies the feature color of the subject in the frame, extracts the same kind of color as the feature color from the constituent frame, specifies the subject, and this extraction step is performed for each frame. To do.
[0015]
According to a fourth aspect of the present invention, the specifying unit specifies a subject color by extracting a color approximate to the feature color from the component frame and then superimposing the extracted portions. The image processing apparatus according to claim 1.
[0016]
According to the present invention of claim 4, the specifying means designates the characteristic color of the subject in the frame, extracts a color of the same type as the characteristic color from the constituent frame, and then superimposes the extracted portions to identify the subject. The identification process is performed for each frame.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0018]
FIG. 1 is a block diagram showing a configuration of an image processing apparatus 2 according to the present embodiment.
[0019]
When a recording medium is set in the recording medium slot 24, the recording medium controller 22 operates to read data stored in the recording medium. When instruction information is input from the operation unit 4 such as a keyboard or a mouse, the instruction information is sent to the CPU 12 via the interface 8, a control signal is sent from the CPU 12 to the recording medium controller 22, and the recording medium is sent via the recording medium controller 22. Necessary image data is extracted from the image data and stored in the RAM 20. The ROM 18 stores a control program. In the image processing unit 14, control-instructed image processing is performed on the image data stored in the RAM 20 by a control signal of the CPU 12 based on the instruction information from the operation unit 4, and the result is displayed via the display controller 10. Are displayed on the display 6. The user inputs necessary instruction information from the operation unit 4 while viewing the display 6.
[0020]
FIG. 2 is a diagram showing a schematic process until a still image obtained by enlarging a subject of interest from moving image data of a moving image file is synthesized.
[0021]
A moving image file composed of a plurality of continuous still images is read, and moving image frames in the moving image file are disassembled to extract frames within a specified time range. Then, it goes through three steps.
[0022]
The first step is subject feature color extraction processing 30 that performs subject feature color extraction processing. The subject feature color extraction process 30 designates a subject by pointing several points in the subject and creates a feature color list at each point.
[0023]
The second step is an area extraction process 32 for performing an image area extraction process. In this step, a color approximate to the characteristic color created in the first step is extracted, the subject is specified by superimposing the extracted portions, and the subject area of all frames is overlaid to obtain the moving area of the image. Detect the movement range of the subject.
[0024]
The third step is a region cutout process 34. In this step, the region is cut out based on the movement range of the subject detected in the second step, and a still image is synthesized.
[0025]
Details of each step will be described.
[0026]
Details of the first step will be described. FIG. 3 is a diagram showing a feature color designation method in the first step, and FIG. 4 is a flowchart showing a flow of the feature color extraction step in the first step.
[0027]
In FIG. 3, first, the feature color of the subject that the user wants to specify is specified in a specific still image frame n in the moving image frame. If the subject that the user wants to specify is a girl, the user points to a part of the subject, for example, point A that is hair, point B that is an arm, and point C that is a torso with the mouse that is the operation unit 4. By pointing, the x and y coordinates of points A, B and C are determined. Then, the R, G, and B values at the A point are taken as the feature color ref (1), the R, G, and B values at the B point are taken as the feature color ref (2), and the R, G, and B values at the C point. The value is taken in and set as the characteristic color ref (3).
[0028]
Referring to FIG. 4, a specific still image frame n is displayed on the display 6 (step 40), and it is determined whether the characteristic color extraction process has been repeated a required number of times (step 42). Points to a portion of the subject (step 44). The characteristic colors at points A, B, and C are extracted by taking in the R, G, and B values (step 46). Thereafter, the process returns to the determination of whether the feature color extraction process has been repeated a required number of times (step 42).
[0029]
The second step will be described. FIG. 5 is a conceptual diagram showing the image area extraction processing step, and FIG. 6 is a flowchart of the image area extraction processing step.
[0030]
In FIG. 5, for the feature color ref (1), the feature color ref (2) of the point B, and the feature color ref (3) of the point C extracted in the first step, the colors approximated to each of them are shown. Extract.
[0031]
For the feature color ref (1) at point A, a portion within the color difference ΔC from the feature color ref (1) at point A is extracted. Since the still image frame n is composed of a bit plane, the difference between ref (i) and the pixel i is taken to check whether the portion is within the color difference ΔC, whether it is 0 or 1. When the value is 0, it is not applicable, and it is regarded as a portion where the colors do not match. When the value is 1, it corresponds and is regarded as a color-matched part. As a result, the characteristic color ref (1) at point A and the approximate color portion are extracted.
[0032]
For the feature color ref (2) at point B and the feature color ref (3) at point C, similar color portions are similarly extracted.
[0033]
The extracted characteristic color portions are superimposed on one still image frame n. The subject area of the still image frame n is defined. The process is performed for all the frames in the specified time range, and the defined subject area is overlapped for all the frames in the specified range. Then, a moving area of the subject image over a plurality of designated frames is obtained. The upper left x coordinate, upper left y coordinate (min_x, min_y), lower right x coordinate, lower right y coordinate (max_x, max_y) of the movement area are specified so that the movement area becomes a rectangle, and the movement area is finally The range of motion of the subject is detected.
[0034]
Referring to FIG. 6, first, the first still image frame to be processed is selected (step 50), it is determined whether or not the frame has been repeated until the end of the frame (step 52), and when the final frame is completed, the process proceeds to step 64. Otherwise, it is determined in step 54 whether or not the image area extraction for the feature color ref (i) is repeated, and if the extraction is completed, the process proceeds to step 60. If not completed, a color matching area within ref (i) and the color difference ΔC is extracted in the step 56 by the above method. Thereafter, the next characteristic color is set (step 58), and the process returns to step 54.
[0035]
In step 60, in the above-described example, the color matching areas for the characteristic colors of the points A, B, and C are overlapped, and the subject area of the frame n is extracted. When this is completed, the next frame is targeted (step 62), and the process returns to step 52.
[0036]
In step 64, the subject areas of all designated frames are overlapped to obtain a moving area of the subject image. Then, the process proceeds to step 66, and the upper left x coordinate, upper left y coordinate (min_x, min_y), lower right x coordinate, lower right y coordinate (max_x, max_y) is specified and the moving area is finally defined and the process ends.
[0037]
The third step will be described. FIG. 7 is a flowchart showing the region cutout process. In this step, the image area of the movement range of the subject detected in the second step is cut out and a still image is synthesized.
[0038]
First, a target frame in which a moving area is defined is specified (step 70). The corresponding frame is read, and the defined moving area portion is cut out (step 72). The clipped image is output (step 74). It is determined whether the target frame is completed (step 76). If not completed, the target frame in which another moving area is defined is specified (step 78), and the process returns to step 72. If yes in step 76, the process ends.
[0039]
After the image is cut out, for example, it is conceivable to combine the cut out still image as a divided photograph. As shown in FIG. 8, in the synthesized image, the subject in each frame of the still image constituting the moving image is an enlarged image than the normal divided photograph, so that the subject to be noticed can be seen larger and easier to see. Can do. Further, still images can be further synthesized from the divided photographs shown in FIG.
[0040]
Further, by superimposing the cut out still image images, it is possible to print a multi-exposure composite photograph or to combine it with an index-like image.
[0041]
As described above, according to this embodiment, it is possible to obtain an image centered on a subject to be noticed when, for example, a continuously photographed separated photograph image is printed.
[0042]
In addition, since the movement of the subject of interest is known during the attention frame period of the moving image, it is easy to cut out the frame area to be printed with the decomposed photograph.
[0043]
Furthermore, even when the extracted photograph extraction region cannot be automatically cut out automatically by operations such as rect_min_x, rect_min_y, rect_max_x, rect_max_y that define the coordinates of the upper left corner and the lower right corner of the rectangle, the reference image R [x, y ], G [x, y], and B [x, y], it is possible to easily specify an area to be manually cut out.
[0044]
【The invention's effect】
According to the present invention, it is possible to obtain an image that is easier to see than a large subject, and therefore, it is possible to obtain an image centered on a subject that is desired to be noticed when, for example, a continuously photographed separated photograph image is printed.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus.
FIG. 2 is a diagram showing a schematic process until a still image in which a subject of interest is enlarged is synthesized from moving image data of a moving image file.
FIG. 3 is a view showing a feature color designation method in the first step.
FIG. 4 is a flowchart showing a flow of a feature color extraction process in the first process.
FIG. 5 is a conceptual diagram showing an image area extraction process.
FIG. 6 is a flowchart of an image area extraction processing step.
FIG. 7 is a flowchart showing an area cutout process.
FIG. 8 is a diagram of a divided photograph in which an enlarged subject image is obtained from moving image data.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 2 ... Image processing apparatus, 4 ... Operation part, 6 ... Display, 12 ... CPU, 14 ... Image processing part, 18 ... ROM, 20 ... RAM, 22 ... Recording medium controller, 24 ... Recording medium slot, 30 ... Subject characteristic color Extraction processing, 32... Image area extraction processing, 34.

Claims (4)

An image processing apparatus that processes moving image data including a plurality of continuous images,
Means for reading out the video data and taking out the frames constituting the video in the specified time range from the video data;
A specifying means for specifying the subject from the frame of the extracted video data;
Detecting means for detecting a range in which the subject has moved in a frame constituting the moving image data;
An image processing apparatus comprising: a range in which the subject has moved, and a cutout unit that cuts out each frame of the moving image data of the image including the periphery of the movement range. The image processing apparatus according to claim 1, further comprising a synthesizing unit that synthesizes each frame of the clipped moving image data. The image processing apparatus according to claim 1, wherein the specifying unit specifies a subject color by specifying a feature color of the subject and extracting a color approximate to the feature color from the constituent frame. The image processing apparatus according to claim 1, wherein the specifying unit specifies a subject color, extracts a color approximate to the feature color from the constituent frame, and then identifies the subject by superimposing the extracted portions. .

Images