JP5799818B2 - Image processing program and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing program and an image processing apparatus.

  Conventionally, many image processing programs for superposing a plurality of images have been proposed and used for various purposes. For example, Patent Document 1 discloses image processing for superimposing two images for product inspection.

JP 2011-20455 A

  By the way, the image superposition may be performed for comparison of two images or for sharpening of the image. In any case, in order to match the objects or to arrange them at a desired position. Accurate overlay is required. However, there is a limit to superposition by the image processing apparatus, and manual superposition needs to be used together. For example, if an image is unclear, manual superposition is not easy. Therefore, there has been a demand for a method capable of performing easy and accurate overlay.

  The present invention has been made to solve the above problems, and an object thereof is to provide an image processing program and an image processing apparatus capable of easily and accurately superimposing images.

  The image processing program according to the present invention includes a step of superimposing a plurality of images, a step of generating a difference image including a difference between pixel values of the superimposed images, and the plurality of images in a state where the difference image is displayed. Receiving from the user an input for positioning the image of the user.

  According to this configuration, when a plurality of images are superimposed, a difference image of each image is generated, and the two images can be positioned while being displayed. For example, an object in two images is displayed. Positioning to match may be performed such that there are no differences in the difference image. Therefore, a plurality of images can be easily and accurately superimposed.

  The positioning of the plurality of images can be performed by various methods. For example, at least one of the plurality of images can be moved, enlarged, reduced, rotated, or distorted.

  In the step of superimposing the plurality of images, both images can be superimposed so that the feature points of the images match. In this way, since these can be automatically superimposed based on the feature points of a plurality of images, the initial overlay operation by the user can be omitted. At this time, in the step of superimposing a plurality of images, in one of the plurality of images, a step of accepting selection of a region including a feature point from a user, and a feature point of the selected region is used as a feature point of another image. And a step of superimposing a plurality of images so as to coincide with each other. Therefore, since the user can arbitrarily set the region where the feature points exist, the overlay accuracy can be improved.

  The image processing program can further include a step of accepting at least one selection of the plurality of images from the user prior to the step of superimposing the plurality of images. Therefore, the user can arbitrarily select an image to be superimposed. In addition, the image can be automatically selected based on an arbitrary setting.

  An image processing apparatus according to the present invention includes an image superimposing unit that superimposes a plurality of images, a difference image creating unit that creates a difference image composed of pixel values of the superimposed images, and the difference image. A positioning receiving unit that receives input for positioning the plurality of images from a user in the displayed state.

  According to the present invention, images can be easily and accurately superimposed.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention. It is a figure of the basic screen before image data is taken in. It is a figure of the basic screen after image data was taken in. It is a figure which shows the dialog box for superimposition. It is a figure which shows an image selection window. It is a figure which shows an image selection window. It is a figure which shows an image selection window. It is a figure which shows the dialog box for superimposition. It is a figure which shows the example by which the superimposition of the image was displayed in the display window. It is a figure which shows the difference image of FIG. It is the figure which performed the positioning of the image in the difference image of FIG. It is a figure which shows the superimposition which is not displayed by the difference in FIG. It is a figure which shows the example of rotation of an image. It is a figure which shows the example of the image superimposed by rotation. It is a figure which shows the example of the distortion of an image. It is a figure which shows an image selection window. It is a figure which shows the dialog box for superimposition. It is a figure which shows the example by which the superimposition of the image was displayed in the display window. It is a figure which shows the difference image of FIG. It is a figure which shows the example of the window used for automatic superimposition. It is a figure which shows the window of FIG. 20 in which the feature point was shown. It is the example which performed the superimposition of the image using the feature point of FIG.

Hereinafter, an image processing program and an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of an image processing apparatus according to an embodiment of the present invention. The image processing program and the image processing apparatus according to the present embodiment are used when a plurality of images are superimposed. Hereinafter, an image to be subjected to image processing may be referred to as a frame image or simply an image.
<1. Overview of Image Processing Device>
As shown in FIG. 1, a general-purpose personal computer can be used as the image processing apparatus 1 according to the present embodiment. The image processing apparatus 1 is installed with an image processing program 2 that is an embodiment of an image processing program according to the present invention. The image processing program 2 is application software for supporting image processing for moving images and still images, and causes the image processing apparatus 1 to execute steps included in the operations described later.

  The image processing apparatus 1 includes a display 10, an input unit 20, a storage unit 30, and a control unit 40, which are connected by a bus line 5 so as to be able to communicate with each other. In the present embodiment, the display 10 is a liquid crystal display, and displays a screen or the like to be described later to the user. The input unit 20 includes a mouse and a keyboard, and accepts an operation from the user for the image processing apparatus 1. The storage unit 30 includes a nonvolatile hard disk, and the control unit 40 includes a CPU, a ROM, a volatile RAM, and the like.

  In the storage unit 30, an image processing program 2 and a software management area 50 are secured. The software management area 50 is an area used by the image processing program 2. In the software management area 50, an original image area 51, a reference image area 52, a selected image area 53, and a post-processing image area 54 are secured. The role of each of the areas 51 to 54 will be described later.

  The control unit 40 reads and executes the image processing program 2 stored in the storage unit 30, so that the image selection unit 41, the image superposition unit 42, the difference image generation unit 43, and the image positioning unit 44 are virtually executed. , And the image generation unit 45. The operation of each unit 41 to 45 will be described later.

<2. Details of Configuration and Operation of Image Processing Apparatus>
When the control unit 40 detects that the user has performed a predetermined operation via the input unit 20, the control unit 40 activates the image processing program 2. When the image processing program 2 is activated, a basic screen W1 (see FIG. 2) is displayed on the display 10. The control unit 40 controls display of all elements such as a screen, a window, a button, and the like displayed on the display 10.

<2-1. Importing image data>
The basic screen W1 accepts a command for capturing image data into the original image area 51 from the user. The image data captured in the original image area 51 is a target of image processing to be described later. The control unit 40 captures image data from the still image file or the moving image file into the original image area 51. In this specification, a still image file is a data file in a still image format, and a moving image file is a data file in a moving image format.

  When capturing image data from a still image file, the user operates the input unit 20 to specify one still image file or one folder. In the former case, the control unit 40 causes the user to input the address path and file name in the storage unit 30 of the still image file. In the latter case, the control unit 40 allows the user to input the address path and folder name in the storage unit 30 of the folder. Thereafter, the control unit 40 stores the specified one still image file or all the still image files in the specified one folder as a still image file group in the original image area 51.

  On the other hand, when capturing image data from a moving image file, the user operates the input unit 20 to input the address path and file name in the storage unit 30 of one moving image file. When the control unit 40 detects that the user has specified a moving image file, the control unit 40 displays a moving image capturing window (not shown) in an overlapping manner on the basic screen W1. The moving image capture window accepts selection of a time line of an arbitrary length from the user among all time lines of the specified moving image file. When the control unit 40 detects that the user has selected a timeline having an arbitrary length via the input unit 20, the control unit 40 generates a still image file group corresponding to the selection. This still image file group has a one-to-one correspondence with the frame group included in the moving image of the timeline related to the user's selection. Thereafter, the control unit 40 stores the still image file group in the original image area 51.

  Therefore, in the present embodiment, the target of image processing to be described later is not a moving image file but a still image file. The still image file is taken into the original image area 51 in file units, folder units, or all or part of time line units of moving image files.

<2-2. Playback of still image files>
When the still image file group is taken into the original image area 51, the control unit 40 displays the display window W2 (see FIG. 3) on the basic screen W1. The display windows W2 are displayed as many times as the number of operations for taking still image file groups into the original image area 51.

  First, one still image file (for example, a still image file corresponding to the first frame on the timeline) included in the still image file group captured in the original image area 51 is displayed in the display window W2. . Note that the control unit 40 arranges the still image files included in the still image file group along the timeline even if the still image file group is not derived from the moving image file but derived from the still image file. It is recognized that The array is automatically determined from file attributes (file name, creation date, update date, etc.).

  As will be described later, the frame displayed in the display window W2 is switched in response to a user operation. The control unit 40 manages the identification information of the frame currently displayed in the display window W2 in real time.

  The control unit 40 can reproduce the still image file group corresponding to the display window W2 as a moving image in the display window W2. As shown in FIG. 3, a window selection pull-down menu T1, a playback button T2, a frame advance button T3, a frame return button T4, and a timeline bar T5 are arranged on the basic screen W1.

  Even when there are a plurality of display windows W2, there is only one active display window W2. The control unit 40 receives a selection of which display window W2 is to be activated from the user via the window selection pull-down menu T1. Hereinafter, the still image file group corresponding to the active display window W2 is referred to as an active file group. A frame currently displayed in the active display window W2 is referred to as an active display frame.

  The playback button T2 accepts a playback command from the user as a moving image of the active file group. When the control unit 40 detects that the user has pressed the play button T2 via the input unit 20, the control unit 40 displays the frames of the active file group sequentially in the frame advance format along the timeline in the active display window W2. Let Note that the reproduction starts from the active display frame at the time when the reproduction button T2 is pressed. The playback button T2 accepts a playback stop command from the user. When the control unit 40 detects that the user has pressed the playback button T2 via the input unit 20 during playback, the control unit 40 fixes the display in the active display window W2 to the active display frame at that time.

  Each of the frame advance button T3 and the frame return button T4 receives an instruction from the user to switch the active display frame to the next previous frame along the timeline of the active file group.

  The timeline bar T5 schematically represents the timeline of the active file group. The timeline bar T5 is equally divided by the number of frames of the active file group in the direction in which the bar extends. The nth divided area from the left on the timeline bar T5 corresponds to the nth frame on the timeline of the active file group (n is a natural number).

  As shown in FIG. 3, the timeline bar T5 displays the divided area A1 corresponding to the selected frame group and the divided area A2 corresponding to the non-selected frame group in different modes. The selected frame group is a frame group corresponding to the currently selected section on the timeline of the active file group. The non-selected frame group is a frame group corresponding to a section that is not currently selected on the timeline of the active file group. In the present embodiment, the area A1 is displayed with a light tone color, and the area A2 is displayed with a dark tone color. The image selection unit 41 receives a selection of an arbitrary section on the timeline of the active file group from the user via the timeline bar T5. For example, an arbitrary frame or section can be selected by clicking or dragging the mouse on the timeline bar T5. Then, as will be described later, image processing is performed on one frame image of the selected frame group.

<2-3. Image processing>
Hereinafter, image processing for the selected frame group will be described. Here, image processing for superimposing a plurality of frame images for sharpening and comparison will be described. The image selection unit 41, the image superimposing unit 42, the difference image generating unit 43, the image positioning unit 45, and the image generating unit 46 described above can execute an image processing module for image superimposing processing. The image processing module is incorporated in the image processing program 2.

  The user operates the basic screen W1 via the input unit 20 to select a frame to be subjected to image processing in the timeline. When a frame group consisting of two or more frames is selected, the last frame image of the frame group is targeted. However, any frame image other than the tail can be targeted.

  Thus, when the image selection unit 41 detects that the image processing for superimposition has been selected from the pull-down menu on the basic screen W1 in a state where the frame image is selected, the frame image selected as described above is used as a reference. The image is stored in the reference image area 52, and a dialog box D1 as shown in FIG. 4 is displayed thereon. When the “Add” button is clicked in this dialog box D1, an image selection window W3 as shown in FIG. 5 is displayed. Similar to the basic screen W1 of FIG. 3, the image selection window W3 includes a timeline bar T0, a reproduction button T1, and the like, and can reproduce a group of images on the timeline. The panel arranged on the upper left side in the window W3 is the reference image panel P1 on which the above-described reference image is displayed, and the panel arranged on the right side displays an image to be superimposed. This is a preview panel P2. In this state, since the image has not yet been selected, the same image as the reference image panel P1 is displayed. Further, in the timeline bar T0, the currently selected frame is shown in gray as a selected frame (reference symbol B), and the lower half is shown in a place indicating the frame selected as the reference image. (Reference C).

  If any part of the timeline bar T0 is selected in this state, the selected part turns gray (reference B), and the frame image is displayed on the preview panel P2 as shown in FIG. Thus, when the frame image to be superimposed is determined while selecting the frame, the “Add” button is clicked. As a result, as shown in FIG. 7, the position of the frame of the image selected in the timeline bar T0 is colored (symbol D), and this image is displayed in the selected image region 53 as a selection image to be overlaid. Remembered. If the OK button is clicked in this state, the selected image is displayed as a preview P3 at the top of the dialog box D1, as shown in FIG. The file names of the selected images are displayed as a list in the list box LB at the center of the dialog box D1. Here, since only one image is selected, only one file name is displayed in the list box LB.

  If the file name in the list box LB is checked in this state, the image superimposing unit 42 detects this and displays an image in which the reference image and the selected image are superimposed on the display window W2, as shown in FIG. To do. In the figure, two cars are shown at the left end of the image, and it can be seen that the car image of the reference image and the car image of the selected image are superimposed on one image.

  In this state, when “difference” in the upper right of the dialog box D1 is checked, the difference image generation unit 43 detects this, and as shown in FIG. 10, a difference image based on the difference between the pixel values of the two images is displayed in the display window. Display on W2. In this difference image, a portion where there is no difference between the pixel values of the two images is shown in black, and a portion where there is a difference between the pixel values is shown in white. However, other color schemes may be used. The two frame images have the same background, but the positions of the automobiles are different. Therefore, the location where the automobile is displayed in each image is represented in white, and the others are displayed in black. In this way, the user can position the image while the difference image is displayed. For example, when the automobiles displayed on the two images are overlapped, necessary numerical values are appropriately input to the positioning input unit PB displayed at the bottom of the dialog box D1. For example, when the numerical values of X and Y shown in the “Move” part are changed, the image positioning unit 44 moves the selected image in the X direction or the Y direction, and superimposes the automobiles. FIG. 11 is a diagram illustrating an example in which automobiles are overlaid while displaying a difference image. As shown in the figure, in this image, since there is almost no white portion in the portion where the automobile is shown, it can be seen that the images of the automobile are superimposed. In this way, after the images are overlaid at an arbitrary position, when the “difference” check box in the dialog box D1 is unchecked, the difference image generation unit 43 detects this, and as shown in FIG. Is displayed in the display window W2. When the OK button is clicked in this state, the image generation unit 45 stores this image in the processed image area 54. That is, only this reference image is subjected to image processing as described above and stored together with other frame images in the timeline.

  In the above description, the numerical value of “movement” of the positioning input unit PB in the dialog box D1 is changed as the manual operation for superimposition, but other operations are also possible. In addition to movement, the positioning input unit PB includes “enlargement / reduction” for enlarging or reducing the selected image, “rotation” for rotating, and “distortion” for distorting the image. When “enlarge / reduce” is selected, the selected image can be enlarged and reduced in the X direction and the Y direction, respectively. When a value larger than 1 is inputted, the selected image is enlarged, and when a value larger than 0 and smaller than 1 is inputted, the selected image is reduced.

  When “Rotation” is selected, three types of rotation are possible. When the numerical value of X is changed, as shown in FIG. 13A, the selected image is rotated around the X axis extending in the horizontal direction at the center in the vertical direction of the image. When the numerical value of Y is changed, as shown in FIG. 13B, the selected image is rotated around the Y axis extending in the vertical direction at the center in the left-right direction of the image. When the numerical value of Z is changed, as shown in FIG. 13C, the selected image is rotated around the Z axis extending vertically from the screen at the center of the image. As an example when the numerical value of Z is changed, it is displayed as shown in FIG.

  When “distortion” is selected, two types of deformations are possible. When a positive number is input as the numerical value of H, as shown in FIG. 15A, the image is deformed so as to form a parallelogram while the upper side of the image is shifted to the left side and the lower side of the image is shifted to the right side. . On the other hand, when a negative number is input as the numerical value of H, the direction of deformation of the upper side and the lower side is reversed. When a positive number is input as the numerical value of V, as shown in FIG. 15B, the image is formed so as to form a parallelogram while the left side of the image is shifted upward and the right side of the image is shifted downward. Is deformed. On the other hand, when a negative number is input as the numerical value of V, the deformation directions of the left side and the right side are reversed.

  In the above description, the case where two images are superimposed has been described, but three or more images can be superimposed. For example, after the above-described image selection window W3 is opened, it is possible to select three or more frame images by repeatedly selecting a desired portion of the timeline bar T0 and clicking the add button. In the example shown in FIG. 16, two frame images are selected as the selected images (there are two signs D). When the OK button is clicked in this state, two file names are displayed in the list box LB of the dialog box D1, as shown in FIG. Among these files, the highlighted file is displayed on the preview panel P3, and can be moved, scaled, etc., so that one of the files is selected and highlighted, and then moved. In this example, both files are checked, but the checked files are superimposed on the display window W2. Therefore, if the check is removed, the frame image is not displayed on the display window W2. FIG. 18 shows an example of the display window W2 in which three frame images are superimposed. Further, when “difference” is checked in the dialog box D1, a difference image as shown in FIG. 19 is displayed in the display window W2. In this state, as described above, any file in the list box LB is selected and moved as necessary.

<3. Features>
As described above, according to the present embodiment, when two images are superimposed, a difference image between the two images is generated and the images can be positioned while being displayed. If the difference image is generated so that there is no difference, that is, a portion displayed in white, the objects can be superimposed. Therefore, it is possible to easily and accurately superimpose images that match the objects in the two images. In particular, when the image is unclear, it is possible to easily match the objects by using the difference image as described above.

  As an application of this image processing, for example, it can be used when the object is sharpened by superimposing the objects in two adjacent frame images in the timeline. It can also be used to confirm the size and position of an object. For example, by superimposing a frame image (reference image) on which an object is displayed with another frame image (selected image) on which an object serving as a reference for length or size is superimposed, The size can be estimated.

<4. Modification>
As mentioned above, although one Embodiment of this invention was described, this invention is not limited to the said embodiment, A various change is possible unless it deviates from the meaning. For example, the following changes can be made.

<4-1>
In the above embodiment, when the selected image is selected, the selected image is superimposed on the reference image as it is, and then the selected image is positioned so as to manually match the object. In addition, the first overlay can be automatically performed. For example, when an automatic overlay command is selected from the pull-down menu on the basic screen W1 with any frame on the timeline selected, the image overlay unit 42 selects the selected frame image as shown in FIG. Is displayed as a reference image. In this image, when an object to be superimposed, that is, a car, is surrounded by a rectangular region R by operating a mouse or a keyboard, the image superimposing unit 42, as shown in FIG. Feature points are automatically extracted, and a rhombus is displayed as a feature point S in the feature point selection window W4. Then, when the OK button is clicked, these feature points are stored. Thereafter, as in the above embodiment, when a selected image is selected from the image selection window W3, the image superimposing unit 42 extracts a corresponding feature point from the selected image, and a feature point of the selected image and a feature point of the reference image Both images are superimposed and displayed on the display window W2 so that they match. FIG. 22 is an example of superposition. As shown in the figure, it can be seen that the automobiles in both images are superimposed so as to coincide with each other by this processing. Thus, when the initial superposition is completed, for fine adjustment, the difference image can be displayed or the numerical value of the positioning input unit PB can be changed for positioning as in the above embodiment.

<4-2>
In the above embodiment, an image is selected from one timeline, but an image is selected from another timeline, or a plurality of images are selected from a single still image file regardless of the timeline, and the above-described overlay is performed. Can also be performed.

<4-3>
In the above embodiment, when the image is positioned, the selected image is moved. However, the reference image may be moved, in other words, at least one of the reference image and at least one selected image. May be moved.

<4-4>
The above-described difference image generation method, overlay method, and feature point extraction method are not particularly limited, and a known method can be employed. As a method for generating the difference image, for example, an image having the absolute value as the pixel value obtained by subtracting the target image value from the reference pixel value can be used as the difference image. In addition, as a method of superposition, for example, an image having a pixel value obtained by adding a reference pixel value and a target pixel value and dividing the sum by two can be used as a superposition image. . As a feature point extraction method, for example, a method such as edge detection or corner detection can be used.

1 Image processing device (computer)
2 Image processing program

Claims (6)

Selecting a plurality of images from a group of images on one timeline;
Overlaying the selected plurality of images;
Generating a difference image composed of pixel value differences of the superimposed images;
Receiving input from the user for positioning the plurality of images in a state in which the difference image is displayed;
Generating an image in which display of the difference image is canceled and the plurality of images after positioning are superimposed;
An image processing program for causing a computer to execute.   The image processing program according to claim 1, wherein positioning of the plurality of images is performed by moving, enlarging, reducing, rotating, or distorting at least one of the plurality of images.   The image processing program according to claim 1 or 2, wherein, in the step of superimposing the plurality of images, the plurality of images are superimposed so that feature points of the plurality of images coincide with each other. In the step of superimposing the plurality of images,
In one of the plurality of images, receiving a selection of a region including a feature point from a user;
Superimposing the plurality of images such that the feature points of the selected region match the feature points of other images;
The image processing program according to claim 3, comprising:   The image processing program according to any one of claims 1 to 4, further comprising a step of accepting selection of at least one of the plurality of images from a user prior to the step of superimposing the plurality of images. An image selection unit for selecting a plurality of images from an image group on one timeline;
An image superimposing unit that superimposes the plurality of selected images;
A difference image creation unit that creates a difference image including a difference between pixel values of the superimposed images;
An image positioning unit for positioning the plurality of images according to an input received from a user;
Equipped with a,
The difference image creating unit is an image processing device that generates an image in which the plurality of images after the positioning are overlaid, the display of the difference image being canceled .