JP5643607B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  In a laboratory or a manufacturing site, in order to analyze the movement of various subjects, a high-speed imaging device capable of imaging subjects at a high frame rate is used (for example, see Patent Document 1).

  For example, when subjects such as a plurality of workpieces conveyed by a belt conveyor are sequentially photographed by a high-speed photographing device, a plurality of similar moving images are acquired. If the shooting timings of a plurality of moving images are shifted, it is difficult to compare the movements of a plurality of subjects even if a plurality of moving images are reproduced simultaneously.

JP 2009-141709 A

  Therefore, it is conceivable to arrange a sensor in the movement path of the subject and determine the shooting timing of each subject in response to the output signal of the sensor. In this case, since a sensor is required, the configuration of the system using the high-speed imaging device is complicated. Further, when each subject moves at high speed, the shooting timing of the high-speed imaging device may be shifted even if the output signal of the sensor is used.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of easily comparing the states of objects in a plurality of moving images with a simple configuration.

(1) An image processing apparatus according to a first invention forms a first image of a plurality of frames constituting a moving image of a first object and a moving image of a second object that is the same as or different from the first object. An image processing apparatus for displaying a second image of a plurality of frames, the first image data of a plurality of frames for displaying the first image of a plurality of frames, and the first image data of a plurality of frames A plurality of first evaluation values associated with each other, a plurality of frames of second image data for displaying a plurality of frames of second images, and a plurality of second images corresponding to the plurality of frames of second image data. Based on the first image data of each frame stored in the storage unit storing the evaluation value of 2, a display unit having the first display area, the second display area, and the third display area Display the first image in the first display area. Based on the second image data of each frame stored in the storage unit, the second image is displayed in the second display area, and formed by a plurality of first evaluation values stored in the storage unit. A control unit that displays the second waveform formed by the first waveform and the plurality of second evaluation values stored in the storage unit in a third display area so as to be movable on the time axis; An operation unit operated by a user to move one of the first and second waveforms displayed in the display area on the time axis relative to the other waveform, and a plurality of first waveforms The evaluation value of 1 changes according to the state of the first object, the plurality of second evaluation values change according to the state of the second object, and the control unit is based on the operation of the operation unit. , One of the first and second waveforms is moved relative to the other waveform on the time axis. Together is to change based on one of the frame of the second image to be displayed on the frame and a second display area of the first image to be displayed in the first display area to the movement of one of the waveform, the control unit Displays a common index for designating an arbitrary position of the first and second waveforms in the third display area so as to be movable on the time axis, and the position of the first waveform designated by the index Correspond to the frame of the first image displayed in the first display area and the position of the second waveform specified by the index and the frame of the second image displayed in the second display area. The display of the first, second, and third display areas is controlled to maintain the corresponding state, and one of the first and second waveforms is changed to the other waveform based on the operation of the operation unit. Move relative to the time axis and The display of the first, second and third display areas is controlled so as to maintain the state .

  In this image processing apparatus, a plurality of frames of first image data, a plurality of first evaluation values, a plurality of frames of second image data, and a plurality of second evaluation values are stored in the storage unit.

  Based on the first image data of each frame, the first image is displayed in the first display area of the display unit, and based on the second image data of each frame, the second image is displayed on the second part of the display unit. Displayed in the display area. In addition, the first waveform formed by the plurality of first evaluation values and the second waveform formed by the plurality of second evaluation values can be moved on the time axis in the third display area of the display unit. Is displayed.

  The plurality of first evaluation values change according to the state of the first object, and the plurality of second evaluation values change according to the state of the second object. Therefore, when the first waveform and the second waveform have the same or similar shape portion, the first object in the first image of the frame corresponding to the same or similar shape portion of the first waveform. And the second object in the second image of the frame corresponding to the same or similar shaped part of the second waveform are in substantially the same state.

  The user can move one of the first and second waveforms displayed in the third display area on the time axis relative to the other waveform by operating the operation unit. Thereby, the correspondence between the frame of the first image displayed in the first display area and the frame of the second image displayed in the second display area can be changed.

  Accordingly, the user relatively moves the same or similar shaped portions of the first waveform and the second waveform, thereby including the first and second objects including substantially the same state of the first and second objects. Images can be displayed in the first and second display areas, respectively. Thereby, the moving image of the first object and the moving image of the second object can be reproduced so that the state of the first object and the state of the second object change at the same timing.

  As a result, the user can easily compare the states of the objects in the plurality of moving images with a simple configuration.

Further , a common index for designating an arbitrary position of the first and second waveforms is displayed in the third display area of the display unit so as to be movable on the time axis. Based on the operation of the operation unit, one of the first and second waveforms is moved relative to the other waveform on the time axis. Also in this case, the position of the first waveform designated by the index and the frame of the first image displayed in the first display area correspond to each other and the position of the second waveform designated by the index and the first waveform. The state corresponding to the frame of the second image displayed in the second display area is maintained.

  If the first waveform and the second waveform have portions of the same or similar shape, the first object in the first image of the frame corresponding to the same or similar shape portion of the first waveform; The second object in the second image of the frame corresponding to the same or similar shaped portion of the second waveform is in approximately the same state.

  Thereby, the user can set one waveform of the first waveform and the second waveform relative to the other waveform in a time axis so that portions of the same or similar shape of the first waveform and the second waveform overlap. By moving it up, the first and second images including substantially the same state of the first and second objects can be displayed in the first and second display areas, respectively. Thereby, the moving image of the first object and the moving image of the second object can be reproduced so that the state of the first object and the state of the second object change at the same timing.

  As a result, the user can easily compare the states of the objects in the plurality of moving images with a simple configuration.

( 2 ) The operation unit is operable to move one of the first and second waveforms on the time axis, and the control unit is based on an operation of moving one waveform by the operation unit. The one waveform and the index displayed in the third display area may be moved on the time axis, and the image in the display area corresponding to the other waveform in the first and second display areas may be changed. .

  In this case, the user can move one of the first and second waveforms on the time axis by operating the operation unit. Accordingly, the same or similar portions of the first waveform and the second waveform can be easily superimposed on the time axis.

( 3 ) The operation unit is operable to change a frame of an image displayed in one of the first and second display regions, and the control unit is based on an operation of changing the image by the operation unit. Then, while changing the image displayed in one display area, the waveform and the index corresponding to the other display area of the first and second waveforms may be moved on the time axis.

  In this case, the user changes the frame of the image displayed in one of the first and second display areas by operating the operation unit, thereby causing the first waveform and the second waveform to have the same or similar shape. Can be easily superimposed on the time axis.

( 4 ) The operation unit can instruct the reproduction of the moving image in the first display area and the reproduction of the moving image in the second display area, and the control unit performs the operation when the operation unit is instructed to reproduce the moving image. A first image of a plurality of frames after the first image displayed in one display area is sequentially displayed and a second image of a plurality of frames after the second image displayed in the second display area May be displayed sequentially.

  In this case, a plurality of frames after the first image displayed in the first display area are instructed by the operation unit to reproduce the moving picture in the first display area and the moving picture in the second display area. The first images are sequentially displayed and the second images of a plurality of frames after the second image displayed in the second display area are sequentially displayed. Thereby, it is possible to reproduce the moving image of the first object and the moving image of the second object so that the state of the first object and the state of the second object change at the same timing. .

( 5 ) The plurality of first evaluation values are values indicating the characteristics of the first image of the plurality of frames, and the plurality of second evaluation values are values indicating the characteristics of the second image of the plurality of frames. May be.

  The plurality of first evaluation values are generated from a first image of a plurality of frames, and the plurality of second evaluation values are generated from a second image of the plurality of frames. Therefore, it is not necessary to separately provide a device for generating a plurality of first evaluation values and a plurality of second evaluation values. This simplifies the configuration of the system including the image processing apparatus.

( 6 ) The value indicating the feature of the first image of the plurality of frames is a value related to the luminance of the first image of the plurality of frames, and the value indicating the feature of the second image of the plurality of frames is the second value of the plurality of frames. It may be a value related to the brightness of the image.

  The brightness of the first image changes according to the state of the first object. In addition, the luminance of the second image changes according to the state of the second object. Therefore, the first waveform changes according to the state of the first object, and the second waveform changes according to the state of the second object. Accordingly, one of the first and second waveforms is moved relative to the other waveform on the time axis so that portions of the same or similar shape of the first waveform and the second waveform overlap. Thus, the first and second images including substantially the same state of the first and second objects can be displayed in the first and second display areas, respectively.

( 7 ) The plurality of first and second evaluation values may be output values of a detection device that detects the states of the first and second objects.

  In this case, the states of the first and second objects can be reliably detected by the detection device. Accordingly, one of the first and second waveforms is moved relative to the other waveform on the time axis so that portions of the same or similar shape of the first waveform and the second waveform overlap. Thus, the first and second images including substantially the same state of the first and second objects can be displayed in the first and second display areas, respectively.

(8) The image processing method according to the second aspect of the invention configures a first image of a plurality of frames constituting a moving image of the first object and a moving image of a second object that is the same as or different from the first object. An image processing method for displaying a second image of a plurality of frames, the first image data of a plurality of frames for displaying the first image of a plurality of frames, and the first image data of a plurality of frames A plurality of first evaluation values associated with each other, a plurality of frames of second image data for displaying a plurality of frames of second images, and a plurality of second images corresponding to the plurality of frames of second image data. Based on the step of storing the evaluation value of 2, the step of displaying the first display area, the second display area and the third display area on the display unit, and the stored first image data of each frame The first image is the first table A first image formed by a plurality of first evaluation values stored and displayed in the second display area based on the second image data of each frame displayed and stored in the area; a step of displaying the movable third display area a second waveform which is formed by the waveform and Symbol憶by a plurality of second evaluation values on the time axis, the operation section operated by the user A step of accepting an operation of moving one of the first and second waveforms displayed in the third display area on the time axis relative to the other waveform; and an accepted operation of the operation unit Based on the first waveform and the second waveform, the first waveform is moved relative to the other waveform on the time axis, and the first image frame and the first waveform are displayed in the first display area. Second image to be displayed in the second display area A step of changing one of the frames based on the movement of one of the waveforms, and a third index for moving a common index for designating an arbitrary position of the first and second waveforms on the time axis. The position of the first waveform that is further displayed in the display area, corresponds to the position of the first waveform specified by the index and the frame of the first image displayed in the first display area, and is specified by the index And the display of the first, second and third display areas are controlled so as to maintain the corresponding state of the frame of the second image displayed in the second display area, and based on the operation of the operation unit The first, second and third display areas are displayed so that one of the first and second waveforms is moved relative to the other waveform on the time axis, and the state is maintained. A plurality of first evaluation values are provided. It changes according to the state of the 1st subject, and a plurality of 2nd evaluation values change according to the state of the 2nd subject.

  In this image processing method, a plurality of frames of first image data, a plurality of first evaluation values, a plurality of frames of second image data, and a plurality of second evaluation values are stored.

  Based on the first image data of each frame, the first image is displayed in the first display area of the display unit, and based on the second image data of each frame, the second image is displayed on the second part of the display unit. Displayed in the display area. In addition, the first waveform formed by the plurality of first evaluation values and the second waveform formed by the plurality of second evaluation values can be moved on the time axis in the third display area of the display unit. Is displayed.

  The plurality of first evaluation values change according to the state of the first object, and the plurality of second evaluation values change according to the state of the second object. Therefore, when the first waveform and the second waveform have the same or similar shape portion, the first object in the first image of the frame corresponding to the same or similar shape portion of the first waveform. And the second object in the second image of the frame corresponding to the same or similar shaped part of the second waveform are in substantially the same state.

  An operation of moving one of the first and second waveforms displayed in the third display area by operating the operation unit from the operation unit relative to the other waveform on the time axis is accepted. It is done. One of the first and second waveforms is moved on the time axis relative to the other waveform based on the accepted operation of the operation unit. Also, one of the first image frame displayed in the first display area and the second image frame displayed in the second display area is changed based on the movement of one waveform. Thereby, the user can change the correspondence between the frame of the first image displayed in the first display area and the frame of the second image displayed in the second display area.

  Accordingly, the user relatively moves the same or similar shaped portions of the first waveform and the second waveform, thereby including the first and second objects including substantially the same state of the first and second objects. Images can be displayed in the first and second display areas, respectively. Thereby, the moving image of the first object and the moving image of the second object can be reproduced so that the state of the first object and the state of the second object change at the same timing.

  As a result, the user can easily compare the states of the objects in the plurality of moving images with a simple configuration.

(9) An image processing program according to a third aspect forms a first image of a plurality of frames constituting a moving image of a first object and a moving image of a second object that is the same as or different from the first object. An image processing program for causing a processing device to execute image processing for displaying a plurality of frames of second images, wherein a plurality of frames of first image data for displaying a plurality of frames of first images, a plurality of frames A plurality of first evaluation values associated with the first image data, a plurality of frames of second image data for displaying a plurality of frames of second images, and a plurality of frames of second image data. A process of storing a plurality of associated second evaluation values, a process of displaying the first display area, the second display area, and the third display area on the display unit; 1 image data The first image is displayed in the first display area based on the second image, the second image is displayed in the second display area based on the stored second image data of each frame, and a plurality of stored displayed in the third display area movably first waveform Oyo second waveform formed by beauty Symbol憶by a plurality of second evaluation values on the time axis formed by the first evaluation value And a process of moving one of the first and second waveforms displayed in the third display area on the time axis relative to the other waveform from the operation unit operated by the user. And moving one waveform of the first and second waveforms relative to the other waveform on the time axis based on the received operation and the operation of the operation unit. First image frame and second display to be displayed in region A process for changing one of the frames of the second image displayed in the area based on the movement of one waveform, and a common index for designating an arbitrary position of the first and second waveforms as a time axis Further displayed in the third display area so as to be movable, the position of the first waveform designated by the index corresponds to the frame of the first image displayed in the first display area, and designated by the index The display of the first, second and third display areas is controlled so that the position of the second waveform to be held and the frame of the second image displayed in the second display area correspond to each other. Based on the operation of the operation unit, one of the first and second waveforms is moved relative to the other waveform on the time axis, and the first and second so as to maintain the state. And processing for controlling display of the third display area, a processing device The plurality of first evaluation values change in accordance with the state of the first object, and the plurality of second evaluation values change in accordance with the state of the second object.

  In this image processing program, a plurality of frames of first image data, a plurality of first evaluation values, a plurality of frames of second image data, and a plurality of second evaluation values are stored.

  Based on the first image data of each frame, the first image is displayed in the first display area of the display unit, and based on the second image data of each frame, the second image is displayed on the second part of the display unit. Displayed in the display area. In addition, the first waveform formed by the plurality of first evaluation values and the second waveform formed by the plurality of second evaluation values can be moved on the time axis in the third display area of the display unit. Is displayed.

  The plurality of first evaluation values change according to the state of the first object, and the plurality of second evaluation values change according to the state of the second object. Therefore, when the first waveform and the second waveform have the same or similar shape portion, the first object in the first image of the frame corresponding to the same or similar shape portion of the first waveform. And the second object in the second image of the frame corresponding to the same or similar shaped part of the second waveform are in substantially the same state.

  An operation of moving one of the first and second waveforms displayed in the third display area by operating the operation unit from the operation unit relative to the other waveform on the time axis is accepted. It is done. One of the first and second waveforms is moved on the time axis relative to the other waveform based on the accepted operation of the operation unit. Also, one of the first image frame displayed in the first display area and the second image frame displayed in the second display area is changed based on the movement of one waveform. Thereby, the user can change the correspondence between the frame of the first image displayed in the first display area and the frame of the second image displayed in the second display area.

  Accordingly, the user relatively moves the same or similar shaped portions of the first waveform and the second waveform, thereby including the first and second objects including substantially the same state of the first and second objects. Images can be displayed in the first and second display areas, respectively. Thereby, the moving image of the first object and the moving image of the second object can be reproduced so that the state of the first object and the state of the second object change at the same timing.

As a result, the user can easily compare the states of the objects in the plurality of moving images with a simple configuration.
(10) The image processing apparatus according to the first reference mode generates a plurality of frames of a first image and a moving image of a second object that is the same as or different from the first object, constituting the moving image of the first object. An image processing apparatus for displaying a second image of a plurality of frames constituting the first image data of a plurality of frames and a first image data of a plurality of frames for displaying the first image of a plurality of frames A plurality of first evaluation values associated with the second image data, a plurality of frames of second image data for displaying a second image of the plurality of frames, and a plurality of frames associated with the second image data of the plurality of frames. A storage unit that stores the second evaluation value, a display unit having a first display area, a second display area, and a third display area, and first image data of each frame stored in the storage unit A first image based on a first table The second image is displayed in the second display area based on the second image data of each frame stored in the storage unit and stored in the storage unit, and the plurality of first evaluation values stored in the storage unit are used. A control unit configured to display a second waveform formed by the first waveform formed and a plurality of second evaluation values stored in the storage unit in a third display area movably on the time axis; And an operation unit operated by a user to move one of the first and second waveforms displayed in the display area relative to the other waveform on the time axis. The first evaluation value changes in accordance with the state of the first object, the plurality of second evaluation values change in accordance with the state of the second object, and the control unit controls the operation of the operation unit. Based on one of the first and second waveforms, the time axis is relative to the other waveform. And moving one of the first image frame displayed in the first display area and the second image frame displayed in the second display area based on the movement of one waveform. It is.
(11) In the image processing method according to the second reference embodiment, the first image of a plurality of frames constituting the moving image of the first object and the moving image of the second object that is the same as or different from the first object are used. An image processing method for displaying a second image of a plurality of frames constituting the first image data of a plurality of frames and a first image data of a plurality of frames for displaying the first image of a plurality of frames. A plurality of first evaluation values associated with the second image data, a plurality of frames of second image data for displaying a second image of the plurality of frames, and a plurality of frames associated with the second image data of the plurality of frames. Based on the step of storing the second evaluation value, the step of displaying the first display area, the second display area and the third display area on the display unit, and the stored first image data of each frame The first image The second image is displayed on the second display area based on the stored second image data of each frame, and is formed by the plurality of stored first evaluation values. A step of displaying a second waveform formed by one waveform and a plurality of second evaluation values stored in the storage unit in a third display area movably on the time axis, and an operation operated by a user Receiving an operation of moving one of the first and second waveforms displayed in the third display area from the unit on the time axis relative to the other waveform, and the received operation unit Based on the above operation, one of the first and second waveforms is moved relative to the other waveform on the time axis, and the frame of the first image is displayed in the first display area. And display in the second display area And changing one of the frames of the second image based on the movement of one of the waveforms, and the plurality of first evaluation values change according to the state of the first object, The second evaluation value changes according to the state of the second object.
(12) The image processing program according to the third reference form generates a first image of a plurality of frames constituting a moving image of the first object and a moving image of a second object that is the same as or different from the first object. An image processing program for causing a processing device to execute image processing for displaying a second image of a plurality of frames constituting the first image data of a plurality of frames for displaying a first image of a plurality of frames, a plurality of images The plurality of first evaluation values associated with the first image data of the frame, the second image data of the plurality of frames for displaying the second image of the plurality of frames, and the second image data of the plurality of frames A process for storing a plurality of second evaluation values associated with each other, a process for displaying the first display area, the second display area, and the third display area on the display unit, and each stored frame First picture The first image is displayed in the first display area based on the data, the second image is displayed in the second display area based on the stored second image data of each frame, and the stored plural The first waveform formed by the first evaluation value and the second waveform formed by the plurality of second evaluation values stored in the storage unit can be moved on the time axis in the third display area. One of the first and second waveforms displayed in the third display area is moved on the time axis relative to the other waveform from the display process and the operation unit operated by the user. Based on the process of accepting the operation and the accepted operation of the operation unit, one of the first and second waveforms is moved relative to the other waveform on the time axis, and the first The frame of the first image displayed in the display area And processing to change one of the frames of the second image displayed in the second display area based on the movement of one waveform, and the plurality of first evaluation values are the first The plurality of second evaluation values change according to the state of the second object.

  According to the present invention, the states of objects in a plurality of moving images can be easily compared with a simple configuration.

1 is a block diagram illustrating a configuration of an image processing system including an image processing apparatus according to an embodiment of the present invention. It is a figure which shows the example of a display of the display part at the time of setting of imaging conditions. It is a schematic diagram which shows the memory | storage state of the video recording part of FIG. 1 during video recording processing. It is a schematic diagram which shows the other example of the memory | storage state of the video recording part of FIG. 1 during video recording processing. It is a figure which shows the example of a display of the display part during a video recording process. It is a figure which shows the example of a display of the display part at the time of completion | finish of a video recording process. It is a schematic diagram of the moving image file memorize | stored in a memory | storage part by file memory | storage process. It is a figure which shows the example of a display of the display part at the time of reproduction | regeneration processing. It is a figure which shows the example of a display of the display part at the time of a comparison process. It is a conceptual diagram which shows an example of the relationship between the image data of several frames contained in the 1st file and the 2nd file, the image of several frames, and several evaluation value. 10 is a graph obtained by enlarging a part of the first waveform and the second waveform displayed in the waveform display area of FIG. 9. It is a figure which shows the 1st example of operation of the display part at the time of a comparison process. 13 is a graph obtained by enlarging a part of the first waveform and the second waveform displayed in the waveform display area of FIG. 12. It is a figure which shows the 2nd example of operation of the display part at the time of a comparison process. FIG. 15 is a graph obtained by enlarging a part of the first waveform and the second waveform displayed in the waveform display area of FIG. 14. It is a flowchart of a comparison process. It is a flowchart of a comparison process. It is a flowchart of a comparison process. It is a flowchart of a comparison process. It is a figure which shows the example of a display of the display part at the time of a measurement process.

  Hereinafter, an image processing apparatus, an image processing method, and an image processing program according to an embodiment of the present invention will be described with reference to the drawings. As will be described later, the image processing apparatus according to the present embodiment includes a display unit. A moving image is displayed by continuously displaying images of a plurality of frames on the display unit.

(1) Basic Configuration of Image Processing System FIG. 1 is a block diagram showing a configuration of an image processing system 100 according to an embodiment of the present invention. As shown in FIG. 1, the image processing system 100 includes a main body unit 1, a camera 2, a lighting device 3, and an operation unit 4. The main body 1 constitutes the image processing apparatus according to the present embodiment. The main unit 1 includes a control unit 10, a display unit 20, a recording unit 30, a storage unit 40, a video memory 50, a waveform collection unit 60, a bus BS, and a plurality of interfaces (not shown). The control unit 10, the display unit 20, the recording unit 30, the storage unit 40, the video memory 50, and the waveform collection unit 60 are connected to the bus BS. The camera 2, the lighting device 3, and the operation unit 4 are each connected to the bus BS via an interface.

  A PLC (programmable logic controller) 5, a sensor 6 and a waveform collection device 7 can be connected to the main body 1. A sensor 8 can be connected to the waveform collection device 7. Details will be described later.

  The camera 2 includes an image sensor such as a CCD (charge coupled device) image sensor or a CMOS (complementary metal oxide semiconductor) image sensor. The exposure time (shutter speed), the shooting frame rate, and the like of the image sensor can be set by the user using the operation unit 4. The shooting frame rate is the number of frames of an image acquired per unit time. Note that the camera 2 of the present embodiment can shoot the subject W at a shooting frame rate of 15 fps (frame per second) or more and 32000 fps or less.

  When the subject W is photographed by the camera 2, an image of the subject W is acquired by the imaging device at the set shooting frame rate, and the acquired image is output as image data. As a result, a plurality of frames of image data corresponding to a plurality of frames of images are provided from the camera 2 to the main body 1. A moving image is constituted by images of a plurality of frames. Hereinafter, image data of a plurality of frames is referred to as moving image data.

  The camera 2 is attached to a support base (not shown) such as a tripod stand together with the illumination device 3. The illumination device 3 includes a light source such as a metal halide lamp or a high power LED (light emitting diode). The illumination device 3 is used for irradiating the subject W with light. When the subject W is sufficiently bright, the lighting device 3 may not be provided.

  The video memory 50 includes a memory such as a RAM (Random Access Memory). A plurality of frames of image data are sequentially supplied from the camera 2, the recording unit 30, or the storage unit 40 to the video memory 50. The given image data of a plurality of frames is sequentially stored in the video memory 50.

  The display unit 20 is configured by, for example, a liquid crystal display panel or an organic EL (electroluminescence) panel. A plurality of frames of images are continuously displayed on the display unit 20 based on the plurality of frames of image data sequentially stored in the video memory 50. Thereby, the moving image is displayed on the display unit 20.

  The operation unit 4 is configured by a pointing device such as a mouse or a trackball. The operation unit 4 may be configured with a keyboard together with or instead of the pointing device, or may be configured with a remote controller or a touch panel. The operation unit 4 may be configured integrally with the main body unit 1.

  The recording unit 30 includes a memory such as a RAM. The recording unit 30 includes a moving image data storage area for storing moving image data composed of a plurality of image data and a waveform data storage area for storing waveform data composed of a plurality of evaluation values. Details of the waveform data composed of a plurality of evaluation values will be described later.

  Recording multiple frames of image data (moving image data) from the camera 2 in the moving image data storage area of the recording unit 30 is called recording. When the user operates the operation unit 4, a moving image of the subject W photographed by the camera 2 is recorded. In the present embodiment, a plurality of evaluation values (waveform data) given from the waveform collection unit 60 are simultaneously stored in the waveform data storage area of the recording unit 30 during the recording process.

  The storage unit 40 is composed of a mass storage device such as a hard disk. When the user operates the operation unit 4, a moving image file including moving image data and waveform data stored in the recording unit 30 is generated, and the generated moving image file is stored in the storage unit 40. The storage unit 40 stores an image processing program according to the present embodiment. Details of the image processing program will be described later.

  The waveform collection unit 60 collects waveform data composed of a plurality of evaluation values, and gives the collected waveform data to the recording unit 30. The plurality of evaluation values are values that change according to the state of the subject W in the images of a plurality of frames. The state of the subject W is the position, orientation, brightness, size, shape, color, or the like of the subject W.

  The evaluation value is, for example, an average luminance value of each frame image of the subject W photographed by the camera 2 (an average value of luminances of a plurality of pixels constituting each frame image). In this case, the waveform collection unit 60 sequentially calculates the average luminance value of the image of each frame as the evaluation value based on the image data of each frame sequentially given from the camera 2 to the recording unit 30. The plurality of average brightness values calculated sequentially are stored as waveform data in the waveform data storage area of the recording unit 30. In this case, a plurality of evaluation values are obtained in synchronization with a plurality of frames of image data. Accordingly, the number of evaluation values per unit time is equal to the number of image data of a plurality of frames per unit time. The evaluation value may be the sum of the luminance values of the images of each frame, or may be the difference between the sum of the luminance values of the images of each frame and the sum of the luminance values of the images of the subsequent frames. It may be a variance of luminance values.

  The waveform collection unit 60 may acquire the value of the output signal of the sensor 6 during the acquisition of the image data of a plurality of frames as the evaluation value. In this case, the plurality of evaluation values are obtained at the same period or different periods from the image data of a plurality of frames. Therefore, the number of evaluation values per unit time does not necessarily match the number of image data of a plurality of frames per unit time. When a plurality of evaluation values are obtained with a shorter cycle than image data of a plurality of frames, a plurality of evaluation values correspond to the image data of each frame. When a plurality of evaluation values are obtained with a period longer than that of a plurality of frames of image data, each evaluation value corresponds to a plurality of frames of image data.

  When the sensor 6 is a temperature sensor, a distance sensor, a pressure sensor, an acceleration sensor, a displacement sensor, or the like, a measured value of the measurement object such as temperature, distance, pressure, acceleration, or displacement is used as an evaluation value as a waveform collection unit. The plurality of acquired evaluation values are sequentially acquired by 60, and stored in the waveform data storage area of the recording unit 30 as waveform data.

  Further, when a microphone is used in place of the sensor 6, audio signals are sequentially acquired as evaluation values by the waveform collection unit 60, and the acquired plurality of audio signals are stored as waveform data in the waveform data storage area of the recording unit 30. Remembered.

  In FIG. 1, the waveform collection device 7 connected to the main body 1 has the same configuration as the waveform collection unit 60. Therefore, when the waveform collection device 7 is connected to the main body unit 1, the value of the output signal of the sensor 8 is sequentially acquired as an evaluation value by the waveform collection device 7, and the plurality of acquired evaluation values are stored in the recording unit 30. It is stored as waveform data in the waveform data storage area.

  The control unit 10 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM. The CPU of the control unit 10 controls the operation of other components in the camera 2, the lighting device 3, and the main body unit 1 based on the operation of the operation unit 4 by the user. The CPU of the control unit 10 executes the image processing program stored in the storage unit 40 on the RAM. A system program is stored in the ROM.

(2) Basic Operation of Image Processing System The basic operation of the image processing system 100 in FIG. 1 will be described.

(2-1) Setting of shooting conditions The shooting conditions of the camera 2 are set when the user operates the operation unit 4. FIG. 2 is a diagram illustrating a display example of the display unit 20 when setting photographing conditions. As shown in FIG. 2, when setting the shooting conditions, an image display area Sa, a waveform display area Sb, a condition setting area Sc, and a recording button Se are displayed on the screen of the display unit 20.

  In the image display area Sa, an image based on the image data stored in the video memory 50 is displayed. A waveform graph based on the waveform data collected by the waveform collection unit 60 is displayed in the waveform display area Sb. The horizontal axis of the waveform graph is the time axis, and the vertical axis indicates the evaluation value.

  In the condition setting area Sc, basic shooting conditions of the camera 2 are displayed. As shooting conditions, for example, a plurality of input frames Sc1, Sc2, and Sc3 for setting a shutter speed, a shooting frame rate, and a resolution are displayed. The user inputs the shutter speed, shooting frame rate, and resolution values into the input frames Sc1, Sc2, and Sc3 using the operation unit 4, respectively. Based on the values input to the input frames Sc1, Sc2, and Sc3, the shutter speed, shooting frame rate, and resolution of the camera 2 are set by the control unit 10.

  In the present embodiment, for example, when the user operates the operation unit 4, a trigger signal serving as a reference for the end timing of a recording process described later is given to the control unit 10. In the condition setting area Sc, input frames Sc4 and Sc5 for trigger setting of the control unit 10 based on the trigger signal are further displayed.

  The user inputs values of a total recording time and a recording end time, which will be described later, into the input frames Sc4 and Sc5 using the operation unit 4, respectively. Based on the values respectively input to the input frames Sc4 and Sc5, the trigger setting is performed by the control unit 10. Details of the trigger setting will be described.

  In the present embodiment, the trigger setting determines a storage time (hereinafter referred to as total recording time) of image data stored in the recording unit 30 by one recording process, and a trigger signal is given to the control unit 10. The time from the point in time until the recording process is completed (hereinafter referred to as the recording end time) is determined.

  For example, when the trigger is set with the total recording time set to 30 seconds and the recording end time set to 0 seconds, the recording process ends when a trigger signal is given to the control unit 10 after the recording is started. To do. Thereby, the moving image data and waveform data acquired from 30 seconds before the time when the trigger signal is given to the control unit 10 until the time when the trigger signal is given are stored in the recording unit 30.

  On the other hand, when the trigger setting is performed with the total recording time being 30 seconds and the recording end time being 30 seconds, 30 seconds have elapsed since the trigger signal was given to the control unit 10 after the recording was started. At this point, the recording process ends. Thereby, the video recording unit 30 stores the moving image data and waveform data acquired until 30 seconds have elapsed since the trigger signal was given to the control unit 10.

  In addition, when the trigger setting is performed with the total recording time set to 30 seconds and the recording end time set to 15 seconds, 15 seconds have passed since the trigger signal was given to the control unit 10 after the recording was started. At this point, the recording process ends. As a result, the video recording unit 30 stores the moving image data and waveform data acquired from 15 seconds before the time when the trigger signal is given to the control unit 10 until 30 seconds have passed.

  Instead of setting the total recording time as the trigger setting, the total number of frames of image data stored in the recording unit 30 by the recording process may be set. Further, instead of setting the recording end time as the trigger setting, the number of frames of image data stored in the recording unit 30 from the time when the trigger signal is given may be set.

  A trigger signal may be given from the PLC 5 to the control unit 10. In this case, for example, a trigger signal can be given to the control unit 10 in response to a cycle preset in the PLC 5 or a specific operation timing of an external device connected to the PLC 5. Further, output signals of various sensors may be given to the control unit 10 as trigger signals.

  The recording button Se is operated by the user using the operation unit 4 to start a recording process described later.

(2-2) Recording Process FIG. 3 is a schematic diagram showing a storage state of the recording unit 30 in FIG. 1 during the recording process. As described above, the recording unit 30 includes the moving image data storage area 30A for storing moving image data (image data of a plurality of frames) and the waveform data storage area 30B for storing waveform data (a plurality of evaluation values). Is set.

  In the main body 1, as shown in FIG. 3A, the recording process is started by the control unit 10 when the recording button Se in FIG. 2 is operated. In the recording process, the image data v1 corresponding to the first frame image given from the camera 2 is stored in the moving image data storage area 30A of the recording unit 30. In addition, the first evaluation value i1 given from the waveform collecting unit 60 is stored in the waveform data storage area 30B of the recording unit 30.

  Subsequently, as shown in FIG. 3B, image data v2, v3, and v4 corresponding to the second, third, and fourth frame images given from the camera 2 are stored in the moving image data storage area 30A of the recording unit 30. The image data v1, v2, v3, and v4 are sequentially shifted in the moving image data storage area 30A while being sequentially stored. At the same time, the second, third, and fourth evaluation values i2, i3, i4 given from the waveform collection unit 60 are sequentially stored in the waveform data storage area 30B of the recording unit 30, and the evaluation values i1, i2 , I3, i4 are sequentially shifted in the waveform data storage area 30B.

  If the amount of image data stored in the moving image data storage area 30A during the recording process exceeds the storage capacity of the moving image data storage area 30A, the image data of the latest frame (current frame) is newly added to the moving image data storage area 30A. The image data of the oldest frame stored in the moving image data storage area 30A is deleted every time.

  Similarly, when the amount of evaluation value stored in the waveform data storage area 30B during the recording process exceeds the storage capacity of the waveform data storage area 30B, the latest evaluation value is newly stored in the waveform data storage area 30B. Every time, the past evaluation value stored in the waveform data storage area 30B is deleted.

  In the example of FIG. 3C, in a state where 12 pieces of image data v1 to v12 are stored in the moving image data storage area 30A, two pieces of image data v13 and v14 are sequentially stored. Are sequentially deleted. Further, in the state where 12 evaluation values i1 to i12 are stored in the waveform data storage area 30B, the evaluation values i13 and i14 of two images are sequentially stored, so that the past evaluation values i1 and i1 are stored. i2 is deleted sequentially. Thereby, the image data v3 to v14 are stored in the moving image data storage area 30A, and the evaluation values i3 to i14 are stored in the waveform data storage area 30B.

  As described above, in the moving image data storage area 30A and the waveform data storage area 30B during the recording process, the moving image data and the waveform data are sequentially stored in a so-called ring buffer format.

  In the example of FIG. 3C, the evaluation values i3 to i14 stored in the waveform data storage area 30B have a one-to-one correspondence with the image data v3 to v14 stored in the moving image data storage area 30A. As described above, when a plurality of evaluation values are obtained at a different period from the image data of a plurality of frames, the evaluation value stored in the waveform data storage area 30B is paired with the image data stored in the moving picture data storage area 30A. It does not correspond to one.

  FIG. 4 is a schematic diagram showing another example of the storage state of the recording unit 30 of FIG. 1 during the recording process. In the example of FIG. 4, a plurality of evaluation values are obtained with a shorter cycle than the image data of a plurality of frames. Therefore, the evaluation values i7 to i42 stored in the waveform data storage area 30B do not correspond one-to-one with the image data v3 to v14 stored in the moving image data storage area 30A. Three consecutive evaluation values correspond to one frame of image data. For example, the evaluation values i7, i8, i9 correspond to the image data v3, and the evaluation values i10, i11, i12 correspond to the image data v4.

  When a plurality of evaluation values are obtained with a longer cycle than that of image data of a plurality of frames, each evaluation value corresponds to image data of a plurality of frames.

  FIG. 5 is a diagram illustrating a display example of the display unit 20 during the recording process. As shown in FIG. 5, a trigger button Sf is displayed on the display unit 20 instead of the recording button Se of FIG.

  When the trigger button Sf is operated by the user using the operation unit 4, a trigger signal is given to the control unit 10. Thereby, as described above, the recording process ends at the timing based on the trigger setting.

(2-3) File Storage Process FIG. 6 is a diagram showing a display example of the display unit 20 at the end of the recording process. As shown in FIG. 6, at the end of recording, a file button Sj and a playback button Sg are displayed on the display unit 20 instead of the trigger button Sf of FIG.

  When the user operates the file button Sj using the operation unit 4, the control unit 10 performs file storage processing. In the file storage process, a moving image file including moving image data and waveform data stored in the recording unit 30 is generated, and the generated moving image file is stored in the storage unit 40 of FIG.

  FIG. 7 is a schematic diagram of a moving image file stored in the storage unit 40 by the file storage process.

  For example, the file storage process is performed in a state where the image data v1 to v12 are stored in the moving image data storage area 30A of the recording unit 30 in FIG. 3 and the evaluation values i1 to i12 are stored in the waveform data storage area 30B. In this case, as shown in FIG. 7, a moving image file FA including the image data v1 to v12, the evaluation values i1 to i12, and the file information fi is generated, and the generated moving image file FA is stored in the storage unit 40.

  The file information fi includes, for example, information indicating a file name and a data storage format. By repeating the above recording process and file storage process, a plurality of moving image files FA, FB, FC, and FD are stored in the storage unit 40 of FIG.

(2-4) Reproduction Process Next, the reproduction process will be described. The reproduction process is any of a plurality of frames of image data sequentially given from the camera 2, a plurality of frames of image data stored in the recording unit 30, or a plurality of frames of image data in a moving image file stored in the storage unit 40. It refers to displaying a moving image on the display unit 20 based on continuous image data of a plurality of frames.

  First, image reproduction processing based on image data in a moving image file stored in the storage unit 40 will be described.

  Icons corresponding to the plurality of moving image files stored in the storage unit 40 are displayed on the display unit 20 of FIG. In this case, the user selects one of the plurality of icons displayed on the display unit 20 using the operation unit 4, thereby selecting the selected movie file from the plurality of movie files stored in the storage unit 40. Movie data can be selected as a playback target.

  FIG. 8 is a diagram illustrating a display example of the display unit 20 during the reproduction process. As shown in FIG. 7, for example, when one moving image file is selected by the user using the operation unit 4, the display unit 20 in FIG. A display area Sb, a condition setting area Sc, and a playback button Sg are displayed, and a comparison button Sx is displayed instead of the file button Sj in FIG.

  When the user operates the playback button Sg using the operation unit 4, the control unit 10 starts the playback process. In the reproduction process, the control unit 10 causes the video memory 50 to sequentially store a plurality of frames of image data in the selected moving image file in the storage unit 40. Thereby, an image based on the image data sequentially stored in the video memory 50 is displayed in the image display area Sa. In this manner, a moving image is reproduced based on a plurality of frames of image data in the selected moving image file. Further, the control unit 10 stores the waveform data in the selected moving image file in the storage unit 40 in the video memory 50. Thereby, a waveform graph based on the waveform data is displayed in the waveform display area Sb.

  In this case, the control unit 10 currently has an evaluation value corresponding to the frame of the image currently displayed in the image display area Sa (hereinafter referred to as a corresponding evaluation value) on the waveform graph of the waveform display area Sb. The display position bar L is displayed. When the correspondence evaluation value changes due to a change in the frame of the image displayed in the image display area Sa, the control unit 10 moves the current display position bar L to the left and right on the time axis.

  Next, the reproduction process before the recording process will be described. Prior to the recording process, the control unit 10 sequentially stores image data of a plurality of frames provided from the camera 2 in the video memory 50. Thereby, an image based on the image data of a plurality of frames is displayed in the image display area Sa of FIG. In this way, the moving image of the subject photographed by the camera 2 is reproduced. Further, the control unit 10 causes the video memory 50 to store the waveform data given from the waveform collection unit 60. Thereby, a waveform graph based on the waveform data is displayed in the waveform display area Sb.

  Further, an image reproduction process based on image data stored in the recording unit 30 at the end of the recording process will be described. After the recording process, when the user operates the playback button Sg of FIG. 6 using the operation unit 4, the control unit 10 sequentially stores the image data of a plurality of frames in the recording unit 30 in the video memory 50. Thereby, an image based on the image data sequentially stored in the video memory 50 is displayed in the image display area Sa. In this manner, a moving image is reproduced based on the moving image data stored in the recording unit 30. Further, the control unit 10 stores the waveform data stored in the recording unit 30 in the video memory 50. Thereby, a waveform graph based on the waveform data is displayed in the waveform display area Sb.

  When the comparison button Sx is operated by the user using the operation unit 4, a comparison process described later is started.

(3) Comparison processing (3-1) Overview of comparison processing An overview of comparison processing will be described. In the present embodiment, during the comparison process, a plurality of image display areas and waveform display areas are displayed on the display unit 20 as will be described later. A plurality of moving images are respectively displayed in a plurality of image display areas based on the moving image data of the plurality of moving image files. Also, a plurality of waveform graphs are displayed in a common waveform display area based on the waveform data of the plurality of moving image files. Thereby, the user can compare the graphs of a plurality of moving images and a plurality of waveforms.

  In the comparison process, the user operates the operation unit 4 to select a plurality of moving image files to be compared. An example in which two moving image files are selected as comparison processing targets will be described.

  In the following description, the two moving image files to be compared are called a first file and a second file, respectively. A moving image based on the moving image data of the first file is called a first moving image, and a moving image based on the moving image data of the second file is called a second moving image. Furthermore, the waveform graph displayed in the waveform display area based on the waveform data of the first file is called a first waveform, and the waveform graph displayed in the waveform display area Sb based on the waveform data of the second file is the first waveform. Two waveforms are called.

  Note that the first file is a moving image file selected as a reproduction target during the reproduction process described with reference to FIG. 8, for example. Therefore, in this example, the first file is selected by the user in advance before the comparison process.

  On the other hand, the second file is, for example, a moving image file that is selected as a comparison process target by the user at the start of the comparison process. When the comparison process is started by operating the comparison button Sx in FIG. 8, icons corresponding to the plurality of moving image files stored in the storage unit 40 are displayed on the display unit 20 in FIG. 1. The user can select a desired moving image file as the second file by selecting any of a plurality of icons displayed on the display unit 20 using the operation unit 4.

  FIG. 9 is a diagram illustrating a display example of the display unit 20 at the time of comparison processing. FIG. 10 illustrates a plurality of frame image data, a plurality of frame images, and a plurality of evaluation values included in the first file and the second file. It is a conceptual diagram which shows an example of a relationship. FIG. 11 is a graph in which a part of the first waveform and the second waveform displayed in the waveform display area Sb of FIG. 9 is enlarged. In FIG. 11, the graph of the first waveform is shown in the upper stage, and the graph of the second waveform is shown in the lower stage. Furthermore, a plurality of image data corresponding to a plurality of evaluation values are shown on each graph in FIG.

  By selecting the first file and the second file, as shown in FIG. 9, two image display areas Sa1 and Sa2, a waveform display area Sb, a playback button Tc, and a pause are displayed on the screen of the display unit 20. A button Td, a reverse playback button Te, two seek bars Tf1, Tf2, a synchronization button Tm, a comparison end button Tn, and two operation target designation buttons Tp1, Tp2 are displayed. The two seek bars Tf1 and Tf2 respectively have sliders Tg1 and Tg2 that can move in the horizontal direction.

  An image based on the image data of each frame of the first file is displayed in one image display area Sa1, and an image based on the image data of each frame of the second file is displayed in the other image display area Sa2. A graph of the first waveform WL1 based on the waveform data of the first file and the second waveform WL2 based on the waveform data of the second file is displayed in the common waveform display area Sb. The horizontal axis of the waveform graph is the time axis, and the vertical axis indicates the evaluation value.

  On the graph of the waveform display region Sb, a current display position bar L common to the first waveform WL1 and the second waveform WL2 is displayed. The common current display position bar L intersects the first waveform WL1 and the second waveform WL2.

  During the comparison process, the correspondence evaluation value of the first file corresponding to the frame of the image currently displayed in the image display area Sa1 and the correspondence of the second file corresponding to the frame of the image currently displayed in the image display area Sa2 The evaluation value is designated by the current display position bar L.

  In the upper part of FIG. 10, images VA1 to VA7 corresponding to the image data va1 to va7 of the first to seventh frames included in the first file are sequentially shown along the time axis from left to right. The first to seventh evaluation values ia1 to ia7 are associated with the image data va1 to va7 of the first to seventh frames, respectively. The images VA1 to VA7 in this example are obtained by photographing the subject W1 that falls toward the floor bt.

  In the lower part of FIG. 10, images VB1 to VB7 corresponding to the image data vb1 to vb7 of the first to seventh frames included in the second file are sequentially shown along the time axis from left to right. The first to seventh evaluation values ib1 to ib7 are associated with the image data vb1 to vb7 of the first to seventh frames, respectively. The images VB1 to VB7 in this example are obtained by photographing the subject W2 falling toward the floor bt.

  In the initial state, the first to seventh frame image data va1 to va7 of the first file and the first to seventh frame image data vb1 to vb7 of the second file correspond to each other on the time axis. Similarly, in the initial state, the first to seventh evaluation values ib1 to ib7 of the first file and the first to seventh evaluation values ib1 to ib7 of the second file correspond to each other on the time axis. ing.

  In the example of FIG. 9, the current display position bar L is displayed at the position of the time point t3 on the time axis of the waveform display area Sb. In this case, as shown in FIG. 11, the correspondence evaluation value of the first file is the evaluation value ia3. As a result, the image VA3 (FIG. 10) based on the image data of the third frame corresponding to the evaluation value ia3 of the first file is displayed in the image display area Sa1 of FIG.

  Moreover, as shown in FIG. 11, the corresponding evaluation value of the second file is the evaluation value ib3. As a result, the image VB3 (FIG. 10) based on the image data of the third frame corresponding to the evaluation value ib3 of the second file is displayed in the image display area Sa2 of FIG.

  Here, as shown in the upper part of FIG. 10, the image VA3 at the moment when the subject W1 collides with the floor bt corresponds to the image data va3 of the third frame of the first file. On the other hand, as shown in the lower part of FIG. 10, the image VB5 at the moment when the subject W2 collides with the floor bt corresponds to the image data va5 of the fifth frame of the second file.

  In this case, even if the first moving image based on the image data va1 to va7 of the first to seventh frames and the second moving image based on the image data vb1 to vb7 of the first to seventh frames are simultaneously reproduced at the same frame rate, The image VA3 at the moment when the subject W1 collides with the floor bt and the image VB5 at the moment when the subject W2 collides with the floor bt are not simultaneously displayed in the image display areas Sa1 and Sa2 of the display unit 20.

  Therefore, the correspondence between the frames of the images displayed simultaneously in the two image display areas Sa1 and Sa2 is changed. That is, the correspondence on the time axis between the image data va1 to va7 of the first to seventh frames of the first file and the image data vb1 to vb7 of the first to seventh frames of the second file is changed.

  For example, the display timings of the images VA1 to VA7 are delayed by two frames with respect to the display timings of the images VB1 to VB7. Thus, the image VA3 can be displayed in the image display area Sa1, and the image VB5 can be displayed in the image display area Sa2.

(3-2) First Operation Example of Display Unit During Comparison Processing FIG. 12 is a diagram showing a first operation example of the display unit 20 during comparison processing, and FIG. 13 shows the waveform display area Sb of FIG. It is the graph which expanded each of the displayed 1st waveform and 2nd waveform. In FIG. 13, the graph of the first waveform is shown in the upper stage, and the graph of the second waveform is shown in the lower stage. Furthermore, a plurality of image data corresponding to a plurality of evaluation values are shown on each graph in FIG.

  The two operation target designation buttons Tp1 and Tp2 in FIG. 12 correspond to the first file and the second file, respectively. During the comparison process, the user selects the operation target designation button Tp1 by using the operation unit 4 to operate the play button Tc, the pause button Td, the reverse play button Te, and the slider Tg1 for the first file. Becomes effective. On the other hand, when the operation target designation button Tp2 is selected by the user using the operation unit 4, the operations of the play button Tc, the pause button Td, the reverse play button Te, and the slider Tg2 become effective for the second file. .

  The two seek bars Tf1 and Tf2 correspond to the first file and the second file, respectively. As described above, the two seek bars Tf1 and Tf2 have the sliders Tg1 and Tg2 that can move in the horizontal direction, respectively. In the selected state of the operation target designation button Tp1, the position of the slider Tg1 corresponds to the position on the time axis of the current display position bar L currently displayed in the waveform display area Sb. In the selected state of the operation target designation button Tp2, the position of the slider Tg2 corresponds to the position on the time axis of the current display position bar L currently displayed in the waveform display area Sb.

  In the selected state of the operation target designation button Tp1, the slider Tg1 moves in one direction on the seek bar Tf1 (in this example, from the left to the right of the screen of the display unit 20) as the reproduction of the first moving image proceeds. As reverse playback of the first moving image proceeds, the slider Tg1 moves in the other direction (in this example, from the right to the left of the screen of the display unit 20) on the seek bar Tf1. Details of the reverse reproduction will be described later.

  Similarly, in the selected state of the operation target designation button Tp2, the slider Tg2 moves in one direction on the seek bar Tf2 (in this example, from the left to the right of the screen of the display unit 20) as the reproduction of the second moving image proceeds. As reverse playback of the second moving image proceeds, the slider Tg2 moves in the other direction (in this example, from the right to the left of the screen of the display unit 20) on the seek bar Tf2.

  Here, the user moves the slider Tg1 of the seek bar Tf1 using the operation unit 4 in the selected state of the operation target designation button Tp1, thereby displaying the first waveform WL1 currently displayed in the waveform display region Sb as the current display position. Along with the bar L, it can be moved relative to the second waveform WL2 on the time axis.

  In addition, the user moves the slider Tg2 of the seek bar Tf2 using the operation unit 4 in the selected state of the operation target designation button Tp2, thereby displaying the second waveform WL2 currently displayed in the waveform display area Sb as the current display position bar. Along with L, it can be moved relative to the first waveform WL1 on the time axis.

After the operation target designation button Tp1 is selected from the display state of the display unit 20 in FIG. 9, the slider Tg1 is displayed on the seek bar Tf1 in one direction (display) using the operation unit 4 as shown by a thick arrow in FIG. The screen is moved from left to right on the screen of the unit 20.

  In this case, as indicated by a dotted arrow in FIG. 12, in the waveform display region Sb, the first waveform WL1 and the current display position bar L move relative to the second waveform WL2 on the time axis as the slider Tg1 moves. To do.

  In the example of FIG. 12, the current display position bar L moves from the position of the time point t3 to the position of the time point t5 on the time axis of the waveform display area Sb. At this time, since the first waveform WL1 moves together with the current display position bar L, the corresponding evaluation value of the first file is maintained at the evaluation value ia3 as shown in FIG. As a result, the image VA3 (FIG. 10) based on the image data va3 of the third frame corresponding to the evaluation value ia3 of the first file is displayed in the image display area Sa1 of FIG.

  On the other hand, since the second waveform WL2 does not move, the corresponding evaluation value of the second file changes from the evaluation value ib3 to the evaluation value ib5 as shown in FIG. Thereby, the image VB5 (FIG. 10) based on the image data vb5 of the fifth frame corresponding to the evaluation value ib5 of the second file is displayed in the image display area Sa2 of FIG.

  As described above, when both the first waveform WL1 and the current display position bar L move on the time axis in the waveform display area Sb, the frame of the image displayed in the image display area Sa1 does not change from the state of FIG. On the other hand, when the second waveform WL2 does not move on the time axis in the waveform display region Sb and the current display position bar L moves, the frame of the image displayed in the image display region Sa2 changes from the state of FIG.

  As a result, the correspondence between the frames of the images displayed simultaneously in the two image display areas Sa1 and Sa2 is changed. That is, the correspondence on the time axis between the image data va1 to va7 of the first file and the image data vb1 to vb7 of the second file is changed from the state of FIG. 11 to the state of FIG.

  In this way, the user can change the correspondence between the frames of the images displayed in the image display areas Sa1 and Sa2 by operating the operation target designation button Tp1 and the slider Tg1 using the operation unit 4.

  Further, the user can change the correspondence between the frames of the images displayed in the image display areas Sa1 and Sa2 by operating the operation target specifying button Tp2 and the slider Tg2 using the operation unit 4.

  In this example, after the user operates the operation target designation button Tp1 using the operation unit 4, the user moves the slider Tg1 so that the first waveform WL1 displayed in the waveform display area Sb overlaps the second waveform WL2. Let Thus, the time between the image data va1 to va7 of the first file and the image data vb1 to vb7 of the second file so that the image VA3 is displayed in the image display area Sa1 and the image VB5 is displayed in the image display area Sa2. Correspondence on the axis is changed.

  Thereafter, the synchronization button Tm is operated. As a result, images of a plurality of frames after the image VA3 currently displayed based on the moving image data of the first file are sequentially displayed in the image display area Sa1, and are displayed at the current time based on the moving image data of the second file. The images of a plurality of frames after the displayed image VB5 are sequentially displayed in the image display area Sa2. Thereby, the first moving image and the second moving image are reproduced simultaneously. Hereinafter, the simultaneous reproduction of the first moving image and the second moving image with the same or similar portions of the first waveform WL1 and the second waveform WL2 overlapped is referred to as synchronous reproduction. According to the synchronized playback, the state of the subject W1 in the first moving image and the state of the subject W2 in the second moving image change at substantially the same timing.

  When the reverse playback button Te is operated by the user using the operation unit 4 during the synchronous playback, the first moving image and the second moving image are simultaneously reverse played. When the user operates the pause button Td using the operation unit 4 during the synchronized playback, the playback of the first moving image and the second moving image is temporarily stopped. Further, when the user continues to operate the playback button Tc, the first moving image and the second moving image are played back at high speed until the operation is released. By continuing to operate the reverse playback button Te by the user, the first moving image and the second moving image are reversely played back at high speed until the operation is canceled. During synchronized playback, the current display position bar L moves on the time axis relative to the first waveform WL1 and the second waveform WL2 in the waveform display area Sb.

(3-3) Second Operation Example of Display Unit During Comparison Processing FIG. 14 is a diagram showing a second operation example of the display unit 20 during comparison processing, and FIG. 15 shows the waveform display region Sb of FIG. It is the graph which expanded each of the displayed 1st waveform and 2nd waveform. In FIG. 15, a graph of the first waveform is shown in the upper stage, and a graph of the second waveform is shown in the lower stage. Further, a plurality of image data corresponding to a plurality of evaluation values are shown on each graph in FIG.

  The reproduction button Tc is operated by the user using the operation unit 4 in the selected state of the operation target designation button Tp1 in FIG. In this case, based on the moving image data of the first file, images of a plurality of frames after the currently displayed image are sequentially displayed in the image display area Sa1. Thereby, the first moving image is reproduced. The reverse playback button Te is operated by the user using the operation unit 4. In this case, based on the moving image data of the first file, a plurality of frames of images are sequentially displayed in the image display area Sa1 in the direction opposite to the playback direction on the time axis from the currently displayed image. Thereby, the first moving image is reversely reproduced.

  When the user presses the pause button Td using the operation unit 4 during playback or reverse playback of the first video, playback or reverse playback of the first video is paused. Further, when the user continues to operate the playback button Tc, the first moving image is played back at high speed until the operation is released. When the user continuously operates the reverse playback button Te, the first moving image is reversely played back at high speed until the operation is released.

  When the first moving image is played back or played back in the selected state of the operation target designation button Tp1, the current display position bar L and the second waveform WL2 are timed relative to the first waveform WL1 in the waveform display area Sb. Move on the axis. In this case, the image displayed in the image display area Sa2 does not change.

  On the other hand, the reproduction button Tc is operated by the user using the operation unit 4 in the selected state of the operation target designation button Tp2 in FIG. In this case, based on the moving image data of the second file, images of a plurality of frames after the currently displayed image are sequentially displayed in the image display area Sa2. Thereby, the second moving image is reproduced. The reverse playback button Te is operated by the user using the operation unit 4. In this case, based on the moving image data of the second file, images of a plurality of frames are sequentially displayed in the image display area Sa2 in the direction opposite to that during reproduction on the time axis from the currently displayed image. Thereby, the second moving image is reversely reproduced.

  When the user operates the pause button Td using the operation unit 4 during playback or reverse playback of the second video, playback or reverse playback of the second video is paused. In addition, when the user continues to operate the playback button Tc, the second moving image is played back at high speed until the operation is released. By continuing to operate the reverse playback button Te by the user, the second moving image is reversely played back at high speed until the operation is released.

  When the second moving image is played back or reversely played in the selected state of the operation target designation button Tp2, the current display position bar L and the first waveform WL1 are timed relative to the second waveform WL2 in the waveform display area Sb. Move on the axis. In this case, the image displayed in the image display area Sa1 does not change.

After the operation target designation button Tp1 is selected from the display state of the display unit 20 in FIG. 9, the reverse playback button Te is operated by the user using the operation unit 4 as shown by a thick arrow in FIG.

  In this case, as indicated by a dotted arrow in FIG. 14, in the waveform display region Sb, the second waveform WL2 and the current display position bar L move relative to the first waveform WL1 on the time axis.

  In the example of FIG. 14, the current display position bar L moves from the position of the time point t3 to the position of the time point t1 on the time axis of the waveform display area Sb. At this time, since the second waveform WL2 moves together with the current display position bar L, the corresponding evaluation value of the second file is maintained at the evaluation value ib3 as shown in FIG. Accordingly, the image VB3 (FIG. 10) based on the third frame image data vb3 corresponding to the evaluation value ib3 of the second file is displayed in the image display area Sa2 of FIG.

  On the other hand, since the first waveform WL1 does not move, the corresponding evaluation value of the first file changes from the evaluation value ib3 to the evaluation value ib1 as shown in FIG. As a result, the image VA1 (FIG. 10) based on the image data va1 of the first frame corresponding to the evaluation value ia1 of the first file is displayed in the image display area Sa1 of FIG.

  As described above, when both the second waveform WL2 and the current display position bar L move on the time axis in the waveform display area Sb, the frame of the image displayed in the image display area Sa2 does not change from the state of FIG. On the other hand, when the first waveform WL1 does not move on the time axis in the waveform display area Sb and the current display position bar L moves, the frame of the image displayed in the image display area Sa1 changes from the state of FIG.

  As a result, the correspondence between the frames of the images displayed simultaneously in the two image display areas Sa1 and Sa2 is changed. That is, the correspondence on the time axis between the image data va1 to va7 of the first file and the image data vb1 to vb7 of the second file is changed from the state of FIG. 11 to the state of FIG.

  In this way, the user operates the operation target designation button Tp1 using the operation unit 4 and also operates the playback button Tc or the reverse playback button Te, whereby the frames of the images displayed in the image display areas Sa1 and Sa2 respectively. Correspondence can be changed.

  In addition, the user operates the operation target designation button Tp2 using the operation unit 4 and also operates the playback button Tc or the reverse playback button Te, whereby the correspondence between the frames of the images respectively displayed in the image display areas Sa1 and Sa2. Can be changed.

  In this example, after the user operates the operation target specifying button Tp1 using the operation unit 4, the user operates the reverse playback button Te so that the second waveform WL2 displayed in the waveform display area Sb overlaps the first waveform WL1. To do. Accordingly, the time between the image data va1 to va7 of the first file and the image data vb1 to vb7 of the second file so that the image VA1 is displayed in the image display area Sa1 and the image VB3 is displayed in the image display area Sa2. Correspondence on the axis is changed.

  Thereafter, the synchronization button Tm is operated. As a result, images of a plurality of frames after the image VA1 currently displayed based on the moving image data of the first file are sequentially displayed in the image display area Sa1, and displayed at the current time based on the moving image data of the second file. Images of a plurality of frames after the image VB3 being displayed are sequentially displayed in the image display area Sa2. Accordingly, the first moving image and the second moving image are simultaneously reproduced in a state where the same or similar portions of the first waveform WL1 and the second waveform WL2 overlap each other. That is, the first moving image and the second moving image are synchronously reproduced. By the synchronized playback, the state of the subject W1 in the first moving image and the state of the subject W2 in the second moving image change at substantially the same timing.

(3-4) Comparison Processing Flow FIGS. 16 to 19 are flowcharts of the comparison processing. As described above, this comparison process is started when the user operates the comparison button Sx in FIG. 8 using the operation unit 4 during the reproduction process. In the present embodiment, the comparison process constitutes at least a part of the process executed by the image processing program stored in the storage unit 40.

  When the comparison process is started, the control unit 10 in FIG. 1 determines whether or not a plurality of moving image files to be compared is selected (step S11). As described above, in this example, comparison processing is performed on two moving image files. In this case, for example, the control unit 10 determines whether or not the first file and the second file are selected.

  When the first file and the second file are selected, the control unit 10 displays a graph of the first waveform WL1 (FIG. 9) and the second waveform WL2 (FIG. 9) based on the waveform data of the first file and the second file. Is displayed in the waveform display area Sb (FIG. 9) of the display unit 20 (step S12).

  Further, the control unit 10 displays the current display position bar L at the top position on the time axis on the waveform graph in the waveform display area Sb (step S13).

  In addition, the control unit 10 displays an image based on the image data of the first frame of the first file (the image of the frame corresponding to the position of the first waveform WL1 specified by the current display position bar L) in one image display area Sa1. (FIG. 9) is displayed (step S14). Further, the control unit 10 displays an image based on the image data of the first frame of the second file (an image of a frame corresponding to the position of the second waveform WL2 specified by the current display position bar L) in the other image display area Sa2. (FIG. 9) is displayed (step S15).

  Next, the control unit 10 determines whether or not the end of the comparison process is instructed by operating the comparison end button Tn in FIG. 9 (step S16). When the end of the comparison process is commanded, the control unit 10 ends the comparison process.

  When the end of the comparison process is not instructed, the control unit 10 determines whether or not the synchronous reproduction is instructed by operating the synchronization button Tm in FIG. 9 (step S21).

  When the synchronized playback is not instructed, the control unit 10 determines which of the first and second files is selected as the operation target by operating the operation target specifying button Tp1 or the operation target specifying button Tp2 in FIG. S22).

  In the following description, the moving image file selected in step S22 is called an operation file, and the moving image file not selected in step S22 is called a non-operation file. An image corresponding to the image data of the operation file is called an operation image, and a waveform corresponding to the waveform data of the operation file is called an operation waveform. Furthermore, an image corresponding to the image data of the non-operation file is called a non-operation image, and a waveform corresponding to the waveform data of the non-operation file is called a non-operation waveform. An image display area in which an operation image is displayed is referred to as an operation image area, and an image display area in which a non-operation image is displayed is referred to as a non-operation image area. Furthermore, the slider corresponding to the operation file is called an operation slider, and the slider corresponding to the non-operation file is called a non-operation slider.

In step S22, when the operation target is not selected, that is, when neither of the operation target designation buttons Tp1 and Tp2 in FIG. 9 is selected, the control unit 10 returns to the process of step S16.

  On the other hand, when the operation target is selected, the control unit 10 determines whether or not the operation slider has been moved by the user using the operation unit 4 (step S23).

  When the operation slider is not moved, the control unit 10 proceeds to the process of step S26 described later.

  When the operation slider is moved, the control unit 10 moves the current display position bar L and the operation waveform on the time axis of the waveform display area Sb according to the movement amount of the operation slider (step S24). Further, the control unit 10 causes the non-operation image of the frame corresponding to the position of the non-operation waveform designated by the current display position bar L to be displayed in the non-operation target image area (step S25).

  Subsequently, the control unit 10 determines whether or not a moving image reproduction or reverse reproduction is instructed by an operation of the reproduction button Tc or the reverse reproduction button Te in FIG. 9 (step S26). When the reproduction or reverse reproduction of the moving image is not instructed, the control unit 10 returns to the process of step S16 described above.

  When the reproduction or reverse reproduction of the moving image is instructed, the control unit 10 reproduces or reversely reproduces the moving image based on the moving image data of the operation file (step S27). Further, the control unit 10 moves the current display position bar L and the non-operation waveform on the time axis of the waveform display area Sb according to the frame of the operation image displayed in the operation image area by playing back or playing back a moving image ( Step S28).

  In this state, the control unit 10 determines whether or not stop of playback or reverse playback is instructed by operating the pause button Td in FIG. 9 (step S29). When stop of playback or reverse playback is instructed, the control unit 10 stops playback or reverse playback of the moving image based on the moving image data of the operation file, and returns to the process of step S16.

  When the synchronous reproduction is instructed in step S21, the control unit 10 moves the current display position bar L on the time axis of the waveform display region Sb (step S31). In addition, the control unit 10 sequentially displays a plurality of images based on the image data of a plurality of frames after the currently displayed image of the first file in one image display area Sa1 (step S32). Further, the control unit 10 sequentially displays a plurality of images based on the image data of a plurality of frames after the currently displayed image of the second file in the other image display area Sa2 (step S33).

  Thereafter, the control unit 10 determines whether or not the stop of the synchronous reproduction is instructed by the operation of the pause button Td in FIG. 9 (Step S34). When the stop of the synchronous reproduction is instructed, the control unit 10 stops the processes of steps S31 to S33 and returns to the process of step S16.

  In step S34, the control unit 10 determines whether the last frame of the first file and the second file is placed in the one and the other image display areas Sa1 and Sa2, respectively, instead of determining whether or not the stop of the synchronous playback is instructed. It may be determined whether an image based on the image data is displayed. In this case, the control unit 10 may return to the process of step S16 by displaying the image of the final frame in one and the other image display areas Sa1 and Sa2.

(3-5) Effect As described above, the plurality of evaluation values constituting the waveform data are values that change according to the state of the subject W in the images of the plurality of frames. When the first waveform WL1 and the second waveform WL2 have a portion having the same or similar shape, the subject W1 and the second waveform WL2 in the image corresponding to the same or similar shape portion of the first waveform WL1 are the same or The subject W2 in the image of the frame corresponding to the part having the similar shape is in almost the same state.

  The user can move one of the first waveform WL1 and the second waveform WL2 displayed in the waveform display area Sb by operating the operation unit 4 on the time axis relative to the other waveform. . Thereby, the correspondence between the frame of the image displayed in the image display area Sa1 and the frame of the image displayed in the image display area Sa2 can be changed.

  Therefore, the user relatively moves the same or similar portions of the first waveform WL1 and the second waveform WL2 to display two images including almost the same state of the subjects W1 and W2 in the two image display areas Sa1. , Sa2 can be displayed respectively. Thereby, the first moving image and the second moving image can be reproduced so that the state of the subject W1 and the state of the subject W2 change at the same timing.

As a result, the user can easily compare the states of the subjects W1 and W2 in the plurality of moving images with a simple configuration.

(4) Other Embodiments (4-1) In the above example, the number of moving image files to be compared is two. However, the number of moving image files to be compared is not limited to this, and may be three, or may be four.

  For example, when the number of moving image files to be compared is three, three image display areas are displayed on the display unit 20, and the three image display areas are based on moving image data of three moving image files, respectively. The image of each frame is displayed. In addition, three waveforms based on the waveform data of the three moving image files are displayed in the waveform display area Sb.

  (4-2) In the above, a plurality of images are compared based on moving image data of a plurality of moving image files by comparison processing, and a plurality of waveforms based on waveform data of the plurality of moving image files are compared. Not limited to this, a plurality of frames of images and a plurality of waveforms based on moving image data and waveform data of a common moving image file may be compared with each other.

  For example, the control unit 10 determines that the same movie file is selected as the first file and the second file by selecting the same movie file in the above-described step S11. May be. Thereby, the user can compare a plurality of portions of the waveform based on the waveform data of one moving image file with each other. In addition, a plurality of moving image portions based on moving image data of one moving image file can be synchronously reproduced.

  More specifically, for example, in one moving image file including the image data of the first to 100th frames, the user and a waveform including a plurality of evaluation values corresponding to the image data of the first to 50th frames and the 50th to 100th frames. A waveform composed of a plurality of evaluation values corresponding to 100 frames of image data can be compared. Further, the user can synchronously reproduce the moving image based on the image data of the first to 50th frames and the moving image based on the image data of the 50th to 100th frames.

  (4-3) During the comparison process, as shown in FIG. 9, two seek bars Tf1 and Tf2 corresponding to the first file and the second file are displayed on the display unit 20, respectively. Not only this but one seek bar may be displayed on the display part 20. In this case, for example, when the operation target designation button Tp1 of FIG. 9 is selected, the slider Tg1 corresponding to the first file is displayed on the seek bar, and when the operation target designation button Tp2 is selected, the slider corresponds to the second file. The slider Tg2 may be displayed on the seek bar. Furthermore, when the synchronization button Tm in FIG. 9 is operated, a slider common to the first file and the second file may be displayed on the seek bar.

  In addition to the above, the display unit 20 may display one seek bar having a plurality of sliders Tg1 and Tg2 respectively corresponding to the first file and the second file. By displaying only one seek bar on the display unit 20, the image display area on the display unit 20 can be enlarged.

  (4-4) In the example of FIG. 9, the first waveform WL1 and the second waveform WL2 are indicated by a solid line and a dotted line, respectively, in the waveform display region Sb of the display unit 20 during the comparison process. Not limited to this, either the first waveform WL1 or the second waveform WL2 may be highlighted (highlighted display, etc.).

  In this case, the waveform corresponding to the moving image file selected in step S22 is highlighted, so that the user can easily recognize the operation target file and the operation target waveform.

  (4-5) In the example of FIG. 9, the first waveform and the second waveform are displayed in the waveform display area Sb of the display unit 20 so as to overlap each other. Not limited to this, two waveform display areas Sb respectively corresponding to the first waveform WL1 and the second waveform WL2 may be displayed on the display unit 20 in multiple stages. In this case, the first waveform WL1 and the second waveform WL2 are displayed in the two waveform display areas Sb, respectively, so that the user can clearly recognize the first waveform WL1 and the second waveform WL2.

  (4-6) In step S22 described above, the control unit 10 highlights the seek bar and the slider corresponding to the operation target file when one of the first and second files is selected as the operation target. The seek bar and the slider corresponding to the non-operation target file may be displayed in gray scale or a broken line. In this case, the user can easily recognize the operation target file and the non-operation target file.

  (4-7) As described above, the evaluation value may be the average luminance value of the image of each frame, may be the sum of the luminance values of the image of each frame, or the luminance value of the image of a certain frame. May be the difference between the brightness value of the image in each frame and the variance of the brightness value in each frame. Further, when the sensor 6 is a temperature sensor, a distance sensor, a pressure sensor, an acceleration sensor, a displacement sensor, or the like, the evaluation value is a measured value such as the temperature, distance, pressure, acceleration, or displacement of the measurement object. May be.

  One moving image file may include a plurality of waveform data different from each other. For example, one moving image file may include waveform data composed of average luminance values of images of a plurality of frames and waveform data composed of a plurality of measurement values measured by the sensor 6 at a predetermined cycle.

  When the first file and the second file described above include two different waveform data, the two waveforms and the second waveform based on the two waveform data of the first file are displayed in the graph of the waveform display area Sb of FIG. Two waveforms based on the two waveform data of the file may be superimposed and displayed.

  Alternatively, two waveforms based on one waveform data of the first file and the second file and two waveforms based on the other waveform data of the first file and the second file are based on the operation of the operation unit 4 by the user. May be selectively displayed in the waveform display area Sb.

  In these cases, the user can recognize various information regarding the first moving image and the second moving image by visually recognizing the plurality of waveforms displayed in the waveform display region Sb of the display unit 20.

  (4-8) A new moving image file including moving image data and waveform data of the first file and moving image data and waveform data of the second file by performing synchronous reproduction of the first file and the second file during the comparison process. May be generated as a comparative moving image file and stored in the storage unit 40.

  At the time of generating the comparison video file, for example, based on the relationship between two images displayed simultaneously in the image display areas Sa1 and Sa2 in FIG. A plurality of frame image data may be associated with each other, and a plurality of evaluation values of the first file may be associated with a plurality of evaluation values of the second file. In this case, when the comparison moving image file is read by the control unit 10, the synchronous reproduction state during the comparison process is easily reproduced on the screen of the display unit 20.

  (4-9) The control unit 10 selects the operation target waveform or non-operation based on the operation of the operation unit 4 by the user after the first file and the second file are selected as the operation target by the process of step S22 of FIG. The waveform is moved on the time axis of the graph in the waveform display area Sb.

  Not limited to this, the control unit 10 starts each comparison value, and thereby each evaluation value corresponding to the image data of each frame of the first file and each evaluation value corresponding to the image data of each frame of the second file. The correlation value or the difference value between the first waveform and the second waveform may be relatively moved on the time axis of the graph of the waveform display region Sb based on the calculated plurality of correlation values or difference values. Good.

  In this case, when the part of the first waveform similar to the part of the first waveform and the part of the second waveform exist at different positions on the time axis, the first waveform and the second waveform on the time axis are overlapped so that the parts similar to each other overlap. The two waveforms are moved relative to each other. In this case, the current display position bar L may move on the time axis together with one of the first waveform and the second waveform, or may stop on the time axis.

  Thus, by performing the comparison process, the first waveform and the second waveform are relatively moved on the time axis of the waveform display region Sb. As a result, the correspondence between the image frame displayed in the image display area Sa1 and the image frame displayed in the image display area Sa2 is easily changed.

  (4-10) During the comparison process, a process (hereinafter referred to as a tracking process) for displaying the locus of the tracking target moving in the moving image by tracking the subject in the moving image may be performed. In order to perform the tracking process, the user performs tracking setting using the operation unit 4 in advance.

  At the time of tracking setting, for example, the user uses the operation unit 4 to display a template as a tracking target image and a range to search for a tracking target in an image of the next frame on an image displayed in one of the image display areas. The search range and the symbol indicating the position where the tracking target exists are specified. By reproducing the moving image including the image for which the tracking setting has been performed in this manner, the position of the tracking target is identified in each of a plurality of frame images constituting the moving image.

  As described above, the first moving image and the second moving image are respectively reproduced in the two image display areas Sa1 and Sa2 during the comparison process. Therefore, when the subject W1 is tracked in the first moving image and at the same time the subject W2 is tracked in the second moving image, the user applies the same to the two images displayed in the two image display areas Sa1 and Sa2. It is necessary to set each tracking setting.

  Therefore, when the tracking process is performed for a plurality of moving images during the comparison process, the tracking setting is performed on the image displayed in one of the two image display areas Sa1 and Sa2, and the other is performed. The tracking setting having the same content as the tracking setting for one image display area may be reflected in the image displayed in the image display area. This eliminates the need for the user to repeatedly perform the same tracking setting during the tracking process during the comparison process.

  (4-11) During the comparison process, the shift amount between the position of the specific portion of the subject W1 in the first moving image and the position of the specific portion of the subject W2 in the second moving image is calculated, and the calculation result is displayed. Processing (hereinafter referred to as measurement processing) may be performed.

  FIG. 20 is a diagram illustrating a display example of the display unit 20 during the measurement process. During the measurement process, the user designates a desired position in one image display area Sa1 (in this example, a corner portion of the subject W1 of the image VA1) using the operation unit 4. As a result, the symbol m1 is displayed at the designated position on the image VA1 in the image display area Sa1. At this time, in the other image display area Sa2, the symbol m2 is displayed at a position corresponding to the position of the symbol m1 in the image display area Sa1.

  Subsequently, the user designates a desired position in the other image display area Sa2 (in this example, a corner portion of the subject W2 of the image VB3) using the operation unit 4. As a result, the symbol n1 is displayed at the designated position on the image VB3 in the image display area Sa2. At this time, in one image display area Sa1, the symbol n2 is displayed at a position corresponding to the position of the symbol n1 in the image display area Sa2.

  In this state, the subject is based on the distance between the two symbols m1 and n2 displayed in one image display area Sa1 (or the distance between the two symbols n1 and m2 displayed in the other image display area Sa2). An actual deviation amount Q of the subject W2 with respect to W1 is calculated by the control unit 10. The calculation result is displayed in each of the two image display areas Sa1 and Sa2.

  Thereby, the user can easily measure the amount of deviation between the position of the specific portion of the subject W1 in the first moving image and the position of the specific portion of the subject W2 in the second moving image.

  (4-12) The shooting frame rate of the subject W1 of the first moving image may be different from the shooting frame rate of the subject W2 of the second moving image. In this case, the playback frame rate of at least one of the first moving image and the second moving image may be adjusted so that images of the frames of the subjects W1 and W2 acquired at substantially the same shooting timing are displayed simultaneously. . The playback frame rate is the number of frames of an image displayed on the display unit 20 per unit time.

  For example, when the shooting frame rates of the subject W1 and the subject W2 are 1000 fps and 250 fps, respectively, the playback frame rates of the first moving image and the second moving image are adjusted to 60 fps and 15 fps, respectively, and synchronized playback is performed. Thereby, in the image display areas Sa1 and Sa2 of the display unit 20, the images of the frames of the subjects W1 and W2 acquired at substantially the same shooting timing are displayed. Thereby, the user can accurately compare changes in the states of the objects W1 and W2 in the first moving image and the second moving image.

  There are cases where the sampling rate of the plurality of evaluation values included in the first file and the sampling rate of the plurality of evaluation values included in the second file are different from each other. The sampling rate of evaluation values is the number of evaluation values acquired by the waveform collection device 7 per unit time. At the time of synchronous reproduction, the control unit 10 sequentially reads a plurality of evaluation values included in the first file and a plurality of evaluation values included in the second file at regular intervals. As a result, the current display position bar L is moved on the time axis in the waveform display area Sb.

  Even in this case, the first file acquired by the control unit 10 at substantially the same sampling timing as the reading cycle of the plurality of evaluation values of the first file and the reading cycle of the plurality of evaluation values of the second file at the time of synchronous playback. The plurality of evaluation values of the second file may be read at the same time. Thereby, the user can accurately compare changes in the states of the objects W1 and W2 in the first moving image and the second moving image.

(5) Correspondence between each component of claim and each part of embodiment The following describes an example of the correspondence between each component of the claim and each part of the embodiment. It is not limited.

  In the above embodiment, the subject W1 is an example of the first object, the images of the subject W1 displayed in the images VA1 to VA7 and the image display area Sa1 are examples of the first image, and the subject W2 is This is an example of the second object, and the image of the subject W2 displayed in the images VB1 to VB7 and the image display area Sa2 is an example of the second image.

  The main body 1 is an example of an image processing apparatus, the image data va1 to va7 and the image data of a plurality of frames constituting the moving image data of the first file are examples of the first image data, and the evaluation values ia1 to ia7. A plurality of evaluation values constituting the waveform data of the first file are examples of the first evaluation value.

  Furthermore, the image data vb1 to vb7 and the image data of a plurality of frames constituting the moving image data of the second file are examples of the second image data, and a plurality of pieces of the evaluation data ib1 to ib7 and the waveform data of the second file are constituted. The evaluation value is an example of a second evaluation value, and the storage unit 40 is an example of a storage unit.

  The image display area Sa1 is an example of the first display area, the image display area Sa2 is an example of the second display area, the waveform display area Sb is an example of the third display area, and the display unit 20 Is an example of the display unit, the first waveform WL1 is an example of the first waveform, and the second waveform WL2 is an example of the second waveform.

  Furthermore, the sensors 6 and 8 and the waveform collection device 7 are examples of detection devices, the control unit 10 is an example of a control unit, the operation unit 4 is an example of an operation unit, and the current display position bar L is a common index. The control unit 10 is an example of a processing device.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  The present invention can be effectively used for various image processing.

DESCRIPTION OF SYMBOLS 1 Main body part 2 Camera 3 Illumination device 4 Operation part 5 PLC
6, 8 Sensor 7 Waveform collection device 10 Control unit 20 Display unit 30 Recording unit 30A Movie data storage area 30B Waveform data storage area 40 Storage unit 50 Video memory 60 Waveform collection unit 100 Image processing system BS bus bt Floor FA, FB, FC , FD Movie file fi File information i1 to i42, ia1 to ia7, ib1 to ib7 Evaluation value L Current display position bar Sa, Sa1, Sa2 Image display area Sb Waveform display area Sc Condition setting area Sc1 to Sc5 Input frame Se Recording button Sf Trigger button Sg Playback button Sj File button Sx Comparison button Tc Playback button Td Pause button Te Reverse playback button Tf1, Tf2 Seek bar Tg1, Tg2 Slider Tm Synchronization button Tn Comparison end button Tp1, Tp2 Operation target specification button t1 time t9 VA1~VA7, VB1~VB7 image v1~v14, va1~va7, vb1~vb7 image data W, W1, W2 subject WL1 first waveform WL2 second waveform

Claims (9)

A first image of a plurality of frames constituting a moving image of a first object and a second image of a plurality of frames constituting a moving image of a second object that is the same as or different from the first object An image processing apparatus,
First image data of a plurality of frames for displaying the first image of the plurality of frames, a plurality of first evaluation values associated with the first image data of the plurality of frames, a second of the plurality of frames A storage unit for storing a plurality of second image data for displaying a plurality of frames and a plurality of second evaluation values associated with the second image data for the plurality of frames;
A display unit having a first display area, a second display area, and a third display area;
Based on the first image data of each frame stored in the storage unit, the first image is displayed in the first display area, and based on the second image data of each frame stored in the storage unit. The second image is displayed in the second display area, the first waveform formed by the plurality of first evaluation values stored in the storage unit, and the plurality of stored in the storage unit A control unit that displays the second waveform formed by the second evaluation value on the third display area so as to be movable on the time axis;
An operation unit operated by a user to move one of the first and second waveforms displayed in the third display area on the time axis relative to the other waveform; Prepared,
The plurality of first evaluation values change according to the state of the first object, the plurality of second evaluation values change according to the state of the second object,
The control unit moves one waveform of the first and second waveforms relative to the other waveform on the time axis based on an operation of the operation unit, and the first display Changing one of the first image frame to be displayed in the region and the second image frame to be displayed in the second display region based on the movement of the one waveform;
The control unit further displays a common index for designating an arbitrary position of the first and second waveforms in the third display area so as to be movable on a time axis, and is designated by the index. The position of the first waveform and the frame of the first image displayed in the first display area correspond to each other and the position of the second waveform specified by the index and the second display area The display of the first, second and third display areas is controlled so as to maintain a state corresponding to the frame of the second image to be displayed, and the first and second display areas are controlled based on the operation of the operation unit. One of the second waveforms is moved relative to the other waveform on the time axis, and the display of the first, second, and third display areas is controlled so as to maintain the state. An image processing apparatus. The operation unit is operable to move one of the first and second waveforms on the time axis,
The control unit moves the one waveform and the index displayed in the third display area on a time axis based on an operation of moving one waveform by the operation unit, The image processing apparatus according to claim 1, wherein an image in a display area corresponding to the other waveform in the second display area is changed. The operation unit is operable to change a frame of an image displayed in one of the first and second display areas.
The control unit changes an image displayed in the one display area based on an operation of changing an image by the operation unit, and corresponds to the other display area of the first and second waveforms. The image processing apparatus according to claim 1, wherein the waveform and the index are moved on a time axis. The operation unit can instruct reproduction of a moving image in the first display area and reproduction of a moving image in the second display area,
The control unit sequentially displays a first image of a plurality of frames after the first image displayed in the first display area when the operation unit instructs to play a moving image. The image processing apparatus according to claim 1, wherein the second image of a plurality of frames after the second image displayed in the second display area is sequentially displayed. The plurality of first evaluation values are values indicating characteristics of the first image of the plurality of frames, and the plurality of second evaluation values are values indicating characteristics of the second image of the plurality of frames. The image processing apparatus according to claim 1. The value indicating the feature of the first image of the plurality of frames is a value relating to the luminance of the first image of the plurality of frames, and the value indicating the feature of the second image of the plurality of frames is the value of the first image of the plurality of frames. The image processing apparatus according to claim 5, wherein the image processing apparatus is a value relating to the luminance of the second image. The image processing apparatus according to claim 1, wherein the plurality of first and second evaluation values are output values of a detection device that detects states of the first and second objects. A first image of a plurality of frames constituting a moving image of a first object and a second image of a plurality of frames constituting a moving image of a second object that is the same as or different from the first object An image processing method comprising:
First image data of a plurality of frames for displaying the first image of the plurality of frames, a plurality of first evaluation values associated with the first image data of the plurality of frames, a second of the plurality of frames Storing a plurality of frames of second image data for displaying a plurality of frames and a plurality of second evaluation values associated with the plurality of frames of second image data;
Displaying the first display area, the second display area, and the third display area on the display unit;
A first image is displayed in the first display area based on the first image data of each stored frame, and a second image is displayed based on the second image data of each stored frame. wherein is displayed in the second display area, first formed by the first waveform and before crisis憶been said plurality of second evaluation value which is formed by the stored plurality of first evaluation values Displaying the second waveform in the third display area so as to be movable on the time axis;
An operation of moving one of the first and second waveforms displayed in the third display area on the time axis relative to the other waveform from an operation unit operated by a user is received. Steps,
Based on the accepted operation of the operation unit, one waveform of the first and second waveforms is moved relative to the other waveform on the time axis, and the first display area Changing one of a first image frame to be displayed on the second display frame and a second image frame to be displayed on the second display area based on movement of the one waveform;
A common index for designating an arbitrary position of the first and second waveforms is further displayed in the third display area so as to be movable on the time axis, and the first waveform designated by the index is displayed. Corresponding to the position of the first image displayed in the first display area and the second waveform displayed in the second display area and the position of the second waveform specified by the index. The display of the first, second and third display areas is controlled so as to maintain the state corresponding to the frame of the image of the first and second waveforms of the first and second waveforms based on the operation of the operation unit One of the waveforms is moved relative to the other waveform on the time axis, and the display of the first, second, and third display areas is controlled to maintain the state. ,
The plurality of first evaluation values change according to the state of the first object, and the plurality of second evaluation values change according to the state of the second object. Image processing method. A first image of a plurality of frames constituting a moving image of a first object and a second image of a plurality of frames constituting a moving image of a second object that is the same as or different from the first object An image processing program for causing a processing device to execute image processing,
First image data of a plurality of frames for displaying the first image of the plurality of frames, a plurality of first evaluation values associated with the first image data of the plurality of frames, a second of the plurality of frames Storing a plurality of second image data for displaying a plurality of frames and a plurality of second evaluation values associated with the second image data for the plurality of frames;
Processing for displaying the first display area, the second display area, and the third display area on the display unit;
A first image is displayed in the first display area based on the first image data of each stored frame, and a second image is displayed based on the second image data of each stored frame. wherein is displayed in the second display area, first formed by the first waveform and before crisis憶been said plurality of second evaluation value which is formed by the stored plurality of first evaluation values Processing for displaying the second waveform in the third display area so as to be movable on the time axis;
An operation of moving one of the first and second waveforms displayed in the third display area on the time axis relative to the other waveform from an operation unit operated by a user is received. Processing,
Based on the accepted operation of the operation unit, one waveform of the first and second waveforms is moved relative to the other waveform on the time axis, and the first display area A process of changing one of a first image frame to be displayed on the second display frame and a second image frame to be displayed on the second display area based on the movement of the one waveform;
A common index for designating an arbitrary position of the first and second waveforms is further displayed in the third display area so as to be movable on the time axis, and the first waveform designated by the index is displayed. Corresponding to the position of the first image displayed in the first display area and the second waveform displayed in the second display area and the position of the second waveform specified by the index. The display of the first, second and third display areas is controlled so as to maintain the state corresponding to the frame of the image of the first and second waveforms of the first and second waveforms based on the operation of the operation unit A process of moving one of the waveforms relative to the other waveform on the time axis and controlling the display of the first, second and third display areas so as to maintain the state,
Causing the processing device to execute,
The plurality of first evaluation values change according to the state of the first object, and the plurality of second evaluation values change according to the state of the second object. Image processing program.

Images