JP2017130134A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly specify timing when a specific motion is performed among prescribed motions of a subject.SOLUTION: An imaging apparatus 100 includes: acquisition means (an image acquisition part 5a) for acquiring a plurality of images obtained by consecutively imaging prescribed motions of a subject; region setting means (a region setting part 5c) for setting a region at a prescribed position in each of the plurality of images acquired by the acquisition means; and specification means (a specification part 5f) for specifying timing when the subject performs the specific motion in accordance with the amount of change and a change time among the plurality of images in the region set by the region setting means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  2. Description of the Related Art Conventionally, an apparatus that detects an impact moment while photographing a golf swing is known (see, for example, Patent Document 1). Specifically, the apparatus detects a moment of impact by providing a detection area of a predetermined size at a predetermined position where the golf club passes within the shooting frame, and detecting a change in a pixel value in the detection area. .

Japanese Patent Laying-Open No. 2015-69432

  However, in the apparatus disclosed in Patent Document 1, the moment of impact may not be detected correctly due to the effects of preparatory movements such as waggle and takeback and movements after the follow swing.

  The present invention has been made in view of such problems, and an object of the present invention is to provide an image processing apparatus that can correctly specify the timing at which a specific operation is performed among predetermined operations of a subject, An image processing method and a program are provided.

In order to solve the above problems, an image processing apparatus according to the present invention provides:
An acquisition unit that acquires a plurality of images obtained by continuously capturing a predetermined motion of a subject, a region setting unit that sets a region at a predetermined position in each of the plurality of images acquired by the acquisition unit, and the region setting And a specifying unit that specifies a timing at which the subject has performed a specific operation in accordance with a change amount and a change time between the plurality of images in the region set by the unit.

The image sharing method according to the present invention includes:
An image processing method using an image processing apparatus, which acquires a plurality of images obtained by continuously capturing a predetermined motion of a subject, and sets an area at a predetermined position in each of the acquired plurality of images And a process of specifying a timing at which the subject has performed a specific operation in accordance with a change amount and a change time between the plurality of images in a set region.

The program according to the present invention is
An acquisition unit that acquires a plurality of images obtained by continuously capturing a predetermined motion of a subject, and a region setting that sets a region at a predetermined position in each of the plurality of images acquired by the acquisition unit. And a specifying means for specifying a timing at which the subject has performed a specific action according to a change amount and a change time between the plurality of images in the area set by the area setting means. Yes.

  According to the present invention, it is possible to correctly specify the timing at which a specific operation is performed among predetermined operations of the subject.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. 3 is a flowchart illustrating an example of an operation related to an imaging process by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to a motion detection process in the imaging process of FIG. 2. 3 is a flowchart illustrating an example of an operation related to an operation timing specifying process by the imaging apparatus of FIG. 1. (A)-(e) is a figure which shows an example of the frame image which comprises a moving image, (f) is a figure which shows an example of the image change amount information graphed.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.
As illustrated in FIG. 1, the imaging apparatus 100 according to the present embodiment includes a central control unit 1, a memory 2, an imaging unit 3, an image data generation unit 4, an image processing unit 5, an image recording unit 6, A display unit 7 and an operation input unit 8 are provided.
The central control unit 1, the memory 2, the imaging unit 3, the image data generation unit 4, the image processing unit 5, the image recording unit 6, and the display unit 7 are connected via a bus line 9.

  The central control unit 1 controls each unit of the imaging device 100. Specifically, although not shown, the central control unit 1 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and includes various processing programs (illustrated) for the imaging apparatus 100. Various control operations are performed according to (omitted).

  The memory 2 is configured by, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by each unit such as the central control unit 1 and the image processing unit 5.

  The imaging unit 3 images a subject that performs a predetermined operation (for example, a golf swing). Specifically, the imaging unit 3 includes a lens unit 3a, an electronic imaging unit 3b, and an imaging control unit 3c.

The lens unit 3a includes a plurality of lenses such as a zoom lens and a focus lens, for example.
The electronic imaging unit 3b is composed of, for example, an image sensor such as a complementary metal-oxide semiconductor (CMOS) or a charge coupled device (CCD), and converts an optical image that has passed through various lenses of the lens unit 3a into a two-dimensional image signal. To do.
In addition, although illustration is abbreviate | omitted, the imaging part 3 may be provided with the aperture_diaphragm | restriction which adjusts the quantity of the light which passes the lens part 3a.

  The imaging control unit 3c controls imaging of the subject by the imaging unit 3. In other words, the imaging control unit 3c includes a timing generator, a driver, and the like, although not illustrated. Then, the imaging control unit 3c scans and drives the electronic imaging unit 3b with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 3b every predetermined period, and the electronic imaging unit 3b. The frame image is read from the imaging area for each screen and is output to the image data generation unit 4.

The image data generation unit 4 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the frame image transferred from the electronic imaging unit 3b, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data). Further, the image data generation unit 4 encodes each of a plurality of frame images constituting the moving image in a predetermined format (for example, MPEG format, motion JPEG format, etc.), and generates moving image image data.
In addition, the image data generation unit 4 transfers the generated image data to the memory 2 used as a buffer memory.

The image processing unit 5 includes an image acquisition unit 5a, a decoding unit 5b, an area setting unit 5c, a calculation unit 5d, an image change amount information acquisition unit 5e, and a specifying unit 5f.
Note that each unit of the image processing unit 5 includes, for example, a predetermined logic circuit, but the configuration is an example and the present invention is not limited thereto.

The image acquisition unit 5a acquires an image to be processed by the motion detection process.
That is, the image acquisition unit (acquisition unit) 5a acquires a plurality of frame images obtained by continuously capturing the motion of the subject. Specifically, the image acquisition unit 5a includes, for example, a plurality of frames generated by the image data generation unit 4 when a series of motions (for example, golf swing) by a subject is captured by the imaging unit 3. Image data of a moving image composed of images is acquired as a processing target of motion detection processing.
For example, the imaging unit 3 captures a series of operations of a predetermined sport by a subject at a predetermined imaging frame rate (for example, 1000 fps), and the image data generation unit 4 includes a plurality of frame images associated with frame numbers. Is generated and transferred to the memory 2.
And the image acquisition part 5a reads the image data of the moving image stored in the memory 2, and acquires it as a process target of a motion detection process.
Note that the image acquisition unit 5a may read out the image data of the moving image recorded in the image recording unit 6 after the imaging of the subject by the imaging unit 3 and acquire it as a processing target of the motion detection process.

The decoding unit 5b decodes moving image image data including a plurality of frame images.
Specifically, for example, the decoding unit 5b applies the moving image to the moving image image data encoded in a predetermined format (for example, MPEG format or motion JPEG format) acquired by the image acquisition unit 5a. A decoding process according to the image data encoding method is performed to generate YUV data of a plurality of frame images.
At this time, the decoding unit 5b may perform a process of reducing each frame image to a predetermined size (for example, VGA or QVGA size) based on, for example, the display resolution of the display panel 7a.

The area setting unit 5c sets a subject detection area.
That is, the region setting unit (region setting unit) 5c specifies the timing at which the subject performs a specific operation at a predetermined position for each of a plurality of frame images constituting the moving image acquired by the image acquisition unit 5a. A plurality of detection areas are set. Specifically, for example, in the case of a moving image related to a golf swing by a subject, the region setting unit 5c includes an impact detection region A for detecting the club head when the club head has an impact of capturing a golf ball. The position is set to the position below the center (the foot of the subject) in each of the plurality of frame images constituting the moving image (see FIGS. 5A to 5E). In the impact detection area A, in addition to the impact described above (see FIG. 5D), the club head is hung from side to side (for example, waggle, etc .; see FIG. 5A), take back (back swing) The club head has a horizontally long rectangular shape so that the club head when it is performed (see FIG. 5B) can also be detected.
In addition, the area setting unit 5c displays the moving image as the top detection area B for detecting the club head when the takeback (back swing) is performed and the top position is reached (see FIG. 5C). It is set at a position on the center (above the head of the subject) in each of a plurality of frame images to be configured (see FIGS. 5A to 5E). The top detection area B is a horizontally long rectangle so that the club head when a follow swing is performed (see FIG. 5E) can be detected in addition to the top (see FIG. 5C). It has a shape.
The region setting unit 5c sets the impact detection region A and the top detection region B at predetermined positions of the first frame image (see FIG. 5A) among the plurality of frame images, thereby remaining frame images. Similarly, the detection area is set at a position corresponding to the detection area of the first frame image.
Here, the positions and sizes of the impact detection area A and the top detection area B are manually adjusted based on a predetermined operation of the operation input unit 8 by the user in the first frame image displayed on the display panel 7a, for example. It may be set, or may be automatically set with the contents defined as defaults in advance.
In addition, the sizes of the impact detection area A and the top detection area B are relative to each other so as to be a predetermined ratio with respect to the size of the entire screen based on the direction in which the subject is photographed, the angle of view of the imaging unit 3, and the like. It may be set with any size, or may be set with an absolute size so as to have a predetermined number of pixels.

The calculation unit 5d calculates the amount of change between a plurality of images in the detection area.
That is, the calculation unit 5d calculates the amount of change between a plurality of frame images in each of the impact detection area A and the top detection area B set by the area setting unit 5c. Specifically, for example, with respect to each frame image decoded by the decoding unit 5b, the calculation unit 5d performs the frame image F2 between the frame image F2 and the frame image F1 in order from the first frame image F1. If there is a frame image F3, the difference between the pixel values in the impact detection area A and the top detection area B with the adjacent frame image is taken, and the difference between the pixel values between these adjacent frame images. Are sequentially integrated along the time axis to calculate a change amount between a plurality of frame images. When the calculation unit 5d finishes calculating the amount of change between the plurality of frame images, the calculation unit 5d displays image change amount information indicating the amount of change between the plurality of frame images as image data of a moving image including the plurality of frame images. And recorded in the image recording unit 6. Here, in the case of a golf swing, the amount of change between a plurality of frame images is represented, for example, as a graph shown in FIG. Specifically, the solid line graph shown in FIG. 5F indicates the amount of change between a plurality of frame images in the impact detection area A. This solid line graph shows that the graph is undulating during waggle, takeback, and impact. On the other hand, the alternate long and short dash line graph indicates the amount of change between a plurality of frame images in the top detection region B. In this one-dot chain line graph, it can be seen that the graph undulates at the top after takeback and at the follow after impact.

The image change amount information acquisition unit 5e acquires image change amount information to be processed by the operation timing specifying process.
In other words, the image change amount information acquisition unit 5e acquires image change amount information indicating the change amount between a plurality of frame images in each of the impact detection area A and the top detection area B calculated by the calculation unit 5d. Specifically, for example, when a user-desired moving image is designated based on a predetermined operation of the operation input unit 8 by the user among moving images recorded in the image recording unit 6, image change amount information acquisition is performed. The unit 5e acquires image change amount information recorded in association with the image data of the moving image. Here, the moving images that can be specified by the user are limited to moving images that are recorded in the image recording unit 6 in association with image change amount information.

The specifying unit 5f specifies the timing when the subject performs a specific operation.
That is, the specifying unit (specifying unit) 5f determines the timing (for example, the impact timing of the golf swing) when the subject performs a specific action based on the image change amount information acquired by the image change amount information acquiring unit 5e. Identify. Specifically, the specifying unit 5f firstly sets the image change amount information in the impact detection area A among the image change amount information acquired by the image change amount information acquiring unit 5e (for example, the solid line shown in FIG. 5F). (A data representing the graph of FIG. 5), a period in which the amount of change between a plurality of frame images in the impact detection area A is equal to or greater than a predetermined (threshold value) (for example, a1, a2, a3 in the solid line graph shown in FIG. 5F). (Hereinafter referred to as the first period). When the first period is extracted, the specifying unit 5f uses a plurality of pieces of information in the top detection region B from the image change amount information in the top detection region B (for example, data representing a one-dot chain line graph illustrated in FIG. 5). A period (for example, b1 and b2 in the dashed-dotted line graph shown in FIG. 5F) (hereinafter referred to as the second period) is extracted in which the change amount between the frame images is equal to or greater than a predetermined value. When the second period is extracted, the specifying unit 5f selects the second period (for example, the first second period (for example, FIG. 5F) when a plurality of second periods are extracted. It is determined whether or not there is a first period after b1)) of the dashed-dotted line graph. When it is determined that there is a first period after the second period (the first second period when a plurality of second periods are extracted), the specifying unit 5f A peak point (P3 in the solid line graph in FIG. 5F) of the first period (for example, a3 in the solid line graph in FIG. 5F) immediately after the period is specified as the impact timing.
When the second period is not extracted, the specifying unit 5f determines whether or not there is a first period whose length is within a predetermined time among the extracted first periods. judge. If it is determined that there is a first period (for example, a2 and a3 in the solid line graph shown in FIG. 5F) whose length is within a predetermined time, the specifying unit 5f It is determined whether there are a plurality of periods. When it is determined that there are a plurality of the first periods, the specifying unit 5f selects the last first period (for example, a3 in the solid line graph of FIG. 5F) among the plurality of first periods. ) Peak point (P3 in the solid line graph in FIG. 5 (f)) is specified as the impact timing, and the peak in the first period (for example, a2 in the solid line graph in FIG. 5 (f)) immediately before that is specified. The point (P2 in the solid line graph in FIG. 5F) is specified as the takeback start timing. On the other hand, when it is determined that the first period is not plural, that is, the first period is one, the specifying unit 5f determines the peak point of this first period (for example, the solid line in FIG. 5F). P3) of the graph is specified as the impact timing.

The image recording unit 6 is configured by, for example, a nonvolatile memory (flash memory). The image recording unit 6 records image data of various images encoded by a coding unit (not shown) of the image data generation unit 4 using a predetermined coding method.
Specifically, the image recording unit 6 records, for example, image data of a moving image in which a series of movements of the subject are continued by the imaging unit 3.

  The image recording unit 6 may be configured, for example, such that a recording medium (not shown) is detachable and controls reading of data from the mounted recording medium and writing of data to the recording medium.

  The display unit 7 displays still images and moving images. Specifically, the display unit 7 includes a display panel 7a and a display control unit 7b.

The display panel 7a displays an image in the display area. Specifically, the display unit 7 updates the live view image while sequentially updating a plurality of image frames generated by imaging the subject by the imaging unit 3 in the still image capturing mode and the moving image capturing mode. Is displayed.
Examples of the display panel 7a include a liquid crystal display panel and an organic EL display panel, but are not limited to these examples.

  The display control unit 7b performs control to display a predetermined image on the display screen of the display panel 7a based on image data of a predetermined size read from the image recording unit 6 and decoded by the decoding unit 5b. Specifically, the display control unit 7b includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. Then, the digital video encoder reads the luminance signal Y and the color difference signals Cb and Cr decoded by the decoding unit 5b and stored in the VRAM (not shown) from the VRAM via the VRAM controller. A video signal is generated and output to the display panel 7a.

The operation input unit 8 is for inputting various instructions to the apparatus main body.
Specifically, the operation input unit 8 includes, for example, an operation unit (not shown) including up / down / left / right cursor buttons and a determination button related to a selection instruction of a mode, a function, and the like.
When the user operates various buttons on the operation unit, the operation input unit 8 outputs an operation instruction corresponding to the operated button to the central control unit 1. The central control unit 1 causes each unit to execute a predetermined operation (for example, imaging of a subject) in accordance with an operation instruction output from the operation input unit 8 and input.
The operation input unit 8 may include a touch panel (not shown) provided integrally with the display panel 7a of the display unit 7.

Next, the imaging process will be described with reference to FIGS. 2, 3, and 5.
FIG. 2 is a flowchart illustrating an example of an operation related to the imaging process.

  The imaging process described below is executed when a motion detection mode is instructed from a plurality of operation modes displayed on a menu screen (not shown) based on a predetermined operation of the operation input unit 8 by the user. Process.

<Imaging process>
As shown in the flowchart of FIG. 2, when imaging is instructed based on a predetermined operation of the operation input unit 8 by the user, the imaging unit 3 performs a golf swing operation at a predetermined imaging frame rate (for example, 1000 fps). The image data generation unit 4 generates image data of a moving image obtained by encoding a plurality of frame images using a predetermined encoding method (step S1).
Then, the image data generation unit 4 transfers the generated moving image image data to the memory 2.

Subsequently, the image processing unit 5 performs a motion detection process (see FIG. 3) for detecting a motion by calculating a change amount between each frame image from the image data of the moving image (step S2), and ends the imaging process. To do.
The motion detection process will be described later.

<Motion detection processing>
Hereinafter, the motion detection process will be described in detail with reference to FIG.
FIG. 3 is a flowchart illustrating an example of an operation related to the motion detection process.

  As shown in FIG. 3, the image acquisition unit 5 a reads out image data of a moving image composed of a plurality of frame images obtained by continuously capturing golf swing motions from the memory 2 and uses it as a processing target for motion detection processing. Obtain (step S11).

Next, the decoding unit 5b sequentially decodes all frame images constituting the moving image image data from the first frame image (step S12).
Subsequently, the area setting unit 5c, for example, in a state where any one of the frame images (for example, the frame image having the frame number “1” (see FIG. 5A)) is displayed on the display panel 7a. The impact detection area A is set at a position below the center of the frame image (under the foot of the subject) based on a predetermined operation of the operation input unit 8 by the above, and at a position above the center of the frame image (above the head of the subject). A top detection area B is set (step S13; see FIG. 5A). As a result, the impact detection area A and the top detection area B are similarly set for the other frame images (see FIGS. 5B to 5E) constituting the moving image.

  Next, for example, for each frame image constituting the moving image, the calculation unit 5d calculates a difference in pixel value in each of the impact detection area A and the top detection area B from the adjacent frame image in order from the first frame image. Then, a value obtained by sequentially integrating the pixel value differences between the adjacent frame images along the time axis is calculated as a change amount between the plurality of frame images (step S14). Subsequently, the calculation unit 5d records the image change amount information indicating the calculated change amounts between the plurality of frame images in the image recording unit 6 in association with the image data of the moving image including the plurality of frame images. (Step S15), the motion detection process is terminated.

Next, the operation timing specifying process will be described with reference to FIGS.
FIG. 4 is a flowchart illustrating an example of an operation related to the operation timing specifying process.

  The operation timing specifying process described below is performed when an operation timing specifying mode is selected from a plurality of operation modes displayed on a menu screen (not shown) based on a predetermined operation of the operation input unit 8 by the user. It is a process to be executed.

<Operation timing identification process>
As shown in the flowchart of FIG. 4, first, the image processing unit 5 determines whether or not a user-desired moving image has been designated based on a predetermined operation of the operation input unit 8 by the user (step S21).
When it is determined in step S21 that the user-desired moving image has been designated (step S21; Yes), the image change amount information acquisition unit 5e receives the image data of the designated moving image from the image recording unit 6. The image change amount information read out and recorded in association with the image data of the moving image is acquired as a processing target of the operation timing specifying process (step S22).
On the other hand, when it is determined in step S21 that the user-desired moving image has not been designated (step S21; No), the process of step S21 is repeated until the designation is made.

Next, the specifying unit 5f represents image change amount information (for example, a solid line graph illustrated in FIG. 5F) in the impact detection area A among the image change amount information acquired by the image change amount information acquiring unit 5e. From the data), it is determined whether or not there is a first period in which the amount of change between the plurality of frame images is greater than or equal to a predetermined (threshold value) in the impact detection area A (step S23).
When it is determined in step S23 that there is the first period (for example, a1, a2, a3 in the solid line graph of FIG. 5F) (step S23; Yes), the specifying unit 5f detects the top. From the image change amount information in the region B (for example, data representing a one-dot chain line graph shown in FIG. 5F), the second period in which the change amount between the plurality of frame images in the top detection region B is equal to or greater than a predetermined value. It is determined whether or not there is (step S24).
On the other hand, when it determines with there being no 1st period in step S23 (step S23; No), an operation timing specific process is complete | finished. At this time, under the control of the display control unit 7b, a message such as “Specific operation timing (for example, impact timing) cannot be specified” is displayed.

Next, when it is determined in step S24 that there is a second period (for example, b1 and b2 in the dashed-dotted line graph in FIG. 5F) (step S24; Yes), the specifying unit 5f It is determined whether there is a first period after the second period (or the first second period when a plurality of second periods are extracted) (step S25).
In step S25, after the second period (the first second period when a plurality of second periods are extracted), the first period (for example, a3 in the solid line graph of FIG. 5F). ) Is determined (step S25; Yes), the specifying unit 5f impacts the peak point (P3 in the solid line graph of FIG. 5F) in the first period immediately after the second period. (Step S26), and the operation timing specifying process is terminated.
On the other hand, when it is determined in step S25 that there is no first period after the second period (the first second period when a plurality of second periods are extracted) (step S25; No) ), The operation timing specifying process is terminated. At this time, under the control of the display control unit 7b, a message such as “Specific operation timing (for example, impact timing) cannot be specified” is displayed.

In Step S24, when it is determined that there is no second period (Step S24; No), the specifying unit 5f determines whether or not there is a first period whose length is within a predetermined time. Determination is made (step S27).
When it is determined in step S27 that there is a first period (for example, a2 and a3 in the solid line graph of FIG. 5F) whose length is within a predetermined time (step S27; Yes), The specifying unit 5f determines whether there are a plurality of the first periods (step S28).
On the other hand, when it is determined in step S27 that there is no first period in which the length of the period is within the predetermined time (step S27; No), the operation timing specifying process ends. At this time, under the control of the display control unit 7b, a message such as “Specific operation timing (for example, impact timing) cannot be specified” is displayed.

Next, in Step S28, when it is determined that there are a plurality of first periods whose length is within a predetermined time (Step S28; Yes), the specifying unit 5f includes the plurality of first periods. The peak point of the last first period (P3 in the solid line graph of FIG. 5 (f)) is specified as the impact timing (step S29), and the peak point of the first period (FIG. 5 (f) ) P2) of the solid line graph is specified as the takeback start timing (step S30), and the operation timing specifying process is terminated.
On the other hand, in step S28, there are not a plurality of first periods whose length is within a predetermined time, that is, the first period (for example, a3 in the solid line graph of FIG. 5F) is one. If determined to be present (step S28; No), the identifying unit 5f identifies the peak point of this first period (P3 in the solid line graph of FIG. 5 (f)) as the impact timing (step S31). Then, the operation timing specifying process is terminated.

  As described above, according to the imaging apparatus 100 of the present embodiment, a moving image composed of a plurality of frame images obtained by continuously capturing a predetermined motion of a subject is acquired, and a predetermined position in each of the plurality of frame images is acquired. Area (impact detection area A and top detection area B) is specified, and the timing at which the subject has performed a specific action is specified according to the change amount and change time between the plurality of frame images in the area. In the case of a golf swing, a specific motion is a motion that differs in the amount of change and change time between multiple frame images from the moment of impact, such as a preparatory motion in a golf swing such as waggle or takeback, or a motion after a follow swing. It can be excluded from the candidates, and the accuracy of identifying the impact timing can be improved.

  Further, according to the imaging device 100 of the present embodiment, when the length of a period in which the amount of change between a plurality of frame images in the impact detection area A is equal to or greater than a predetermined (threshold) is within a predetermined time, When the predetermined timing is specified as the timing at which the subject performs a specific operation, and the length of the period in which the change amount between the plurality of frame images in the impact detection area A is equal to or greater than the predetermined (threshold) is longer than the predetermined time In this case, the predetermined timing in the period is not specified as the timing at which the subject performs a specific operation. For example, in the case of a golf swing, before the start of a swing such as a waggle with a long variation time between a plurality of frame images. It can be suppressed that the operation of is specified in error.

  Further, according to the imaging apparatus 100 of the present embodiment, among the periods in which the amount of change between the plurality of frame images in the impact detection area A is greater than or equal to a predetermined (threshold), the period is within a predetermined time. In the case of a golf swing, for example, in the case of a golf swing, the last timing in the last period is specified as the timing at which the subject has performed a specific action. The predetermined timing of the period is specified as the impact timing, the predetermined timing of the immediately preceding period is specified as the takeback start timing, and each operation timing in the golf swing can be preferably specified.

  Further, according to the imaging apparatus 100 of the present embodiment, a range through which a part of the subject (for example, a club head) passes when performing an operation other than a specific operation among the predetermined operations of the subject is included. Since the impact detection area A and the top detection area B are set as described above, for example, in the case of a golf swing, a motion other than impact, such as a preparatory motion in a golf swing such as waggle or takeback, or a motion after a follow swing By detecting this, it becomes possible to relatively specify the operation timing of the impact among these operations.

Further, according to the imaging apparatus 100 of the present embodiment, when there is a period in which the amount of change between a plurality of frame images is greater than or equal to a predetermined (threshold) in each of the impact detection area A and the top detection area B, Based on the order, the timing at which the subject performs a specific operation is specified. Specifically, for example, in the case of a golf swing, a second period (a plurality of second periods are extracted) in which the amount of change between a plurality of frame images in the top detection region B is equal to or greater than a predetermined (threshold value). Specifies the peak point in the first period (period in which the amount of change between the plurality of frame images in the impact detection area A is equal to or greater than a predetermined (threshold)) immediately after the first second period) as the impact timing. It will be.
Therefore, by setting the top detection area B in addition to the impact detection area A by the area setting unit 5c, the predetermined setting detected by the top detection area B (for example, the top swing action) Since it becomes possible to specify the operation timing of the impact, the accuracy of the specifying process can be improved.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the above embodiment, when the length of a period in which the amount of change between a plurality of frame images in the impact detection area A is equal to or greater than a predetermined (threshold) is within a predetermined time, the predetermined timing in the period is The subject is specified as the timing at which the subject performs a specific operation, but this is an example and the present invention is not limited to this. For example, when the time required for the subject to perform a specific motion can be predicted, the predetermined timing in the period is set when the length of the period is within a predetermined range, that is, the predicted required time. The timing when the subject performs a specific operation may be specified.

  Further, in the above embodiment, for each frame image constituting the moving image, the difference between pixel values in the impact detection area A and the top detection area B from the adjacent frame image is taken, and these adjacent frames are taken. A value obtained by sequentially integrating the pixel value differences between images along the time axis is calculated as a change amount between a plurality of frame images. However, this is an example, and the present invention is not limited to this. For example, for each frame image constituting the moving image, the absolute value of the difference between the pixel values in each of the impact detection area A and the top detection area B from the adjacent frame image is calculated as the amount of change between the plurality of frame images. You may do it.

  In the above embodiment, the imaging process (see FIG. 2) is performed in the motion detection mode, and the operation timing specifying process (see FIG. 4) is performed in the operation timing specifying mode. It is not limited to this. For example, the imaging process shown in FIG. 2 and the operation timing specifying process shown in FIG. 4 may be performed in a series of flows. In such a case, the same processing as in the above embodiment is performed from step S11 to step S14 of the motion detection processing (see FIG. 3), and in step S15, the calculation unit 5d calculates the amount of change between the plurality of frame images calculated. Is transferred to the memory 2 in association with the image data of the moving image composed of the plurality of frame images, and the motion detection process is terminated. In the operation timing specifying process (see FIG. 4), first, the image change amount information acquisition unit 5e reads the image change amount information from the memory 2, acquires the image change amount information, and then, similarly to the above embodiment. Then, the processing of step S23 to step S31 is performed.

  In the above embodiment, two types of areas, the impact detection area A and the top detection area B, are set. However, the present invention is not limited to this example. The setting of the top detection area B may be omitted based on a predetermined operation of the unit 8.

  In the above embodiment, the specifying unit 5f, for example, when there are a plurality of periods in which the amount of change between a plurality of images in the impact detection area A satisfies a predetermined condition (for example, a threshold value or more). A predetermined timing of a period satisfying a predetermined condition (for example, the shortest period) among the plurality of periods may be specified as a timing at which the subject performs a specific operation. Specifically, for example, when a golf swing is taken as an example, as shown in the solid line graph of FIG. 5F, the specifying unit 5f has a change amount between a plurality of images in the impact detection area A equal to or greater than a threshold value. When there are a plurality of periods (a1, a2, a3) satisfying that, a predetermined timing (for example, peak point P3) of the period (a3) satisfying that the change time is the shortest period among the plurality of periods Is specified as the timing when the subject performs a specific operation (impact timing).

In the above embodiment, the image acquisition unit 5a, the region setting unit 5c, and the specifying unit 5f are driven under the control of the central control unit 1 to function as the acquisition unit, the region setting unit, and the specifying unit. However, the present invention is not limited to this, and may be realized by executing a scheduled program or the like by the CPU of the central control unit 1.
That is, a program including an acquisition processing routine, an area setting processing routine, and a specific processing routine is recorded in a program memory (not shown) that stores the program. Then, the CPU of the central control unit 1 may function as an acquisition unit that acquires a plurality of images obtained by continuously capturing a predetermined operation of the subject by an acquisition process routine. Further, the CPU of the central control unit 1 may function as a region setting unit that sets a region at a predetermined position in each of the plurality of images acquired by the acquisition processing routine by the region setting processing routine. Further, the CPU of the central control unit 1 determines the timing at which the subject performs a specific operation according to the amount of change and the change time between the plurality of images in the region set by the region setting processing routine. You may make it function as a specific means to specify.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Acquisition means for acquiring a plurality of images obtained by continuously capturing a predetermined motion of a subject;
Area setting means for setting an area at a predetermined position in each of the plurality of images acquired by the acquisition means;
A specifying unit that specifies a timing at which the subject has performed a specific operation according to a change amount and a change time between the plurality of images in the region set by the region setting unit;
An image processing apparatus comprising:
<Claim 2>
The region setting means sets the region at the position where the predetermined position is a position through which a part of the subject passes when a predetermined operation of the subject is performed,
The specifying unit specifies a predetermined timing in a period in which a change amount and a change time between the plurality of images in a region set by the region setting unit satisfy a predetermined condition as a timing at which the subject performs a specific operation. The image processing apparatus according to claim 1, wherein:
<Claim 3>
When the length of a period in which the amount of change between the plurality of images in the region set by the region setting unit is greater than or equal to a predetermined range is within a predetermined range, the specifying unit determines the predetermined timing in the period, The image processing apparatus according to claim 2, wherein the image processing apparatus is specified as a timing at which a subject performs a specific operation.
<Claim 4>
The specifying unit determines a predetermined timing in the period when the length of a period in which a change amount between the plurality of images in the region set by the region setting unit is equal to or greater than a predetermined time is within a predetermined time. Is specified as the timing at which the specific operation is performed, while the length of the period in which the amount of change between the plurality of images in the region set by the region setting unit is greater than or equal to a predetermined time is longer than a predetermined time, The image processing apparatus according to claim 3, wherein the predetermined timing in the period is not specified as a timing at which the subject performs a specific operation.
<Claim 5>
The specifying unit has a period in which a length of the period is within a predetermined time among a period in which a change amount between the plurality of images in the region set by the region setting unit is not less than a predetermined time. The image processing apparatus according to claim 4, wherein a predetermined timing in a period corresponding to a predetermined order among the plurality of periods is specified as a timing at which the subject performs a specific operation.
<Claim 6>
When there are a plurality of periods in which the change amount and change time between the plurality of images in the region set by the region setting unit satisfy a predetermined condition, the specifying unit determines a predetermined order among the plurality of periods. The image processing apparatus according to claim 2, wherein a predetermined timing in a period corresponding to is specified as a timing at which the subject performs a specific operation.
<Claim 7>
In the case where there are a plurality of periods in which the amount of change between the plurality of images in the region set by the region setting unit satisfies a predetermined condition, the specifying unit has a change time satisfying a predetermined condition among the plurality of periods The image processing apparatus according to claim 2, wherein the predetermined timing of the period is specified as a timing at which the subject performs a specific operation.
<Claim 8>
When there are a plurality of periods in which the amount of change between the plurality of images in the region set by the region setting unit satisfies a predetermined condition, the specifying unit determines a predetermined period of the shortest change time among the plurality of periods. The image processing apparatus according to claim 7, wherein the timing is specified as a timing at which the subject performs a specific operation.
<Claim 9>
The region setting means sets the region to include a range through which a part of the subject passes when performing an operation other than a specific operation among the predetermined operations of the subject. The image processing apparatus according to claim 1.
<Claim 10>
The region setting means sets a plurality of the regions,
If there is a period in which the amount of change between the plurality of images is greater than or equal to a predetermined value in each of the plurality of areas set by the area setting unit, the specifying unit specifies the subject based on the order of the periods. The image processing apparatus according to claim 1, wherein a timing at which the operation is performed is specified.
<Claim 11>
The image processing apparatus according to claim 1, wherein the change amount between the plurality of images in the region set by the region setting unit is an absolute value of a difference in pixel value between the images.
<Claim 12>
The amount of change between the plurality of images in the region set by the region setting means is a value obtained by integrating a difference in pixel values between the images along a time axis. Image processing device.
<Claim 13>
The predetermined motion of the subject is a golf swing,
The image processing apparatus according to claim 5, wherein the timing at which the subject performs a specific operation is an impact timing.
<Claim 14>
The specifying unit has a period in which a length of the period is within a predetermined time among a period in which a change amount between the plurality of images in the region set by the region setting unit is not less than a predetermined time. The predetermined timing of a period corresponding to the last of the plurality of periods is specified as the timing of the impact, and the predetermined timing of the period immediately before is specified as the start timing of takeback. An image processing apparatus according to 1.
<Claim 15>
An image processing method using an image processing apparatus,
A process of acquiring a plurality of images in which a predetermined motion of a subject is continuously captured;
A process of setting an area at a predetermined position in each of a plurality of acquired images;
A process of specifying a timing at which the subject has performed a specific operation according to a change amount and a change time between the plurality of images in a set region;
An image processing method comprising:
<Claim 16>
The computer of the image processing device
Acquisition means for acquiring a plurality of images in which a predetermined motion of a subject is continuously captured;
Area setting means for setting an area at a predetermined position in each of the plurality of images acquired by the acquisition means;
A specifying unit for specifying a timing at which the subject performs a specific operation according to a change amount and a change time between the plurality of images in the region set by the region setting unit;
A program characterized by functioning as

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Central control part 3 Imaging part 5 Image processing part 5a Image acquisition part 5b Decoding part 5c Area setting part 5d Calculation part 5e Image change amount information acquisition part 5f Specific part A Impact detection area B Top detection area

Claims (16)

Acquisition means for acquiring a plurality of images obtained by continuously capturing a predetermined motion of a subject;
Area setting means for setting an area at a predetermined position in each of the plurality of images acquired by the acquisition means;
A specifying unit that specifies a timing at which the subject has performed a specific operation according to a change amount and a change time between the plurality of images in the region set by the region setting unit;
An image processing apparatus comprising: The region setting means sets the region at the position where the predetermined position is a position through which a part of the subject passes when a predetermined operation of the subject is performed,
The specifying unit specifies a predetermined timing in a period in which a change amount and a change time between the plurality of images in a region set by the region setting unit satisfy a predetermined condition as a timing at which the subject performs a specific operation. The image processing apparatus according to claim 1, wherein:   When the length of a period in which the amount of change between the plurality of images in the region set by the region setting unit is greater than or equal to a predetermined range is within a predetermined range, the specifying unit determines the predetermined timing in the period, The image processing apparatus according to claim 2, wherein the image processing apparatus is specified as a timing at which a subject performs a specific operation.   The specifying unit determines a predetermined timing in the period when the length of a period in which a change amount between the plurality of images in the region set by the region setting unit is equal to or greater than a predetermined time is within a predetermined time. Is specified as the timing at which the specific operation is performed, while the length of the period in which the amount of change between the plurality of images in the region set by the region setting unit is greater than or equal to a predetermined time is longer than a predetermined time, The image processing apparatus according to claim 3, wherein the predetermined timing in the period is not specified as a timing at which the subject performs a specific operation.   The specifying unit has a period in which a length of the period is within a predetermined time among a period in which a change amount between the plurality of images in the region set by the region setting unit is not less than a predetermined time. The image processing apparatus according to claim 4, wherein a predetermined timing in a period corresponding to a predetermined order among the plurality of periods is specified as a timing at which the subject performs a specific operation.   When there are a plurality of periods in which the change amount and change time between the plurality of images in the region set by the region setting unit satisfy a predetermined condition, the specifying unit determines a predetermined order among the plurality of periods. The image processing apparatus according to claim 2, wherein a predetermined timing in a period corresponding to is specified as a timing at which the subject performs a specific operation.   In the case where there are a plurality of periods in which the amount of change between the plurality of images in the region set by the region setting unit satisfies a predetermined condition, the specifying unit has a change time satisfying a predetermined condition among the plurality of periods The image processing apparatus according to claim 2, wherein the predetermined timing of the period is specified as a timing at which the subject performs a specific operation.   When there are a plurality of periods in which the amount of change between the plurality of images in the region set by the region setting unit satisfies a predetermined condition, the specifying unit determines a predetermined period of the shortest change time among the plurality of periods. The image processing apparatus according to claim 7, wherein the timing is specified as a timing at which the subject performs a specific operation.   The region setting means sets the region to include a range through which a part of the subject passes when performing an operation other than a specific operation among the predetermined operations of the subject. The image processing apparatus according to claim 1. The region setting means sets a plurality of the regions,
If there is a period in which the amount of change between the plurality of images is greater than or equal to a predetermined value in each of the plurality of areas set by the area setting unit, the specifying unit specifies the subject based on the order of the periods. The image processing apparatus according to claim 1, wherein a timing at which the operation is performed is specified.   The image processing apparatus according to claim 1, wherein the change amount between the plurality of images in the region set by the region setting unit is an absolute value of a difference in pixel value between the images.   The amount of change between the plurality of images in the region set by the region setting means is a value obtained by integrating a difference in pixel values between the images along a time axis. Image processing device. The predetermined motion of the subject is a golf swing,
The image processing apparatus according to claim 5, wherein the timing at which the subject performs a specific operation is an impact timing.   The specifying unit has a period in which a length of the period is within a predetermined time among a period in which a change amount between the plurality of images in the region set by the region setting unit is not less than a predetermined time. The predetermined timing of a period corresponding to the last of the plurality of periods is specified as the timing of the impact, and the predetermined timing of the period immediately before is specified as the start timing of takeback. An image processing apparatus according to 1. An image processing method using an image processing apparatus,
A process of acquiring a plurality of images in which a predetermined motion of a subject is continuously captured;
A process of setting an area at a predetermined position in each of a plurality of acquired images;
A process of specifying a timing at which the subject has performed a specific operation according to a change amount and a change time between the plurality of images in a set region;
An image processing method comprising: The computer of the image processing device
Acquisition means for acquiring a plurality of images in which a predetermined motion of a subject is continuously captured;
Area setting means for setting an area at a predetermined position in each of the plurality of images acquired by the acquisition means;
A specifying unit for specifying a timing at which the subject performs a specific operation according to a change amount and a change time between the plurality of images in the region set by the region setting unit;
A program characterized by functioning as

Images