JP5920264B2 - Image identification device, image identification system, image identification method, and program - Google Patents

Description

The present invention relates to an image identification apparatus, images specific system related images specific method, and a program.

  2. Description of the Related Art Conventionally, a technique for detecting impact sound generated when a ball is hit with a golf club with a microphone and displaying a digest image of a moving image taken by a digital camera is known (see, for example, Patent Document 1).

JP 2008-236124 A

  However, in Patent Document 1, it is difficult to detect a specific sound such as an impact sound with a microphone in an environment with ambient noise. In addition, if sound is collected at a position away from the subject, there is a problem that a delay due to the transmission time of the impact sound occurs, and the image cannot be accurately specified based on a specific moment.

  An object of the present invention is to appropriately specify a desired image.

In order to solve the above problems, the image identification apparatus according to the invention of claim 1 of the present invention, the attached that sensor more object or tool to which the subject is gripped is imaged on the imaging means, the movement of the subject obtaining means for obtaining a first information corresponding, from a plurality of images captured by the imaging unit, analyzes the image of a predetermined number corresponding to the first information, the subject from the predetermined number of image Specifying means for specifying an image corresponding to the specific movement .

The invention according to claim 12 of the present invention is an image specifying system including a transmission side terminal and a reception side terminal, wherein the transmission side terminal is a subject imaged by an imaging means or a tool held by the subject. mounted, comprising: a sensor for outputting a first information corresponding to the motion of the subject, a transmitting means for transmitting the information output by the pre-Symbol sensor to said receiving terminal, wherein the receiving terminal, based on the first information output by the sensor is transmitted in association with the transmission means, from a plurality of images captured by the imaging unit, analyzes the image of a predetermined number corresponding to the first information And specifying means for specifying an image corresponding to a specific action of the subject.

  According to the present invention, a desired image can be appropriately specified.

It is a figure which shows schematic structure of the 1st image specific system of the 1st Embodiment of this invention. It is a block diagram which shows the function structure of the 1st tool side terminal which comprises a 1st image specific system. It is a figure which shows typically the state with which the 1st tool side terminal was attached to the tennis racket. It is a figure which shows typically the state which hits a tennis ball using the tennis racket with which the 1st tool side terminal was attached. It is a block diagram which shows schematic structure of a 1st imaging device. (A) is a figure showing a posture of a user's ready position. (B) is a figure which shows the attitude | position of a user's takeback. (C) is a figure which shows a user's step-in attitude | position. (D) is a figure which shows the attitude | position of a user's ball impact. (E) is a figure which shows a user's swinging attitude | position. (F) is a figure showing a user's follow-through posture. It is a graph which shows the angular velocity with respect to time in stroke operation | movement. It is a flowchart which shows the 1st imaging | photography control process performed with a 1st tool side terminal, and the 1st imaging | photography process performed with a 1st imaging device. (A) is a figure which shows the frame image 2 frames before from an impact image. (B) is a diagram showing a frame image one frame before the impact image. (C) is a figure which shows the frame image as an impact image. (D) is a diagram showing a frame image one frame after the impact image. It is a figure which shows schematic structure of the 2nd image specific system of 2nd Embodiment. It is a flowchart which shows the 2nd imaging | photography control process performed with a 2nd tool side terminal, and the 2nd imaging | photography process performed with a 2nd imaging device. (A) is a figure which shows the change of the acceleration with respect to the time in a nice shot. (B) is a figure which shows the change of the acceleration with respect to the time by a miss shot. It is a figure which shows the folder structure of a recording medium.

  Hereinafter, first and second embodiments of the present invention will be described in detail in order with reference to the accompanying drawings. The present invention is not limited to the illustrated example.

(First embodiment)
A first embodiment according to the present invention will be described with reference to FIGS. First, with reference to FIGS. 1-4, the apparatus structure of this Embodiment is demonstrated.

  With reference to FIG. 1, an image identification system 100 according to the present embodiment will be described. FIG. 1 is a diagram showing a schematic configuration of the image specifying system 100.

  As shown in FIG. 1, the image specifying system 100 includes an image determination device that is attached and fixed to a tennis racket 300, a tool-side terminal 1 as a transmission-side terminal, and information communication via the tool-side terminal 1 and a wireless communication line. And an imaging device 2 serving as a receiving side terminal for imaging a stroke motion in which the user hits the tennis ball B using the tennis racket 300.

  Next, the tool side terminal 1 will be described with reference to FIGS. FIG. 2 is a block diagram showing a functional configuration of the tool-side terminal 1 constituting the image specifying system 100. As shown in FIG. FIG. 3 is a diagram schematically showing a state in which the equipment side terminal 1 is attached to the tennis racket 300. FIG. 4 is a diagram schematically showing a state in which the tennis ball B is hit using the tennis racket 300 to which the equipment side terminal 1 is attached.

  In the following description, one direction substantially orthogonal to the face surface of the tennis racket 300 is defined as the X-axis direction, the direction substantially orthogonal to the X-axis direction and along the extending direction of the grip portion 301 is defined as the Y-axis direction, and X One direction substantially orthogonal to the axial direction and the Y-axis direction is taken as a Z-axis direction.

  As shown in FIG. 2, the tool-side terminal 1 of the present embodiment includes a central control unit 101, a memory 102, a sensor unit 103, a time measuring unit 105, a display unit 106, a wireless processing unit 107, and an operation input. Part 108. The central control unit 101, the memory 102, the sensor unit 103, the time measuring unit 105, the display unit 106, and the wireless processing unit 107 are connected via a bus line 109.

  Moreover, as shown in FIG. 3, the terminal main body of the tool side terminal 1 is detachably provided in, for example, a tennis racket 300 as a tool for hitting the tennis ball B. Specifically, the terminal main body is attached to the inside of the shaft portion 303 between the grip portion 301 held by the user of the tennis racket 300 and the frame portion 302 constituting the face surface.

  Here, the terminal main body is attached so that the center of the terminal main body is positioned on the axis extending in the Y-axis direction along the extending direction of the grip portion 301, for example, inside the shaft portion 303. . The terminal body may be directly attached to the shaft portion 303, for example, or may be attached using a predetermined attachment jig (not shown).

  The central control unit 101 controls each unit of the tool side terminal 1. Specifically, the central control unit 101 includes, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and a tool-side terminal stored in the ROM, although not illustrated. Various control operations are performed in accordance with various processing programs for 1 (not shown). The CPU reads out various programs from the ROM and expands them in the RAM, and the central control unit 101 performs various processes in cooperation with the expanded programs and the CPU. At that time, the CPU stores various processing results in a storage area in the RAM, and displays the processing results on the display unit 106 as necessary.

  The RAM includes a program storage area for developing a processing program executed by the CPU, a data storage area for storing input data, a processing result generated when the processing program is executed, and the like. The ROM stores a program stored in the form of computer-readable program code, specifically, a system program that can be executed by the tool-side terminal 1, various processing programs that can be executed by the system program, and these various processing programs. Data used for execution is stored. It is assumed that the ROM stores at least a first shooting control program to be described later.

  The memory 102 is constituted by, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by the central control unit 101 or the like.

  The sensor unit 103 is a sensor such as a gyro sensor that detects an angular velocity of rotation about a predetermined axis of the terminal body. That is, the sensor unit 103 detects an angular velocity of rotation about a predetermined axis (for example, Z axis) of the terminal body when the user performs an action of hitting the tennis ball B using the tennis racket 300. To do. Specifically, the sensor unit 103 extends substantially parallel to a face surface (one surface) including a portion where the tennis ball B of the tennis racket 300 is hit, and is substantially orthogonal to the extending direction of the grip unit 301. An angular velocity Gz of the Z axis of rotation of the terminal body about the Z axis is detected (see FIG. 4). Then, the sensor unit 103 outputs the detected value of the Z-axis angular velocity Gz to the central control unit 101. The sensor unit 103 detects the values of the X-axis angular velocity Gx and the Y-axis angular velocity Gy together with the Z-axis angular velocity Gz and outputs them to the central control unit 101.

  FIG. 4 schematically shows a state in which the user hits the tennis ball B with the right-handed forehand using the tennis racket 300 as viewed from above the user in the Z-axis direction. A state in which the tennis ball B is hit is schematically shown as viewed in the Z-axis direction from above the user.

  Although not shown in the figure, the timer unit 105 includes, for example, a timer circuit, and measures the current time, generates time information, and outputs the time information to the central control unit 101. In addition, the time measuring part 105 is good also as time information including calendar | calendar, time information, such as a date and a day of the week.

  The display unit 106 is provided, for example, at a predetermined position on the surface of the terminal body (see FIG. 3). The display unit 106 is composed of, for example, a liquid crystal display panel and displays various types of information. For example, when the tennis ball B is hit by the tennis racket 300, the display unit 106 displays various information such as the speed and the rotation amount of the tennis ball B specified using a predetermined specifying method. May be.

  The wireless processing unit 107 performs wireless communication with the imaging device 2 connected via a predetermined wireless communication line. Specifically, the wireless processing unit 107 includes, for example, a Bluetooth (registered trademark) module (BT module) 107a, and the BT module 107a is a BT module 203a (described later) of the wireless processing unit 203 of the imaging device 2. ) Wireless communication according to the Bluetooth standard. That is, the BT module 107a performs communication setting processing called “pairing” in advance, so that the mutual device information and authentication key data are exchanged with a communication partner such as the imaging device 2 by wireless signals, and then the communication setting processing is performed. Without being performed every time, the communication partner is automatically or semi-automatically connected or disconnected.

  The wireless processing unit 107 may include, for example, a wireless LAN (Local Area Network) module and the like, and may perform wireless LAN communication with the wireless processing unit 203 of the imaging device 2.

  The operation input unit 108 includes, for example, data input keys for inputting numerical values, characters, etc., up / down / left / right movement keys for performing data selection, feeding operations, and the like, various function keys, and the like. Further, the operation input unit 108 outputs a key pressing signal pressed by the user to the central control unit 101. Note that a touch panel (not shown) may be disposed on the display screen of the display unit 106 as the operation input unit 108 and various instructions may be input according to the touch position of the touch panel.

  Next, the imaging device 2 will be described with reference to FIG. FIG. 5 is a block diagram illustrating a schematic configuration of the imaging apparatus 2.

  The image specifying system 100 includes an imaging device 2 that is arranged so as to capture the state in which the tennis ball B is hit with the tennis racket 300 from different directions. Note that a plurality of imaging devices 2 may be provided. For example, any one of the plurality of imaging devices 2 is arranged so as to be positioned behind the user who hits the tennis ball B with the tennis racket 300, while the other imaging devices 2 are It is arranged to be located on the side. The plurality of imaging devices 2 are connected so as to be able to communicate with each other via a predetermined wireless communication line, and any one imaging device 2 serves as a master for cooperative imaging, and the remaining other imaging devices 2 serve as slaves. Become. In addition, although the content of the operation differs depending on whether each imaging device 2 is a master or a slave, the configuration itself is substantially the same. Note that cooperative imaging refers to imaging in which imaging operations by a plurality of imaging devices 2 are linked.

  As illustrated in FIG. 5, the imaging device 2 includes a central control unit 201, a memory 202, a wireless processing unit 203, an imaging unit 204, an image data processing unit 205, a recording medium control unit 206, and a time measuring unit 207. And a display unit 209 and an operation input unit 210. The central control unit 201, the memory 202, the wireless processing unit 203, the imaging unit 204, the image data processing unit 205, the recording medium control unit 206, the timing unit 207, and the display unit 209 are connected via a bus line 211. .

  The central control unit 201 controls each unit of the imaging device 2. Specifically, the central control unit 201 includes, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), which are not illustrated, and the imaging device 2 stored in the ROM. Various control operations are performed in accordance with various processing programs (not shown). The CPU reads out various programs from the ROM and expands them in the RAM, and the central control unit 201 performs various processes in cooperation with the expanded programs and the CPU. At that time, the CPU stores various processing results in a storage area in the RAM, and displays the processing results on the display unit 209 as necessary.

  The RAM includes a program storage area for developing a processing program executed by the CPU, a data storage area for storing input data, a processing result generated when the processing program is executed, and the like. The ROM stores programs stored in the form of computer-readable program codes, specifically, system programs that can be executed by the imaging apparatus 2, various processing programs that can be executed by the system programs, and execution of these various processing programs. The data used for the storage is stored. It is assumed that the ROM stores at least a first photographing program described later.

  The memory 202 is composed of, for example, a DRAM or the like, and temporarily stores data processed by the central control unit 201 or the like.

  The wireless processing unit 203 performs wireless communication with an external device O such as the tool-side terminal 1 or the imaging device 2 connected via a predetermined wireless communication line. Specifically, the wireless processing unit 203 includes, for example, a Bluetooth module (BT module) 203a and a wireless LAN module 203b.

  For example, the BT module 203a performs wireless communication according to the Bluetooth standard with the BT module 107a of the tool side terminal 1 in substantially the same manner as the BT module 107a of the tool side terminal 1.

  For example, the wireless LAN module 203b is configured to directly establish a wireless communication line with the wireless LAN module 203b of the wireless processing unit 203 of another imaging apparatus 2 without going through an external access point (fixed base station). Operates in Peer (ad hoc mode). In this ad hoc mode, for example, various communication control information such as the communication method, encryption information, channel, and IP address of the wireless communication line are set in advance. The wireless LAN module 203b is in a wireless communicable range and performs wireless communication with the wireless LAN module 203b of the wireless processing unit 203 of another imaging device 2 in which common communication control information is set. .

  The imaging unit 204 images a subject. Specifically, the imaging unit 204 includes a lens unit 204a, an electronic imaging unit 204b, and an imaging control unit 204c.

  The lens unit 204a is composed of a plurality of lenses such as a zoom lens and a focus lens, for example. The electronic imaging unit 204b is composed of, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and converts an optical image that has passed through various lenses of the lens unit 204a into a two-dimensional image signal. To do. Although illustration is omitted, the imaging unit 204 may include a diaphragm that adjusts the amount of light passing through the lens unit 204a.

  The imaging control unit 204c controls imaging of the subject by the electronic imaging unit 204b. That is, the imaging control unit 204c includes a timing generator, a driver, and the like, although not shown. The imaging control unit 204c scans and drives the electronic imaging unit 204b with a timing generator and a driver, and converts an optical image formed by the lens unit 204a into a two-dimensional image signal with the electronic imaging unit 204b at predetermined intervals. Let Then, the imaging control unit 204c reads frame images for each screen from the imaging area of the electronic imaging unit 204b and causes the image data processing unit 205 to output the frame images.

  The image data processing unit 205 generates image data of the subject. That is, the image data processing unit 205 sequentially processes the frame images captured by the imaging unit 204. Specifically, the image data processing unit 205 performs an analog value signal of each frame image transferred from the electronic imaging unit 204b at a predetermined cycle (for example, 1/400 second) according to the imaging frame rate. After adjusting the gain appropriately for each color component of RGB, the sample and hold circuit (not shown) samples and holds it, converts it to digital data by an A / D converter (not shown), and the color process circuit (not shown) After performing color process processing including interpolation processing and γ correction processing, a digital luminance signal Y and color difference signals Cb, Cr (YUV data) are generated. Note that 400 fps exemplified as the imaging frame rate is an example and is not limited to this, and can be arbitrarily changed as appropriate.

  Further, the image data processing unit 205 compresses the YUV data of each frame image F according to a predetermined encoding method (for example, JPEG format) and outputs the compressed data to the recording medium control unit 206.

  The recording medium control unit 206 is configured such that the recording medium M is detachable, and controls reading of data from the loaded recording medium M and writing of data to the recording medium M. In other words, the recording medium control unit 206 uses the encoding unit (not shown) of the image data processing unit 205 to record each of the recordings encoded in a predetermined compression format (eg, JPEG format, motion JPEG format, MPEG format, etc.). The image data of the frame image is recorded in a predetermined recording area of the recording medium M. Note that the recording medium M is composed of, for example, a nonvolatile memory (flash memory).

  Although not shown, the timer unit 207 is configured to include, for example, a timer circuit, and measures the current time, generates time information, and outputs the time information to the central control unit 201. Specifically, the timer unit 207 measures the time at which each frame image was captured by the imaging unit 204 and generates imaging time information related to the time when the image was captured. Then, the timer unit 207 outputs the generated imaging time information to the central control unit 201.

  The display unit 209 includes a display panel 209a and a display control unit 209b. The display panel 209a displays an image in the display screen. Examples of the display panel 209a include a liquid crystal display panel and an organic EL display panel. However, the display panel 209a is an example and is not limited thereto.

  The display control unit 209b reads display image data temporarily stored in the memory 202, and displays a predetermined image on the display panel 209a based on the image data of a predetermined size decoded by the image data processing unit 205. Control to display on the display screen. Specifically, the display control unit 209b includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like (all not shown). The digital video encoder then outputs the luminance signal Y and the color difference signals Cb and Cr decoded by the image data processing unit 205 and stored in the VRAM from the VRAM via the VRAM controller to a predetermined reproduction frame rate (for example, 30 fps). And a video signal is generated based on these data and output to the display panel 209a.

  The operation input unit 210 is for performing a predetermined operation of the imaging apparatus 2, and includes, for example, a power button for turning on / off the power of the apparatus body, a shutter button for instructing imaging of a subject, an imaging mode, a function, and the like. And a zoom button (both not shown) related to a zoom amount adjustment instruction. Then, the operation input unit 210 outputs a predetermined operation signal to the central control unit 201 according to the operation of each button.

  Next, the operation of the image identification system 100 will be described with reference to FIGS. First, with reference to FIG. 6 and FIG. 7, the correspondence between the user's tennis stroke motion (tennis ball B striking motion) imaged by the image specifying system 100 and the angular velocity data obtained by the sensor unit 103 will be described. . FIG. 6A is a diagram illustrating the posture of the user's ready position. FIG. 6B is a diagram showing the user's take-back posture. FIG. 6C is a diagram showing the user's step-in posture. FIG. 6D is a diagram showing the ball impact posture of the user. FIG. 6E is a diagram illustrating the swinging posture of the user. FIG. 6F is a diagram illustrating the user's follow-through posture. FIG. 7 is a graph showing the angular velocity with respect to time in the stroke operation.

  Here, a case where the stroke motion of the user's tennis forehand is imaged will be described. In the stroke operation, the user grasps the tennis racket 300, the ready position in FIG. 6A, the take-back in FIG. 6B, the step-in of the left foot in FIG. 6C, and the ball impact in FIG. 6D. A series of operations for sequentially swinging forward in FIG. 6E and follow-through state in FIG.

  In the ready position, the user faces the front with respect to the direction in which the tennis ball B comes. When the user enters a system for hitting the tennis ball B, the user pulls the tennis racket 300 to take a takeback posture, steps in the left foot, and fully swings the tennis racket 300. Then, after passing through a ball impact state where the face of the tennis racket 300 and the tennis ball B come into contact, the tennis racket 300 is swung forward and then a follow-through operation is performed.

  As shown in FIG. 7, the sensor unit 103 of the device side terminal 1 outputs data on the angular velocities of the three axes of X, Y, and Z. Each state of the stroke operation can be identified by the Z-axis angular velocity Gz. In the ready position, the angular velocity Gz is almost zero. When takeback is started, the angular velocity Gz increases in the-direction and then decreases to zero, and the takeback ends. Then, when step-in is started, the angular velocity Gz increases in the + direction and then decreases to zero, and step-in ends.

  When the full swing is started, the angular velocity Gz once suddenly greatly increases in the + direction, then peaks in the + direction, and then decreases. A peak point in the + direction of the angular velocity Gz corresponds to a ball impact as an impact point between the tennis ball B and the tennis racket 300.

  Then, after the peak of the angular velocity Gz in the + direction, follow-through is started, the angular velocity Gz decreases to 0, and after passing forward, the angular velocity Gz increases in the − direction and then decreases. And the follow-through ends.

  Next, the operation of the image identification system 100 will be described with reference to FIG. FIG. 8 is a flowchart illustrating a first shooting control process executed by the tool-side terminal 1 and a first shooting process executed by the imaging apparatus 2.

  First, the 1st imaging | photography control processing performed with the tool side terminal 1 is demonstrated. The first shooting control process detects the angular velocity of the tool side terminal 1 and determines the state of movement of the subject using the detected angular velocity, and obtains a shooting start command, a shooting end command, and information on impact based on the determination. This is processing to transmit to the imaging device 2.

  A user as a subject sets the imaging device 2 in an appropriate shooting position in advance, holds the tennis racket 300 having the tool-side terminal 1, and prepares for a stroke operation. In the tool-side terminal 1, for example, a first input that is read from the internal ROM and appropriately expanded in the RAM is triggered by the input of the execution instruction of the first imaging control process from the user via the operation input unit 108. The central control unit 101 executes the first shooting control process in cooperation with the shooting control program and the CPU.

  As shown in FIG. 8, the central control unit 101 starts detection (sampling) of angular velocities Gx, Gy, and Gz by the sensor unit 103 (step S11). Then, the central control unit 101 determines whether or not the latest detected angular velocity Gz is substantially 0 in order to detect the ready position of the user (step S12). When the angular velocity Gz is not approximately 0 (step S12; NO), the process proceeds to step S12 instead of the ready position.

  When the angular velocity Gz≈0 (step S12; YES), it is a ready position, and the central control unit 101 determines that the absolute value of the latest angular velocity Gz is greater than a preset threshold value α in order to detect user takeback. Whether or not (step S13). As shown in FIG. 7, when the takeback is entered, the angular velocity Gz increases in the − direction from 0, and the threshold value α is a threshold value of the angular velocity Gz for determining the takeback. If angular velocity | Gz |> α is not satisfied (step S13; NO), the process proceeds to step S13 instead of takeback.

  When the angular velocity | Gz |> α is satisfied (step S13; YES), the central control unit 101 generates a shooting start command for instructing the start of shooting and transmits the command to the imaging device 2 via the wireless processing unit 107 ( Step S14). Then, the central control unit 101 determines whether or not the absolute value of the latest angular velocity Gz is greater than a preset threshold value β in order to detect the user's full swing (step S15). As shown in FIG. 7, when the full swing is entered, the angular velocity Gz greatly increases in the + direction. The threshold value β is a threshold value of the angular velocity Gz for determining that it is a full swing. If the angular velocity | Gz |> β is not satisfied (step S15; NO), the process proceeds to step S15 instead of a full swing.

  When the angular velocity | Gz |> β is satisfied (step S15; YES), the central control unit 101 determines whether or not the angular velocity Gz suddenly increases in the + direction and peaks in order to detect the ball impact of the user. (Step S16). In step S16, for example, the determination is made based on whether or not the gradient of increase in the angular velocity Gz has become substantially zero. When the angular velocity Gz is not a peak (step S16; NO), the process proceeds to step S16.

  When the angular velocity Gz reaches a peak (step S16; YES), the central control unit 101 acquires the current time information (sensor time information) from the time measuring unit 105 as impact time information indicating the instantaneous time of ball impact ( Step S17). Then, the central control unit 101 determines whether or not the latest angular velocity Gz is substantially 0 in order to detect the first angular velocity Gz≈0 during the user's follow-through (step S18). If the latest angular velocity Gz is not approximately 0 (step S18; NO), the process proceeds to step S18.

  When the angular velocity Gz≈0 (step S18; YES), the central control unit 101 determines whether or not the latest detected angular velocity Gz is substantially 0 for the detection of the end of the follow-through (step S19). . If the angular velocity Gz is not approximately 0 (step S19; NO), the process proceeds to step S19.

  When the angular velocity Gz≈0 is satisfied (step S18; YES), the central control unit 101 generates a shooting end command for instructing the end of shooting, and via the wireless processing unit 107 together with the information at the time of impact acquired in step S17. Is transmitted to the imaging device 2 (step S20), and the first imaging control process is terminated.

  Next, a first photographing process executed by the imaging device 2 will be described. In the first shooting process, the stroke operation of the user as the subject is continuously shot according to the shooting start command and the shooting end command received from the tool side terminal 1, and the image at the time of the impact is determined according to the information at the time of the impact. This is the process to identify.

  It is assumed that shooting condition information for continuous shooting of a user's stroke such as shooting speed [fps] and exposure is set in the imaging device 2 in advance. In the imaging device 2, for example, a first imaging program read from the internal ROM and appropriately expanded in the RAM triggered by the input of the execution instruction of the first imaging process from the user via the operation input unit 210 In cooperation with the CPU, the central control unit 201 executes the first photographing process.

  As shown in FIG. 8, the central control unit 201 determines whether or not a photographing start command has been received from the tool side terminal 1 via the wireless processing unit 203 in response to step S14 (step S31). When the imaging start command has not been received (step S31; NO), the process proceeds to step S31.

  When the shooting start command is received (step S31; YES), the central control unit 201 controls the imaging unit 204 and the image data processing unit 205 to start continuous shooting imaging (step S31). Then, the central control unit 201 controls the imaging unit 204 and the image data processing unit 205 to generate one frame image in accordance with preset shooting condition information (step S32).

  Then, the central control unit 201 acquires the current time information as imaging time information from the time measuring unit 207, controls the recording medium control unit 206, and associates the frame image generated in step S32 with the imaging time information as a recording medium. It stores in M (step S34). Then, in response to step S20, the central control unit 201 determines whether or not an imaging end command and impact information have been received from the tool-side terminal 1 via the wireless processing unit 203 (step S35). When the shooting end command and the impact information are not received (step S35; NO), the process proceeds to step S33.

  When the shooting end command and the impact information are received (step S35; YES), the central control unit 201 controls the imaging unit 204 and the image data processing unit 205 to end the imaging (step S36). Then, the central control unit 201 determines the ball impact of the stroke operation for the frame image stored in the recording medium M by image processing, and specifies the frame image as the impact image (step S37).

  In step S37, for example, the device area on the face of the tennis racket 300 in the frame image and the object area of the tennis ball B are image-analyzed, and the ratio of the area of the ball area to the area of the equipment area is maximized. The ball impact is determined, and the frame image is specified as the impact image. These area identifications will be described more specifically. The central control unit 201 performs a feature extraction process using the shape of the tennis ball B as a template with the frame image as a processing target, and extracts and specifies an object region corresponding to the tennis ball B from the frame image. Further, the central control unit 201, for example, the ratio of the number of pixels of the object region to the total number of pixels of the frame image, the ratio of the size of the frame portion 302 of the tennis racket 300 to the actual size of the tennis ball B, the frame portion 302 A tool region corresponding to the face surface of the frame portion 302 of the tennis racket 300 is extracted and specified from the frame image using a template having a shape of Note that the extraction of the object region and the tool region and the specific method are examples, and are not limited thereto, and can be arbitrarily changed as appropriate.

  In step S37, other than this, image analysis is performed on the center of the equipment area on the face of the frame portion 302 in the frame image and the center of the object area of the tennis ball B, and the distance between the two centers is minimized. It is possible to determine that the state has become a ball impact and specify the frame image as an impact image. Note that the processing in step S37 may be performed in real time every time one frame image is generated in step S33.

  The central control unit 201 assigns the impact time information received in step S35 to the impact image determined in step S37, controls the recording medium control unit 206, and uses the impact time information as sensor time information in the impact image. The information is stored in the recording medium M in association (step S38). Then, the central control unit 201 calculates and assigns sensor time information corresponding to the imaging time information of all frame images other than the impact image from the impact time information. The central control unit 201 also controls the recording medium control unit 206 to store the calculated sensor time information in the recording medium M in association with each frame image (step S39), and ends the first photographing process. Here, the frame image, the imaging time information, and the sensor time information may be stored as the same file, or may be stored as separate files.

  Here, with reference to FIG. 9, a specific example of assignment of sensor time information to the frame image in step S39 will be described. FIG. 9A shows a frame image F1 two frames before the impact image. FIG. 9B shows a frame image F2 one frame before the impact image. FIG. 9C shows a frame image F3 as an impact image. FIG. 9D shows a frame image F4 that is one frame after the impact image.

  The frame images F1, F2, F3, and F4 shown in FIGS. 9A to 9D are a part of the frame images generated in step S33, and the frame images F1, F2, F3, and F4 in step S34. It is assumed that information at the time of imaging is assigned to each. The shooting speed for continuous shooting is set to 100 [fps].

  In step S37, the frame image F3 shown in FIG. 9C is specified as an impact image, the sensor time information (impact time information) is [5: 53: 40.33], and the image pickup time information is [5: 53: 46.55]. In step S39, the sensor time information of the frame image F2 that is one frame before the impact image shown in FIG. 9B includes [5: 53: 40.33] from the impact time information [5:53:40]. 53: 40.32]. Similarly, the frame image F1 shown in FIG. 9 (a) and the frame image F4 shown in FIG. 9 (d) are respectively 20 [ms] before and 10 [ms] after the sensor information at the time of impact. Information is assigned.

  For example, by storing the angular velocity data as the output information of the sensor unit 103 and the sensor time information, the frame image stored in the recording medium M can be analyzed together with the angular velocity data with the sensor time information in common. Is possible.

  As described above, according to the present embodiment, the tool-side terminal 1 is attached to the tennis racket 300 held by the user imaged by the imaging unit 204 or the like, and is output by the sensor unit 103 that detects the movement of the user. Based on the above, the user's motion state (takeback, ball impact, follow-through) in the captured image is determined. For this reason, it is possible to appropriately specify a desired image by associating the state of the user's operation with the frame image.

  Moreover, the tool side terminal 1 determines the timing of the start and end of imaging according to the user's operation state (takeback, follow-through) based on the information output by the sensor unit 103. For this reason, the tool side terminal 1 can automatically control the start and end of shooting by transmitting a shooting start command and a shooting end command according to the start and end timing of shooting to the imaging device 2. For this reason, while being able to reduce a user's burden, only the necessary part from take back to follow-through can be imaged, and the capacity of the image to be captured can be reduced.

  The sensor unit 103 is an angular velocity sensor that detects angular velocity and outputs the information. For this reason, even in an environment where ambient noise is generated, the user's operation state can be determined without delay.

  The tool side terminal 1 is attached to a tennis racket 300 held by a user that is imaged by the imaging unit 204 and the like, and a time corresponding to detection by the sensor unit 103 that outputs information corresponding to the movement of the subject is counted. The time is measured by 105 and sensor time information is output. The information output by the sensor unit 103 and the sensor time information are associated with each other and transmitted to the imaging apparatus 2 by the wireless processing unit 107. The imaging device 2 identifies an image corresponding to a specific state of the user from the images captured by the imaging unit based on the transmitted information output by the sensor unit 103 and the sensor time information. For this reason, the desired impact image matched with the sensor time information (impact time information) can be appropriately specified according to the user's motion state (ball impact).

  In addition, the imaging device 2 determines the state of the user's operation from the image captured of the user, measures the time corresponding to the imaging of the imaging unit 204 and the like by the timing unit 207, and outputs the imaging time information. Furthermore, based on the determination result, the image information corresponding to the specific state of the user is specified by associating the sensor time information with the image capturing time information. For this reason, it is possible to associate the sensor time information and the imaging time information with respect to all the images.

(Second Embodiment)
A second embodiment according to the present invention will be described with reference to FIGS. In the first embodiment, in the shooting of the tennis stroke motion, the detected angular velocity is used to determine the state of the user's motion and the start and end of shooting based on this state. In the second embodiment, in photographing golf stroke motion, the detected motion is used to determine the state of the user's motion.

  First, an image specifying system 400 as an image determination system according to the present embodiment will be described with reference to FIG. FIG. 10 is a diagram showing a schematic configuration of the image specifying system 400.

  As shown in FIG. 10, the image specifying system 400 is connected to a tool-side terminal 1A as an image determination device that is attached and fixed to the golf club 500, and to the tool-side terminal 1A so as to be capable of information communication via a wireless communication line. The image pickup apparatus 2 </ b> A for picking up an image of a stroke operation in which the user hits the golf ball B <b> 1 using the golf club 500.

  The terminal body of the tool side terminal 1 is detachably provided on a golf club 500 as a tool for hitting the golf ball B1, for example. Specifically, the terminal body is attached to a shaft portion 503 between a grip portion 501 held by a user of the golf club 500 and a head portion 502 constituting a face surface.

  The internal functional configuration of the tool-side terminal 1A is the same as the functional configuration of the tool-side terminal 1 of the first embodiment, and the same reference numerals are given to the respective parts, and different functions will be mainly described. It is assumed that the ROM of the central control unit 101 stores at least a second shooting control program described later.

  The sensor unit 103 is assumed to be an acceleration sensor that detects a three-dimensional acceleration. The X axis is taken in the direction perpendicular to the projection direction of the face surface of the head unit 502 on the axis direction of the grip portion 501 and the face including the lower side of the face surface, and the Y axis is taken in the direction perpendicular to the X axis. The Z axis is taken in the direction perpendicular to the Y axis. The left direction in FIG. 10 is the positive direction of the X axis. The sensor unit 103 detects X-axis, Y-axis, and Z-axis accelerations gx, gy, and gz and outputs them to the central control unit 101 as sensor information.

  The internal functional configuration of the imaging device 2A is assumed to be the same as the functional configuration of the imaging device 2 of the first embodiment, and the same reference numerals are given to the respective parts, and different functions will be mainly described. It is assumed that the ROM of the central control unit 201 serving as a control unit stores at least a second imaging program described later.

  Next, the operation of the image specifying system 400 will be described with reference to FIGS. FIG. 11 is a flowchart showing a second imaging control process executed by the tool side terminal 1A and a second imaging process executed by the imaging apparatus 2A. FIG. 12A is a diagram showing a change in acceleration with respect to time in a nice shot. FIG. 12B is a diagram illustrating a change in acceleration with respect to time in a miss shot. FIG. 13 is a diagram showing a folder structure of the recording medium M.

  First, the 2nd imaging | photography control processing performed with the tool side terminal 1A is demonstrated. The second shooting control process transmits a shooting start command and a shooting end command according to a user operation to the imaging apparatus 2A, detects the acceleration of the tool-side terminal 1A, and uses the detected acceleration to determine the motion state of the subject. This is a process of determining and transmitting operation state information based on the determination to the imaging apparatus 2A.

  A user as a subject sets the imaging device 2A in an appropriate shooting position in advance, holds the golf club 500 having the tool-side terminal 1A, and prepares for a stroke operation. In the tool-side terminal 1A, for example, a second input that is read from the internal ROM and appropriately expanded in the RAM is triggered by the input of the execution instruction of the second imaging control process from the user via the operation input unit 108. The central control unit 101 executes the second shooting control process in cooperation with the shooting control program and the CPU.

  As illustrated in FIG. 11, the central control unit 101 generates a synchronization control signal for synchronizing the time measured by the time measuring unit 105 and the time measured by the time measuring unit 207 of the imaging device 2A, and transmits the wireless processing unit 107 to the imaging device 2A. (Step S41). The synchronization control signal includes, for example, current time information measured by the timer unit 105.

  Then, the central control unit 101 receives an input of club selection information from the user via the operation input unit 108 (step S42). Then, the central control unit 101 determines whether or not an instruction input for starting to shoot a moving image from the user is received via the operation input unit 108 (step S43). When there is no instruction to start shooting (step S43; NO), the process proceeds to step S43.

  When there is an instruction to start shooting (step S43; YES), the central control unit 101 starts detection (sampling) of accelerations gx, gy, and gz by the sensor unit 103 (step S44). Then, the central control unit 101 transmits the shooting start command and the club selection information input in step S42 to the imaging device 2A via the wireless processing unit 107 (step S45).

  Then, the central control unit 101 determines whether or not an instruction input for ending moving image shooting from the user has been received via the operation input unit 108 (step S46). The end of shooting is taken after the end of the user's stroke operation. If there is no instruction to end shooting (step S46; NO), the process proceeds to step S46. If there is an instruction input to end shooting (step S46; YES), the central control unit 101 analyzes the acceleration started in step S44, determines the operation state of the user who is the subject, and creates operation state information. (Step S47). The operation state information is, for example, information indicating a state type indicating whether the user has made a nice shot or a miss shot by a stroke operation.

  As shown in FIG. 12A, consider a graph in which time is taken on the horizontal axis and the distortion (change in acceleration) in the X-axis direction of the shaft portion 503 is taken on the vertical axis. The change in acceleration on the vertical axis corresponds to the acceleration gx. From the ready position, with time, the change in acceleration increases from 0 and decreases. After abruptly decreasing after 0, the peak is reversed. The portion surrounded by a circle corresponds to the impact of the head portion 502 coming into contact with the golf ball B1. After impact, the change in acceleration increases, increases through zero, decreases, and then decreases to zero.

  In the case of a nice shot, since the golf club 500 (head portion 502) hits the golf ball B1 with the core in the circled portion of the graph, the waveform of the change in acceleration takes a clean peak once. Inverted.

  As shown in FIG. 12B, in the case of a miss shot, the golf club 500 (head portion 502) does not hit the golf ball B1 with the core in the circled portion of the graph. The impact is slightly larger, and the acceleration change waveform is dirty and reversed by a plurality of peaks.

  In step S47, the types of operation states of the nice shot shown in FIG. 12A and the miss shot shown in FIG. 12B are determined according to the waveform of the acceleration gx detected during shooting. .

  However, the operation state information determined in step S47 is not limited to information indicating a nice shot and a miss shot. For example, the information may indicate information such as the flight distance of the golf ball B1, the launch angle, and where the golf ball B1 was struck (vertical direction, horizontal direction, etc.). In addition, as described above, the determination method of the operation state from the acceleration is not limited to the method of analyzing the characteristic portion (impact portion or the like) of the acceleration waveform. For example, a method for calculating an estimated value such as a flight distance from an actual measured value of acceleration, or a method by comparing an empirical acceleration pattern measured under a predetermined operation state of the user with an actually measured acceleration pattern may be used. .

  Then, the central control unit 101 transmits the shooting end command and the operation state information created in step S47 to the imaging device 2A via the wireless processing unit 107 (step S48). Then, the central control unit 101 determines whether or not an instruction input for ending the shooting of all moving images has been received from the user via the operation input unit 108 (step S49). If there is no instruction to end all shooting (step S49; NO), the process proceeds to step S42.

  When there is an instruction input to end all shooting (step S49; YES), the central control unit 101 transmits an all shooting end command to the imaging apparatus 2A via the wireless processing unit 107 (step S50). The control process ends.

  Next, a second imaging process executed by the imaging device 2A will be described. In the second shooting process, according to the shooting start command and the shooting end command received from the tool-side terminal 1A, the user's stroke operation as a subject is shot as a movie, and the movie file is associated with the operation state information. It is a process of storing in a folder of each category.

  It is assumed that shooting condition information for moving image shooting of a user's stroke such as shooting speed [fps] and exposure is set in the imaging device 2A in advance. In the imaging apparatus 2A, for example, a second imaging program read from the internal ROM and appropriately expanded in the RAM triggered by the input of the execution instruction of the second imaging process from the user via the operation input unit 210 In cooperation with the CPU, the central control unit 201 executes the second photographing process.

  As shown in FIG. 11, the central control unit 201 receives the synchronization control signal from the tool side terminal 1A via the wireless processing unit 203 in response to step S41, and sets the time measuring unit 207 according to the synchronization control signal. A synchronization process is performed in synchronization with the time measuring unit 105 (step S61). In step S61, for example, the time information measured by the time measuring unit 207 is matched with the time information in the synchronization control signal.

  Then, the central control unit 201 controls the imaging unit 204 and the image data processing unit 205 to start imaging for moving image shooting (step S62). Then, in response to step S45, the central control unit 201 determines whether or not a shooting start command and club selection information have been received from the equipment side terminal 1A via the wireless processing unit 203 (step S63). When the shooting start command and the club selection information are not received (step S63; NO), the process proceeds to step S63.

  When the shooting start command and the club selection information are received (step S63; YES), the central control unit 201 controls the imaging unit 204 and the image data processing unit 205, and the moving image frame according to the preset shooting condition information. An image is generated (step S64). Then, in response to step S48, the central control unit 201 determines whether or not the photographing end command and the operation state information are received from the tool side terminal 1A via the wireless processing unit 203 (step S65). When the photographing end command and the operation state information are not received (step S65; NO), the process proceeds to step S64.

  When the shooting end command and the operation state information are received (step S65; YES), the central control unit 201 generates the club selection information received in step S63 and the operation state information received in step S65 in step S64. It stores in a moving image file including the frame image (step S66). Then, the central control unit 201 controls the recording medium control unit 206 to store the moving image file generated in step S66 in each category folder in the recording medium M in accordance with the operation state information received in step S63 ( Step S67). Each category is a category indicating the type of operation state.

  As an example, it is assumed that a nice shot category folder R1 and a miss shot category folder R2 are formed in the recording medium M as shown in FIG. In step S67, when the operation state information in the movie file indicates a nice shot, the movie file is stored in the folder R1, and when the operation state information in the movie file indicates a miss shot, the movie file is stored in the folder R2. Stored in Note that the types of operation states and the number thereof (number of folders) are not limited to the above example.

  Then, in response to step S50, the central control unit 201 determines whether or not an all imaging end command has been received from the tool side terminal 1A via the wireless processing unit 203 (step S69). When the complete photographing end command has not been received (step S69; NO), the process proceeds to step S63.

  When the all photographing end command is received (step S69; YES), the central control unit 201 controls the imaging unit 204 and the image data processing unit 205 to end the imaging (step S70), and the second photographing process is performed. finish.

  As described above, according to the present embodiment, the tool-side terminal 1A is attached to the golf club 500 held by the user imaged by the imaging unit 204 or the like, and is output by the sensor unit 103 that detects the movement of the user. Based on the above, the state (type) of the user's operation in the captured image is determined. For this reason, it is possible to appropriately specify a desired image by associating the state (type) of the user's operation with the moving image file.

  The sensor unit 103 is an acceleration sensor that detects acceleration and outputs the information. For this reason, even in an environment where ambient noise is generated, the state (type) of the user's operation can be determined without delay.

  In addition, the tool-side terminal 1A determines the type of user action state (nice shot or miss shot) in one or more captured frame images (moving image file) based on the information output by the sensor unit 103. To do. For this reason, it is possible to appropriately specify a desired image or a plurality of images by associating the type of the user's operation state with the moving image file.

  In addition, the tool-side terminal 1A determines the type of user action state (nice shot, miss) in one or a plurality of captured frame images (moving image files) based on the waveform shape of the information output by the sensor unit 103. Shot). For this reason, it is possible to more accurately determine the type of state of the user's operation.

  In addition, the imaging apparatus 2A stores the type of the user's operation state in the determined moving image file and the moving image file in association with each other in the recording medium M as a storage unit. For this reason, a moving image file can be easily searched and read out according to the type of user's operation state.

  In addition, the imaging device 2A stores and stores the moving image file in a folder of the type of the user's operation state in the determined moving image file. For this reason, it is possible to more easily search and read out the moving image file in accordance with the type of operation state of the user.

  The present invention is not limited to the above embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.

  The first and second embodiments may be combined as appropriate. For example, a folder corresponding to the user's ready position or the like is set in the recording medium M, and frame images corresponding to each state are stored in the corresponding folder.

  In the first and second embodiments, the state of the subject is determined using the uniaxial angular velocity or acceleration. However, the present invention is not limited to this. A configuration using triaxial angular velocity or acceleration may be used.

  Further, in the first embodiment, the sensor unit 103 is configured to detect the angular velocity. However, the present invention is not limited to this, and the sensor unit 103 can detect the movement of the tennis racket 300 such as acceleration. Any sensor may be used as long as it can be detected. In the second embodiment, the sensor unit 103 is configured to detect acceleration. However, the present invention is not limited to this, and the sensor unit 103 can detect the motion of the golf club 500 such as angular velocity. Any sensor may be used as long as it is a simple sensor.

  Furthermore, in the first embodiment, the contact of the tennis ball B with the tennis racket 300 is detected as an impact. However, this is only an example and the present invention is not limited to this. Any configuration is possible as long as contact with B can be detected.

  In the first embodiment, the tool side terminal 1 performs the processing of steps S12, S13, S15, S16, S18, and S19 in FIG. 8, but the imaging device 2 performs the processing. Also good. In other words, the tool-side terminal 1 associates the time value measured by the time measuring unit 105 with the detected value of the angular velocity Gz and transmits it to the imaging device 2, and the imaging device 2 is based on the value of the angular velocity Gz and the time information. The configuration may be such that detection at the time of impact of the tennis ball B on the tennis racket 300 and determination of the start and end of shooting are performed, and the frame image associated with the shooting time information corresponding to the impact time is specified.

  In the second embodiment, the tool side terminal 1A performs the process of step S47 in FIG. 11, but the imaging apparatus 2A may perform the process. That is, the tool side terminal 1A associates the detected acceleration gx value with the time information measured by the time measuring unit 105 and transmits it to the imaging apparatus 2A, and the imaging apparatus 2A is based on the angular velocity Gz value and the time information. It may be configured to determine the state of the user's operation and associate this operation state information with the moving image file.

  Furthermore, in the said 1st Embodiment, the tennis racket 300 was illustrated as a tool, and the golf club 500 was illustrated as a tool in the said 2nd Embodiment, However, It is an example and is not restricted to this. For example, any object such as a table tennis racket or a baseball bat can be arbitrarily changed. At this time, it is preferable that the terminal main body of the tool side terminals 1 and 1A is attached and fixed on an axis extending in a direction along the extending direction of the gripping portion gripped by the user.

  Further, the configurations of the image specifying systems 100 and 400 are only examples as illustrated in the first and second embodiments, and are not limited thereto. For example, the wireless communication line has been described as an example of the predetermined communication line. good.

  In the first and second embodiments, tennis and golf sports are exemplified. However, the present invention is not limited to this, and a collision such as an accident is detected and an image at that moment is specified. The present invention can be applied to detecting a contact (collision) from photographed moving images and specifying a moving image before and after the contact (a swing form moving image for tennis and a moving image before and after a collision for an accident).

  Moreover, in the said 1st, 2nd embodiment, although the equipment side terminal 1 and 1A which have the sensor part 103 was set as the structure attached to the tennis racket 300 or the golf club 500, it is limited to this. It is not a thing. The tool side terminals 1 and 1A may be configured to be attached to the user as a subject.

  Furthermore, in the first and second embodiments, the image specifying systems 100 and 400 including the tool-side terminals 1 and 1A and the imaging devices 2 and 2A have been illustrated. It may be configured from one image specifying device.

  In addition, in the first and second embodiments, the central control unit 101 executes the first and second shooting control processes by a program, and the first and second shooting processes are performed by the central control unit. 201 is configured to be executed by a program, but is not limited thereto. The tool-side terminals 1 and 1A include a dedicated circuit such as a logic circuit that performs processing similar to the first and second imaging control processing, and the imaging devices 2 and 2A are similar to the first and second imaging processing A dedicated circuit such as a logic circuit for processing may be provided.

  Furthermore, although ROM was mentioned as a computer-readable medium which stored the program for performing each process of the said 1st, 2nd embodiment, non-volatile memories, such as flash memory other than a hard disk, It is also possible to apply a portable recording medium such as a CD-ROM. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

  In addition, it is needless to say that the detailed configuration and detailed operation of each component of the image specifying system in each of the above embodiments can be appropriately changed without departing from the spirit of the present invention.

Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and the equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
A sensor attached to a subject to be imaged by the imaging means or a tool held by the subject and outputting information corresponding to the movement of the subject;
Determination means for determining a state of operation of the subject in an image captured by the imaging means based on information output by the sensor;
An image determination apparatus comprising:
<Claim 2>
The image determination apparatus according to claim 1, wherein the determination unit determines a type of a state of movement of the subject in one or a plurality of images captured by the imaging unit based on information output from the sensor.
<Claim 3>
3. The determination unit according to claim 2, wherein the determination unit determines a type of an operation state of the subject in one or a plurality of images captured by the imaging unit based on a waveform shape of information output by the sensor. Image determination device.
<Claim 4>
The image determination apparatus according to claim 1, wherein the sensor is an acceleration sensor or an angular velocity sensor, and outputs information related to acceleration or angular velocity.
<Claim 5>
5. The image according to claim 1, further comprising: a timing specifying unit that specifies a start timing and an end timing of imaging according to an operation state of the subject based on information output from the sensor. Judgment device.
<Claim 6>
An image determination device according to any one of claims 1 to 5,
Storage control means for associating and storing in the storage means the type of motion state of the subject in the determined image and the image;
An image determination system comprising:
<Claim 7>
The image determination system according to claim 6, wherein the storage control unit stores and stores an image corresponding to the type in a folder of the type of the motion state of the subject in the determined image.
<Claim 8>
An image identification system comprising a transmission side terminal and a reception side terminal,
The sending terminal
A sensor attached to a subject imaged by the imaging means or a tool held by the subject and outputting information corresponding to the movement of the subject;
First time measuring means for measuring time corresponding to the detection of the sensor and outputting first time information;
Transmission means for associating the information output by the sensor with the first time information and transmitting the information to the receiving terminal,
The receiving terminal is
Based on the information output by the sensor transmitted in association with the transmission unit and the first time information, the image corresponding to the specific state of the subject is identified from the image captured by the imaging unit. Specific means,
An image identification system comprising:
<Claim 9>
The receiving terminal is
State determination means for determining the state of operation of the subject from an image obtained by imaging the subject;
A second time measuring means for measuring the time corresponding to the image pickup by the image pickup means and outputting second time information;
The specifying unit further specifies the image corresponding to the specific state of the subject by associating the first time information with the second time information of the image based on the determination result by the state determining unit. The image specifying system according to claim 8.
<Claim 10>
The image specifying system according to claim 8, wherein the sensor is an acceleration sensor or an angular velocity sensor, and outputs information related to acceleration or angular velocity.
<Claim 11>
A sensor attached to a subject imaged by the imaging means or a tool held by the subject and outputting information corresponding to the movement of the subject;
Time measuring means for measuring time corresponding to detection of the sensor and outputting time information;
Transmitting means for associating the information output by the sensor with the time information and transmitting the information to the receiving terminal;
A transmission terminal comprising:
<Claim 12>
An image corresponding to a specific state of the subject from an image captured by the imaging unit based on information corresponding to the movement of the subject output by the sensor and time information corresponding to the imaging of the subject by the imaging unit A receiving terminal comprising a specifying means for specifying.
<Claim 13>
A step of attaching to a subject imaged by the imaging means or a tool held by the subject and outputting information corresponding to the movement of the subject by a sensor;
Determining the state of motion of the subject in the captured image based on the information output by the sensor step.
<Claim 14>
An image identification method in an image identification system comprising a transmission side terminal and a reception side terminal,
A step of outputting information corresponding to the movement of the subject by a sensor attached to the subject imaged by the imaging means or a tool held by the subject, wherein the transmission side terminal;
The transmitting terminal timing the time corresponding to the movement of the subject and outputting time information;
Associating the information output by the sensor with the time information and transmitting the information to the receiving terminal;
The receiving side terminal specifying an image corresponding to a specific state of the subject from an image captured by the imaging unit based on information output by the sensor and time information. Method.
<Claim 15>
Computer
An output means attached to a subject to be imaged or a tool held by the subject and outputting information corresponding to the movement of the subject by a sensor;
Determination means for determining a state of operation of the subject in the captured image based on information output by the sensor;
Program to function as.

DESCRIPTION OF SYMBOLS 100,400 Image identification system 1,1A Tool side terminal 101 Central control part 102 Memory 103 Sensor part 105 Time measuring part 106 Display part 107 Wireless processing part 107a BT module 108 Operation input part 109 Bus line 2, 2A Imaging device 201 Central control part 202 Memory 203 Wireless processing unit 203a BT module 203b Wireless LAN module 204 Imaging unit 204a Lens unit 204b Electronic imaging unit 204c Imaging control unit 205 Image data processing unit 206 Recording medium control unit M Recording medium 207 Timing unit 209 Display unit 209a Display panel 209b Display control unit 210 Operation input unit 211 Bus line

Claims (18)

From attached that sensor more object or tool to which the subject is gripped is imaged on the imaging unit, an acquisition unit for acquiring first information corresponding to the motion of the subject,
A specification for analyzing a predetermined number of images corresponding to the first information among a plurality of images captured by the imaging unit and specifying an image corresponding to a specific movement of the subject from the predetermined number of images Means,
An image specifying apparatus comprising: The image specifying apparatus according to claim 1, further comprising a storage control unit that associates the first information acquired by the acquiring unit with the image specified by the specifying unit and stores the first information in the storage unit. The acquisition means acquires first time information as the time corresponding to the detection of the sensor in association with the first information,
The image specifying device according to claim 2, wherein the storage control unit stores the first time information acquired by the acquiring unit and the image specified by the image specifying unit in association with each other in the storage unit. . The storage control unit stores the first information in association with a predetermined number of images different from the specified image on the basis of the first time corresponding to the image specified by the specifying unit. The image specifying device according to claim 3, which is stored in a means. A second timing means for obtaining second time information corresponding to the imaging time of the image captured by the imaging means;
The storage control unit associates the first time information acquired by the acquisition unit, the second time information acquired by the second time measuring unit, and the image specified by the image specifying unit. The image specifying device according to claim 3 or 4, wherein the image specifying device is stored in a storage unit. The apparatus further comprises a determination unit that determines a type of the state of motion of the subject in one or a plurality of images captured by the imaging unit based on the first information acquired by the acquisition unit. Item 3. The image specifying device according to Item 1 or 2. The determination unit is configured to determine a type of an operation state of the subject in one or a plurality of images captured by the imaging unit based on a waveform shape of the first information acquired by the acquisition unit. 6. The image specifying device according to 6 . The sensor is an acceleration sensor or an angular velocity sensor, an image specifying device according to any one of claims 1 to output information about acceleration or angular velocity 6. The imaging means images a series of actions of the subject,
Selection means for selecting the predetermined number of images from images obtained by imaging a series of operations of the subject based on the first information from a plurality of images captured by the imaging means;
The image specifying device according to claim 1, further comprising: Based on the first information acquired by the acquisition unit, any one of claims 1 to 9 further comprising a timing specifying means for specifying the timing of the start and end of imaging in accordance with the state of operation of the subject The image specifying device described in 1. The specifying unit analyzes the image of the object in the predetermined number of images, according to claims 1 to identify the image corresponding from the predetermined number of images to a specific motion of the subject in any one of 10 Image identification device . An image identification system comprising a transmission side terminal and a reception side terminal,
The sending terminal
A sensor attached to a subject imaged by the imaging means or a tool held by the subject and outputting first information corresponding to the movement of the subject;
The information output by the pre-Symbol sensor and a transmitting means for transmitting to said receiving terminal,
The receiving terminal is
Based on the first information output by the sensor is transmitted in association with the transmission means, from a plurality of images captured by the imaging unit, analyzes the image of a predetermined number corresponding to the first information and a specifying means for specifying an image corresponding to the specific operation of the subject,
An image identification system comprising: The transmitting terminal is
Further comprising first timing means for timing the time corresponding to the detection of the sensor and outputting first time information;
The transmission means associates the first information output by the sensor with the first time information and transmits the first information to the receiving terminal,
The selection unit is configured to select, based on the first information output by the sensor transmitted in association with the transmission unit and the first time information, from a plurality of images captured by the imaging unit. The image specifying system according to claim 12, wherein a predetermined number of images corresponding to one piece of information is selected. The receiving terminal is
State determination means for determining the state of operation of the subject from an image obtained by imaging the subject;
A second time measuring means for measuring the time corresponding to the image pickup by the image pickup means and outputting second time information;
The specifying unit further specifies the image corresponding to the specific state of the subject by associating the first time information with the second time information of the image based on the determination result by the state determining unit. The image specifying system according to claim 13 . The image specifying system according to claim 11 , wherein the sensor is an acceleration sensor or an angular velocity sensor, and outputs information related to the acceleration or the angular velocity. Obtaining first information corresponding to the movement of the subject from a subject imaged by the imaging unit or a sensor attached to a tool held by the subject;
Analyzing a predetermined number of images corresponding to the first information from a plurality of images captured by the imaging unit, and identifying an image corresponding to a specific movement of the subject from the predetermined number of images; ,
An image specifying method including: An image identification method in an image identification system comprising a transmission side terminal and a reception side terminal,
A step of outputting first information corresponding to the movement of the subject by a sensor attached to the subject imaged by the imaging means or a tool held by the subject;
And transmitting the information output by said sensor to said receiving terminal,
Analyzing a predetermined number of images corresponding to the first information from a plurality of images captured by the imaging unit based on the first information output by the sensor transmitted in association with the transmission unit. And specifying an image corresponding to a specific operation of the subject . A computer having imaging means ;
Acquisition means for acquiring first information corresponding to a movement of the subject from a subject imaged by the imaging means or a sensor attached to a tool held by the subject;
A specifying unit that analyzes a predetermined number of images corresponding to the first information from a plurality of images captured by the imaging unit and specifies an image corresponding to a specific movement of the subject from the predetermined number of images. ,
Program to function as.