JP7398869B2 - Image data extraction device and image data extraction method - Google Patents

Description

The present embodiment relates to an image data extraction device and an image data extraction method for extracting a target image from a moving image composed of a plurality of consecutive frames.

2. Description of the Related Art An image data extraction device is known that extracts a target image from a moving image composed of a plurality of frames.

JP2015-69432A Japanese Patent Application Publication No. 2016-212784

In recent years, digital movies and smartphones have become widespread, and many videos are being shot. Additionally, a video may be taken by a security camera or the like. The amount of captured image data is enormous.

In recent years, image data has also been used for object analysis using machine learning. Object analysis using machine learning is performed based on a huge amount of past basic data. The greater the amount of past basic data, the more accurate the analysis can be. It is desirable that a large amount of image data be stored in advance as basic data used for object analysis using machine learning.

A video consists of multiple consecutive frames. Furthermore, the image data desired by the worker is often a portion of a screen shot as a moving image. Therefore, the image data desired by the worker is extracted as a target image by cutting out a part of the screen shot as a moving image. However, a moving image is composed of a large number of frames, and it takes a lot of effort to create a target image by cutting out a portion of the screen one by one from the large number of frames.

Therefore, when creating a target image by cutting out a portion of the screen from a large number of frames constituting a moving image, a device configured using a computer is used. However, there is a problem in that creating a target image by cutting out a portion of the screen from a large number of frames requires a lot of manual labor, making it difficult for the operator to perform the task.

In order to solve the above-mentioned problems, the present embodiment allows a worker to extract a desired image as a target image from each of a plurality of consecutive frames constituting a video with a short and simple operation. An object of the present invention is to provide an image data extraction device and an image data extraction method.

The image data extraction device of this embodiment is characterized by having the following configuration.
(1) A display unit that plays back a moving image composed of a plurality of consecutive frames.
(2) An input unit into which first coordinates and second coordinates are input into the video being played back by the display unit.
(3) The frame displayed on the display unit at the time the first coordinates are input is stored as the n-th frame, and the frame displayed on the display unit at the time the second coordinates are input is stored. A storage unit that stores the frame as the n+kth frame.
( 4 ) Create first coordinate data from the first coordinates input to the input unit, create second coordinate data from the second coordinates, and combine the first coordinate data and the second coordinates. A selection unit that selects an image inside an enclosed area surrounded by a polygon or an ellipse formed based on coordinate data as a reference image from the n+kth frame.
( 5 ) An extraction unit that extracts, as a target image, an image corresponding to the reference image from each of the plurality of continuous frames constituting the moving image based on the reference image selected by the selection unit.
(6) The storage unit stores the target image extracted by the extraction unit.

A diagram showing an image data extraction device according to the first embodiment External view of the image data extraction device according to the first embodiment A diagram showing the flow of a program of the selection unit of the image data extraction device according to the first embodiment. A diagram showing the flow of a program of the extraction unit of the image data extraction device according to the first embodiment A diagram explaining the operation related to image extraction by the image data extraction device according to the first embodiment. A diagram showing a program flow of the image adjustment unit of the image data extraction device according to the first embodiment

[1. First embodiment]
[1-1. composition]
An image data extraction device 1 as an example of this embodiment will be described with reference to FIGS. 1 and 2. The image data extraction device 1 is a device configured with a microcomputer or the like. The image data extraction device 1 is operated by an operator and used to extract image data from a moving image. The image data extraction device 1 is used to extract data for analyzing objects such as people, articles, and equipment. The extracted images and coordinate data indicating the positions of features are used as training data in machine learning.

In the image data extraction device 1, the following commands and data are input, output, and stored.
Command J1: Command to indicate point A of the image to be extracted Command J2: Command to indicate point B of the image to be extracted Command K1: Command to instruct to enlarge the enclosed area in the vertical direction Command K2: Instruct to reduce the enclosed area in the vertical direction Command K3: Command to expand the enclosed area in the horizontal direction Command K4: Command to reduce the enclosed area in the horizontal direction Command L1: Command to stop playing the video Command L2: Command to start playing the video Command L3: Command to play the video at low speed Command L4: Command to play the video at high speed Command L5: Command to rewind the video Coordinate data A: Indicates the coordinates of point A of the image to be extracted Data Coordinate data B: Data indicating the coordinates of point B of the image to be extracted Video data D1: Data of a video composed of a plurality of consecutive frames Reference image data E1: Data of the target image selected by the operator from video D1 Data Target image data F1: data of multiple target images extracted from multiple frames of video D1 The coordinates of point A in FIG. 5 are the first coordinates in the claims, and the coordinates of point B are the second coordinates in the claims. corresponds to The enclosed area is an area surrounded by a figure configured on the screen based on the coordinates of point A and point B. The image inside the enclosed area is extracted.

The image data extraction device 1 includes an input section 2, a display section 3, a calculation section 4, and a storage section 5. The calculation section 4 includes a selection section 41, an extraction section 42, and an image adjustment section 43. The functions related to image data extraction of the image data extraction device 1 are realized by an installed computer program.

(Input section 2)
The input unit 2 is an input device including a mouse 21 and a keyboard 22. The input section 2 is connected to the calculation section 4. The input unit 2 is operated by an operator to input commands J1 to J2, K1 to K4, and L1 to L5. The input unit 2 outputs the input command to the calculation unit 4.

(Display section 3)
The display unit 3 is a display device configured with a plasma display, a liquid crystal panel, or the like. The display section 3 is connected to the calculation section 4. The display unit 3 displays an image based on the moving image data D1 and the reference image data E1 output from the calculation unit 4.

(Storage unit 5)
The storage unit 5 is composed of a storage medium such as a semiconductor memory or a hard disk. The storage unit 5 is connected to the calculation unit 4. The storage unit 5 stores coordinate data A, coordinate data B, video data D1, reference image data E1, and target image data F1. The storage unit 5 is controlled for writing and reading by the calculation unit 4. The storage unit 5 is built into the console 9.

(Computation unit 4)
The calculation unit 4 is constituted by a CPU of a microcomputer or the like. The calculation section 4 includes a selection section 41, an extraction section 42, and an image adjustment section 43. The selection section 41, the extraction section 42, and the image adjustment section 43 are configured by program modules. The selection section 41, the extraction section 42, and the image adjustment section 43 incorporate a computer program to be described later. The calculation section 4 is connected to the input section 2, the display section 3, and the storage section 5. The calculation unit 4 cooperates with the input unit 2, display unit 3, and storage unit 5 to perform the following calculations and control. The calculation unit 4 is built into the console 9.

a. Calculation and Control of Selection Unit 41 The selection unit 41 receives the above-mentioned command J1 and command J2 from the input unit 2. The selection unit 41 creates coordinate data A (data indicating the coordinates of point A in the image to be extracted) and coordinate data B (data indicating the coordinates of point B in the image to be extracted) based on command J1 and command J2, and stores them in the storage unit. 5 to be memorized.

The selection unit 41 configures a rectangle on the screen based on coordinate data A regarding the coordinates of point A and coordinate data B regarding the coordinates of point B, and selects an image inside an enclosed area surrounded by the configured rectangle. is selected as a reference image from one of a plurality of consecutive frames, and reference image data E1 (data of the target image selected by the operator in the moving image D1) is created and stored in the storage unit 5. The selection unit 41 operates based on the program shown in FIG.

b. Calculation and Control of Extraction Unit 42 The extraction unit 42 extracts video data D1 (a plurality of consecutive An image corresponding to the reference image is extracted as a target image from each of a plurality of consecutive frames constituting a video related to the video data F1 (video data composed of frames), and an image corresponding to the reference image is extracted as a target image. data of a plurality of target images) is created and stored in the storage unit 5. The extraction unit 42 operates based on the program shown in FIG.

The extraction unit 42 assigns a preset name to target image data F1 (data of a plurality of target images extracted from a plurality of frames of the video D1) related to the extracted target image, and causes the storage unit 5 to store the data.

c. Calculation and Control of Image Adjustment Unit 43 The image adjustment unit 43 receives commands L1 to L5 from the input unit 2. The image adjustment section 43 causes the display section 3 to display the moving image data D1 (moving image data composed of a plurality of consecutive frames) stored in the storage section 5 based on commands L1 to L5.

The image adjustment section 43 receives commands K1 to K4 from the input section 2. The image adjustment section 43 changes the length of the enclosed area in the vertical direction or the horizontal direction based on the commands K1 to K4, and causes the display section 3 to display the changed length. The enclosed area is an area surrounded by a rectangle formed on the screen based on coordinate data A regarding the coordinates of point A and coordinate data B regarding the coordinates of point B. The image inside the enclosed area is selected by the selection unit 41 as the reference image. The image adjustment section 43 performs the above operations based on the program shown in FIG.

The above is the configuration of the image data extraction device 1.

[1-2. Effect】
Next, the operation of the image data extraction device 1 of this embodiment will be explained based on FIGS. 1 to 6. The image data extraction device 1 is used to extract data for detecting target persons, articles, and equipment and performing object analysis. The extracted images are used as training data in machine learning. The image data extraction device 1 is operated by an operator to extract image data from a moving image.

The selection unit 41 of the image data extraction device 1 selects a box surrounded by a rectangle formed based on the coordinates of point A, which is the first coordinate, and the coordinate of point B, which is the second coordinate, input to the input unit 2. An image inside the area is selected as a reference image from one of a plurality of consecutive frames constituting the moving image.

Based on the reference image selected by the selection unit 41, the extraction unit 42 of the image data extraction device 1 extracts, as a target image, an image corresponding to the reference image from each of a plurality of consecutive frames constituting the moving image.

[a. Operation of selection unit 41]
The operation of the selection section 41 will be explained below. The selection unit 41 operates according to the program shown in FIG. The program shown in FIG. 3 is built into the calculation unit 4. The program shown in FIG. 3 is repeatedly executed by the calculation unit 4.

(Step S01: Play the video)
The selection unit 41 plays back the video that is the target of target image extraction. Video data D1 (a video composed of a plurality of consecutive frames) is stored in the storage unit 5 in advance. The moving image related to the moving image data D1 is displayed on the display section 3. During playback of a video, commands J1 to J2, K1 to K4, and L1 to L5 are input from the input unit 2 by an operator who extracts a target image. In this embodiment, the commands J1 and J2 are input using the mouse 21 that constitutes the input section 2, and the commands K1 to K4 and L1 to L5 are input using the keyboard 22 that constitutes the input section 2.

(Step S02: Determine whether command J1 has been input)
Next, the selection unit 41 determines whether command J1 (command indicating point A of the image to be extracted) has been input. The worker watches the video reproduced on the display section 3 and moves the cursor displayed on the display section 3 to point A using the mouse 21. Point A is the starting point of the image desired to be extracted. The selection unit 41 detects that the left click of the mouse 21 constituting the input unit 2 is turned on, and determines that the command J1 has been input.

If it is determined that the command J1 has been input (YES in step S02), the selection unit 41 moves to step S03. If it is not determined that the command J1 has been input (NO in step S02), the process enters a state of waiting for the input of the command J1.

(Step S03: Create and store coordinate data A, and display point A)
If it is determined in step S02 that command J1 has been input, the selection unit 41 detects the coordinates of point A on the video, where command J1 has been input, and generates coordinate data A (indicating the coordinates of point A in the image to be extracted). data). The selection unit 41 causes the storage unit 5 to store the coordinate data A. Further, as shown in FIG. 5, the selection section 41 displays a dot "." at point A on the video displayed on the display section 3 based on the coordinate data A.

(Step S04: Determine whether command J2 has been input)
Next, the selection unit 41 determines whether command J2 (command indicating point B of the image to be extracted) has been input. The operator watches the video reproduced on the display section 3 and moves the cursor displayed on the display section 3 to point B using the mouse 21. Point B is the end point of the image desired to be extracted. A rectangle having opposite apex angles at point A and point B is drawn on the moving image displayed on the display unit 3. The selection unit 41 detects that the left click of the mouse 21 constituting the input unit 2 is turned off, and determines that command J2 has been input.

If it is determined that the command J2 has been input (YES in step S03), the selection unit 41 moves to step S05. If it is not determined that the command J2 has been input (NO in step S03), the process enters a state of waiting for the input of the command J2.

(Step S05: Create and store coordinate data B, and display point B and the enclosed area)
If it is determined in step S04 that command J2 has been input, the selection unit 41 detects the coordinates of point B on the video, where command J2 has been input, and generates coordinate data B (indicating the coordinates of point B of the image to be extracted). data). The selection unit 41 causes the storage unit 5 to store the coordinate data B. Further, as shown in FIG. 5, the selection unit 41 forms a rectangle with opposite apex angles at point A and point B on the video displayed on the display unit 3 based on the coordinate data A and the coordinate data B, and forms a rectangle as an enclosing area. indicate.

(Step S06: Store the frame at the time when command J2 is input)
The selection unit 41 stores frames of the video at the time when the command J2 is input. For example, as shown in FIG. 5, if command J1 is input at the time when the nth frame is displayed and command J2 is input at the time when the n+kth frame is displayed, the selection unit 41 selects whether command J2 is input or not. The storage unit 5 stores that the frame is the "n+k"th frame.

(Step S07: Create and store reference image data E1)
The selection unit 41 selects an image inside an area surrounded by a rectangle formed based on the coordinate data A and coordinate data B input to the input unit 2 from one frame out of a plurality of consecutive frames. Select as image. Coordinates related to coordinate data A correspond to first coordinates in the claims, and coordinates related to coordinate data B correspond to second coordinates in the claims.

The selection unit 41 selects, as a reference image, an image inside an enclosed area surrounded by a rectangle whose vertical angles are point A corresponding to coordinate data A and point B corresponding to coordinate data B in the "n+k"th frame. , the reference image data E1 (data of the target image selected by the operator in the moving image D1) is created and stored in the storage unit 5.

The above is the operation of the selection section 41.

[b. Operation of extraction unit 42]
The operation of the extractor 42 will be explained below. The extraction unit 42 operates according to the program shown in FIG. The program shown in FIG. 4 is built into the calculation unit 4. The program shown in FIG. 4 is repeatedly executed by the calculation unit 4.

(Step S11: Create and store target image data F1)
Based on the reference image selected by the selection unit 41, the extraction unit 42 extracts, as a target image, an image corresponding to the reference image from each of a plurality of consecutive frames constituting the moving image.

Specifically, the extraction unit 42 extracts the video data D1 based on the reference image data E1 (data of the target image selected by the operator in the video D1) created by the selection unit 41 and stored in the storage unit 5. An image corresponding to the reference image data E1 is extracted from a plurality of frames constituting (data of a moving image composed of a plurality of consecutive frames), and an image corresponding to the reference image data E1 is data of the target image) is created and stored in the storage unit 5.

(Step S12: Give an image name to the target image data F1)
The extraction unit 42 assigns a preset name to the target image data F1 (data of a plurality of target images extracted from a plurality of frames of the video D1) as a past image already stored in the storage unit 5. and is stored in the storage unit 5.

The above is the operation of the extraction section 42.

[c. Operation of image adjustment unit 43]
The operation of the image adjustment section 43 will be explained below. The image adjustment section 43 operates according to the program shown in FIG. The program shown in FIG. 6 is built into the calculation unit 4. The program shown in FIG. 6 is executed by an interrupt while the selection section 41 or the extraction section 42 is being executed. When any key on the keyboard 22 constituting the input section 2 is pressed, an interrupt occurs and the program shown in FIG. 6 is activated.

Commands K1 to K4 and L1 to L5 are input by pressing the keys shown below on the keyboard 22.
Command K1 (command to enlarge the enclosed area in the vertical direction): Key [↑]
Command K2 (command to reduce the enclosed area in the vertical direction): Key [↓]
Command K3 (command to expand the enclosed area in the horizontal direction): Key [→]
Command K4 (command to reduce the enclosed area in the horizontal direction): Key [←]
Command L1 (command to stop video playback): Key [S]
Command L2 (command to start playing the video): Key [R]
Command L3 (command to play the video at low speed): Key [T]
Command L4 (command for high-speed video playback): Key [U]
Command L5 (command to rewind the video): Key [V]

(Step S21: If command L1 is input, stop playing the video)
When the image adjustment unit 43 determines that the key [S] on the keyboard 22 has been pressed and the command L1 has been input (step S21a), the image adjustment unit 43 stops playing the video related to the video data D1 stored in the storage unit 5, The still image is displayed on the display unit 3 (step S21b).

(Step S22: If command L2 is input, play the video)
When the image adjustment unit 43 determines that the key [R] of the keyboard 22 has been pressed and the command L2 has been input (step S22a), the image adjustment unit 43 plays back the video corresponding to the video data D1 stored in the storage unit 5, and displays the video on the display unit. 3 (step S22b).

(Step S23: If command L3 is input, play the video at low speed)
When the image adjustment unit 43 determines that the key [T] of the keyboard 22 has been pressed and the command L3 has been input (step S23a), the image adjustment unit 43 lowers the playback speed and displays the video corresponding to the video data D1 on the display unit 3. (Step S23b). The image adjustment unit 43 slows down the playback speed of the currently playing video by, for example, 20%. If command L3 is input multiple times, the playback speed of the video is cumulatively slowed down.

(Step S24: If command L4 is input, perform high-speed playback of the video)
When the image adjustment unit 43 determines that the key [U] of the keyboard 22 has been pressed and the command L4 has been input (step S24a), the image adjustment unit 43 increases the playback speed and displays the video corresponding to the video data D1 on the display unit 3. (Step S24b). The image adjustment unit 43 increases the playback speed of the currently played video by, for example, 20%. When command L3 is input multiple times, the video playback speed is cumulatively increased.

(Step S25: If command L5 is input, rewind and play the video)
When the image adjustment unit 43 determines that the key [V] on the keyboard 22 has been pressed and the command L5 has been input (step S25a), the image adjustment unit 43 causes the display unit 3 to display the video related to the video data D1 in rewind playback ( Step S25b). If the command L3 is input multiple times, the rewind playback speed of the video is cumulatively increased.

The reference image data E1 (data of the target image selected by the operator in the moving image D1) is created on the moving image adjusted by the operator in steps S21 to S25 described above.

(Step S26: If command K1 is input, expand the vertical length of the enclosed area)
When the image adjustment unit 43 determines that the key [↑] on the keyboard 22 has been pressed and the command K1 has been input (step S26a), the image adjustment unit 43 enlarges the length of the enclosed area in the vertical direction (step S26b). The enclosed area is an area surrounded by a rectangle formed based on the coordinates of coordinate data A and coordinate data B. The enclosed area is displayed on the moving image played back on the display unit 3. The image adjustment unit 43 increases the length of the currently displayed enclosed area in the vertical direction by, for example, 10 pixels. If the command K1 is input multiple times, the length of the enclosed area in the vertical direction is cumulatively expanded.

(Step S27: If command K2 is input, reduce the vertical length of the enclosed area)
When the image adjustment unit 43 determines that the key [↓] on the keyboard 22 has been pressed and the command K2 has been input (step S27a), the image adjustment unit 43 reduces the length of the enclosed area in the vertical direction (step S27b). The image adjustment unit 43 reduces the length of the currently displayed enclosed area in the vertical direction by, for example, 10 pixels. If the command K2 is input multiple times, the length of the enclosed area in the vertical direction is cumulatively reduced.

(Step S28: If command K3 is input, expand the horizontal length of the enclosed area)
When the image adjustment unit 43 determines that the key [→] on the keyboard 22 has been pressed and the command K3 has been input (step S28a), the image adjustment unit 43 enlarges the horizontal length of the enclosed area (step S28b). The image adjustment unit 43 increases the horizontal length of the currently displayed enclosed area by, for example, 10 pixels. If the command K3 is input multiple times, the horizontal length of the enclosed area is cumulatively expanded.

(Step S29: If command K4 is input, reduce the horizontal length of the enclosed area)
When the image adjustment unit 43 determines that the key [←] on the keyboard 22 has been pressed and the command K4 has been input (step S29a), the image adjustment unit 43 reduces the horizontal length of the enclosed area (step S29b). The image adjustment unit 43 reduces the horizontal length of the currently displayed enclosed area by, for example, 10 pixels. If command K4 is input multiple times, the horizontal length of the enclosed area is cumulatively reduced.

The reference image data E1 (data of the target image selected by the operator in the moving image D1) is created by adjusting the size of the enclosed area in steps S26 to S29 described above.

The above is the operation of the image adjustment section 43.

The above is the operation of the image data extraction device 1. As described above, a boxed area is displayed on the video corresponding to the video data D1 displayed on the display unit 3 of the image data extraction device 1, and based on the boxed area, the reference image data E1 (out of the video D1 selected by the operator) is displayed. data of the target image) is created. Based on the reference image data E1, target image data F1 (data of multiple target images extracted from multiple frames of the video D1) is created.

[1-3. effect]
(1) According to the present embodiment, the image data extraction device 1 includes a display unit 3 that plays back a moving image composed of a plurality of consecutive frames, and a first coordinate and a first coordinate in the moving image played by the display unit 3. 2 coordinates are input, and an image inside an enclosed area surrounded by a polygon or an ellipse configured based on the first and second coordinates input to the input unit 2. A selection unit 41 selects one frame from among a plurality of consecutive frames as a reference image, and a selection unit 41 selects a reference image from each of the plurality of consecutive frames constituting a video based on the reference image selected by the selection unit 41. Since it has an extraction unit 42 that extracts a corresponding image as a target image, and a storage unit 5 that stores the target image extracted by the extraction unit 42, an operator can easily extract a plurality of consecutive images that make up a moving image. It is possible to provide an image data extraction device that can extract a desired image as a target image from each frame.

According to this embodiment, the worker can input the first coordinate and the second coordinate from the input unit 2 on the video displayed on the display unit 3 and select a desired image as the reference image. Therefore, the reference image can be selected with a simple task. The operator is not required to freeze the video and individually select reference images from the still images.

According to the present embodiment, the extraction unit 42 extracts an image corresponding to the reference image as a target image from each of a plurality of consecutive frames constituting a video based on the reference image selected by the selection unit 41. , the operator is not required to freeze the video and select multiple images from the still images. As a result, a large amount of image data can be extracted in a short time. The extracted large amount of image data is used for object analysis using machine learning.

(2) According to the present embodiment, the length of the enclosed area in the vertical or horizontal direction is changed based on the command input to the input unit 2, so the operator can easily determine the size of the reference image. Can be changed.

(3) According to the present embodiment, the playback speed of a plurality of consecutive frames constituting a video is changed based on the command input to the input unit 2, so the operator can play the video at any playback speed. The reference image can be selected on the video. This makes it easier for the operator to select the reference image.

(4) According to the present embodiment, the target image is given the name of the most similar image among the images stored in the past and is stored in the storage unit 5. Therefore, the operator can easily store the target image in the storage unit 5. You can know the contents of the data. Further, similar target images are given similar names and stored in the storage unit 5, so that the operator can easily classify the data. Furthermore, the operator can rely on the name of the target image to extract data and use it to understand the temporal history.

(5) According to the present embodiment, the input unit 2 includes at least one of the mouse 21, the keyboard 22, the voice input device, and the eye tracking device. The image data extraction device 1 can be configured by the devices.

[2. Other embodiments]
Although embodiments including modifications have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention as well as within the scope of the invention described in the claims and its equivalents. Below is an example.

(1) In the above embodiment, the input unit 2 is configured with a mouse 21 and a keyboard 22, and the commands J1 to J2 are inputted by the mouse 21, and the commands K1 to K4 and L1 to L5 are inputted by the keyboard 22. However, these commands may also be input through the input unit 2 configured with another input device. For example, the input unit 2 may be configured to include a voice input device, and the following commands may be input to the voice input device by voice.
Command K1 (instruction to expand the enclosed area in the vertical direction): Voice [up]
Command K2 (instruction to reduce the enclosed area in the vertical direction): Voice [Shita]
Command K3 (instruction to expand the enclosed area in the horizontal direction): Voice [Migi]
Command K4 (command to reduce the enclosed area in the horizontal direction): Voice [hidari]
Command L1 (command to stop video playback): Audio [pause]
Command L2 (command to start playing the video): Audio [play]
Command L3 (command to play video at low speed): Audio [low speed]
Command L4 (command for high-speed video playback): Audio [high-speed]
Command L5 (command to rewind the video): Audio [Rewind]

(2) In the above embodiment, command J1 (instruction indicating point A of the image to be extracted) and command J2 (instruction indicating point B of the image to be extracted) is executed by the mouse 21 of the input unit 2 operated by the operator. input. However, these commands may also be input through the input unit 2 configured with another input device. For example, the input unit 2 may include an eye tracking device, and the commands J1 and J2 may be input to the eye tracking device depending on the direction of the worker's eye.

(3) In the above embodiment, the enclosed area is a rectangle constructed based on the coordinates of point A and point B. However, the shape of the enclosed area is not limited to this. The shape of the enclosing area may be, for example, a triangle, a polygon of pentagon or more, or an ellipse, which is configured based on the coordinates of point A and point B.

(4) In the above embodiment, the reference image data E1 is created from the frame of the video at the time when the command J2 is input, but the frame of the video from which the reference image data E1 is created is not limited to this. The reference image data E1 may be created from the frame of the moving image into which the command J1 is input. Alternatively, frames of the moving image between commands J1 and J2 may be selected to create the reference image data E1.

1... Image data extraction device 2... Input unit 3... Display unit 4... Arithmetic unit 5... Storage unit 9... Console 21... Mouse 22... Keyboard 41... - Selection unit 42...Extraction unit 43...Image adjustment unit

Claims (6)

a display unit that plays a video composed of a plurality of consecutive frames;
an input unit into which first coordinates and second coordinates are input into the video being played back by the display unit;
The frame displayed on the display unit at the time the first coordinates were input is stored as the nth frame, and the frame displayed on the display unit at the time the second coordinates were input is stored. a storage unit that stores the n+kth frame;
Create first coordinate data from the first coordinates input to the input unit, create second coordinate data from the second coordinates, and combine the first coordinate data and the second coordinate data. a selection unit that selects an image inside an enclosed area surrounded by a polygon or an ellipse configured based on the above as a reference image from the n+kth frame;
an extraction unit that extracts an image corresponding to the reference image as a target image from each of the plurality of consecutive frames forming the moving image based on the reference image selected by the selection unit;
has
the storage unit stores the target image extracted by the extraction unit;
Image data extraction device. The length of the enclosed area in the vertical or horizontal direction is changed based on a command input to the input section.
The image data extraction device according to claim 1. The playback speed of the plurality of consecutive frames constituting the video is changed based on a command input to the input unit.
The image data extraction device according to claim 1 or 2. The target image is given a name of the most similar image among the images stored in the past and is stored in the storage unit.
An image data extraction device according to any one of claims 1 to 3. The input unit includes at least one of a mouse, a keyboard, a voice input device, and an eye tracking device.
An image data extraction device according to any one of claims 1 to 4. an input procedure of inputting first coordinates and second coordinates to a video being played by a display unit that plays a video composed of a plurality of consecutive frames;
The frame displayed on the display unit at the time the first coordinates were input is stored as the nth frame, and the frame displayed on the display unit at the time the second coordinates were input is stored. a storage procedure for storing as the n+kth frame;
Create first coordinate data from the first coordinates input by the input procedure, create second coordinate data from the second coordinates, and combine the first coordinate data and the second coordinate data. a selection procedure of selecting an image inside an enclosed area surrounded by a polygon or an ellipse configured based on the above as a reference image from the n+kth frame;
an extraction step of extracting an image corresponding to the reference image as a target image from each of the plurality of consecutive frames constituting the moving image based on the reference image selected by the selection step;
has
The storage step stores the target image extracted by the extraction step.
Image data extraction method.

Images