JP7455598B2 - Image processing device, annotation processing device, image processing method, annotation processing method, image processing program - Google Patents

Description

The present invention relates to an image processing device, an annotation processing device, an image processing method, an annotation processing method, and an image processing program.

2. Description of the Related Art Computers and the like are required to perform processing on images to identify the positions and names of objects appearing in the images. For example, a learned estimation model generated by machine learning is used in this process. In order to train the model, it is required to prepare a large amount of training data that associates images with the positions and names of objects in the images.

Japanese Patent Application Publication No. 2007-316839 Japanese Patent Application Publication No. 2006-234492

The process of associating an image with the position (range) and name of a predetermined object in the image is called annotation process. The position of the object is represented, for example, by the position of the corner of a rectangle that includes the object in the image. In order to improve the efficiency of annotation processing, one frame of the video is manually annotated, and other frames are annotated using automatic annotation tools (CVAT (Computer Vision Annotation Tool), VoTT (Visual Object Tagging Tool), etc.). There is a method to automatically process annotations. This makes it possible to prepare a large amount of training data that associates images with the positions and names of objects, etc. in the images. However, this method has a problem in that errors accumulate and the position of the object in the image gradually deviates from the position of the detected object as the distance from the manually annotated frame increases. If machine learning is performed using training data in which the position of an object in an image differs from the position of a detected object, the quality of the trained model may deteriorate. In order to automatically perform appropriate annotation processing on images of each frame of a moving image, it is preferable to appropriately select a section of the moving image that allows highly accurate automatic annotation processing.

An object of the present invention is to provide a technique that can extract a section of a moving image in which it is easy to track an object.

In order to solve the above problem, the following means are adopted.
That is, the first aspect is
An image processing device that performs processing to identify the position and name of an object appearing in an image,
a storage unit that stores a moving image taken by a camera attached to a moving object and travel information of the moving object that is associated with the moving image;
Based on the running information, a section of the moving image during a period in which the mobile object is in a predetermined running state is extracted from the moving image in order to perform processing for specifying the position and name of an object appearing in the image. an image processing unit to
An image processing device is provided.

The disclosed aspects may be realized by a program being executed by an information processing device. That is, the disclosed configuration can be specified as a program for causing an information processing apparatus to execute the processing executed by each means in the above-described aspect, or a computer-readable recording medium on which the program is recorded. Further, the disclosed configuration may be specified by the method by which the information processing apparatus executes the processing executed by each of the above-mentioned means. The disclosed configuration may be specified as a system including an information processing device that performs the processing performed by each of the above-described means.

According to the present invention, it is possible to provide a technique that can extract a section of a moving image in which it is easy to track an object.

FIG. 1 is a diagram showing a configuration example of an image processing apparatus according to an embodiment. FIG. 2 is a diagram showing an example of the hardware configuration of the information processing device. FIG. 3 is a diagram illustrating an example of the operation flow of the image processing apparatus according to the embodiment. FIG. 4 is a diagram illustrating an example of tracking a stationary object.

Embodiments will be described below with reference to the drawings. The configuration of the embodiment is an example, and the configuration of the invention is not limited to the specific configuration of the disclosed embodiment. In implementing the invention, specific configurations depending on the embodiments may be adopted as appropriate.

[Embodiment]
(Configuration example)
FIG. 1 is a diagram showing a configuration example of an image processing apparatus according to this embodiment. The image processing device 100 includes an image processing section 102, an input section 104, an output section 106, a communication section 108, and a storage section 110. The image processing device 100 stores in the storage unit 110 a moving image captured by a camera mounted on a vehicle, etc., and driving information indicating the driving state of the vehicle at the time the moving image was captured. There is. A vehicle is an example of a moving object. The vehicle running information includes steering information indicating the steering angle of the vehicle and speed information indicating the speed of the vehicle. The moving image and travel information are acquired by the communication unit 108 from a vehicle or other information processing device via a communication network or the like. The image processing device 100 extracts a section of the moving image suitable for automatically performing annotation processing from the moving image stored in the storage unit 110. The image processing device 100 extracts a section of the moving image based on vehicle travel information. The image processing device 100 performs annotation processing on objects included in images of frames included in the extracted moving image section. Objects may include, for example, artificial objects such as signs, vehicles, and buildings, natural objects such as plants, landforms, rocks, animals, and the like. The annotation process is a process of associating an image with the position (range) and name of a predetermined object in the image. The image processing device 100 stores an image in the storage unit 110 in association with the position (range) and name of an object in the image. The name of the object may be information related to the object, such as the state and properties of the object. The annotated image and the position and name of the object can be used as training data in machine learning of a learned estimation model that performs the annotation process.

The image processing unit 102 acquires a moving image stored in the storage unit 110 and vehicle travel information associated with the moving image. The image processing unit 102 extracts a moving image taken when the vehicle is in a predetermined running state, based on vehicle running information. The image processing unit 102 allows a user or the like to input the position (range) and name of an object in one frame of the image included in the extracted moving image using the input unit 104. The position of the object is specified, for example, by (coordinates of the position of each vertex of) a rectangle containing the object. The image processing unit 102 extracts feature points included in the range of the object in the input frame image. The image processing unit 102 tracks the extracted feature points for images of other frames, and calculates a movement vector (vector indicating the direction and size of movement within the image) of the feature point of each object. The image processing unit 102 determines whether each object is a stationary object (an object that is stationary) or a rectilinear moving object (an object that is moving in a straight line) based on the movement vector of the feature point of each object. The image processing unit 102 tracks an object determined to be a stationary object or a rectilinear moving object. The image processing unit 102 performs annotation processing based on the tracking, and stores the position and name of the object in the images of other frames in the storage unit 110. The image processing unit 102 displays images of other frames and the positions and names of objects included in the images on the output unit 106, and allows the user or the like to input the confirmation results through the input unit 104.

The input unit 104 is an input means for accepting information input by a user or the like. The input unit 104 is an input device such as a keyboard or a pointing device. The input unit 104 accepts input by a user of the position (range) and name of an object included in an image displayed on the output unit 106. The input unit 104 also accepts input of a confirmation result of the annotation processing result displayed on the output unit 106.

The output unit 106 is an output unit that outputs information such as displaying information to a user or the like. The output unit 106 is, for example, a display device such as a display. The output unit 106 displays an image of a frame of a moving image, an annotation result, and the position and name of an object included in the image.

The communication unit 108 is a communication interface that communicates with other information processing devices via a communication network or the like. The communication unit 108 receives moving images and driving information from other information processing devices, etc., and stores them in the storage unit 110.

The storage unit 110 stores moving images captured by a camera mounted on a vehicle or the like in association with driving information indicating the driving state of the vehicle or the like. A moving image is a collection of still images (images) of multiple frames. The moving image stored in the storage unit 110 is a moving image captured by a camera mounted on a vehicle. A camera mounted on a vehicle is, for example, fixedly installed at the front of the vehicle and directed in the traveling direction (forward) of the vehicle. The driving information associated with the moving image is information such as the steering angle and speed acquired from the vehicle's control system using the vehicle's CAN (Control Area Network) or the like. For example, by associating time information added to a moving image with time information added to vehicle information, the driving state at the time the moving image was captured can be recognized. Furthermore, the storage unit 110 stores images, which are annotation results, and the positions and names of objects included in the images in association with each other. The driving information of the vehicle may include information about the shooting direction of the camera. Information on the photographing direction of the camera can be used when determining an epipolar constraint line or the like within the image.

FIG. 2 is a diagram showing an example of the hardware configuration of the information processing device. The information processing apparatus shown in FIG. 2 has a general computer configuration. The image processing device 100 is realized by an information processing device 90 as shown in FIG. The information processing device 90 in FIG. 2 includes a processor 91, a memory 92, a storage section 93, an input section 94, an output section 95, and a communication control section 96. These are connected to each other by a bus. The memory 92 and the storage unit 93 are computer-readable recording media. The hardware configuration of the computer is not limited to the example shown in FIG. 2, and components may be omitted, replaced, or added as appropriate.

The information processing device 90 is configured such that a processor 91 loads a program stored in a recording medium into a working area of a memory 92 and executes it, and each component is controlled through the execution of the program, thereby achieving a predetermined purpose. functions can be realized.

The processor 91 is, for example, a CPU (Central Processing Unit) or a DSP (Digital Signal Processor).

The memory 92 includes, for example, RAM (Random Access Memory) and ROM (Read Only Memory). Memory 92 is also called a main storage device.

The storage unit 93 is, for example, an EPROM (Erasable Programmable ROM) or a hard disk drive (HDD). Furthermore, the storage unit 93 can include a removable medium, that is, a portable recording medium. The removable medium is, for example, a USB (Universal Serial Bus) memory, or a disc recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The storage unit 93 is also called a secondary storage device.

The storage unit 93 stores various programs, various data, and various tables on a recording medium in a readable and writable manner. The storage unit 93 stores an operating system (OS), various programs, various tables, and the like. The information stored in the storage section 93 may also be stored in the memory 92. Further, the information stored in the memory 92 may be stored in the storage section 93.

An operating system is software that mediates between software and hardware, manages memory space, manages files, and manages processes and tasks. The operating system includes a communication interface. The communication interface is a program that exchanges data with other external devices connected via the communication control unit 96. External devices include, for example, other computers, external storage devices, and the like.

The input unit 94 includes a keyboard, a pointing device, a wireless remote control, a touch panel, and the like. Further, the input unit 94 can include a video or image input device such as a camera, or an audio input device such as a microphone.

The output unit 95 includes a display device such as an LCD (Liquid Crystal Display), an EL (Electroluminescence) panel, a CRT (Cathode Ray Tube) display, a PDP (Plasma Display Panel), and an output device such as a printer. Further, the output unit 95 can include an audio output device such as a speaker.

The communication control unit 96 connects to other devices and controls communication between the information processing device 90 and the other devices. The communication control unit 96 is, for example, a LAN (Local Area Network) interface board, a wireless communication circuit for wireless communication, or a communication circuit for wired communication. The LAN interface board and wireless communication circuit are connected to a network such as the Internet.

The computer that realizes the image processing device 100 has an image processing section 102, an input section 104, an output section 106, and a communication section 108 by a processor loading a program stored in an auxiliary storage device into the main storage device and executing it. Realize the function as On the other hand, the storage unit 110 is provided in a storage area of a main storage device or an auxiliary storage device.

(Example of operation)
3 is a diagram showing an example of an operation flow of the image processing device 100 of this embodiment. Here, it is assumed that the storage unit 110 of the image processing device 100 stores, in association with each other, video images captured by a camera mounted on a vehicle and received by the communication unit 108 or the like, and vehicle driving information when the video images were captured.

In S101, the image processing unit 102 of the image processing device 100 acquires a moving image and travel information stored in the storage unit 110. The moving image is, for example, a photograph of the direction in which the vehicle is traveling while the vehicle is traveling.

In S102, the image processing unit 102 extracts, from the moving image acquired in S101, a section of the moving image where the vehicle is running in a straight line (straight ahead). The image processing unit 102 processes a moving image captured when the steering angle is 0 degrees and the speed exceeds 0 km/h, in the driving information indicating the driving state of the vehicle associated with the moving image. Extract the interval. The fact that the steering angle is 0 degrees and the speed exceeds 0 km/h indicates that the vehicle is traveling straight without stopping. For example, in the driving information associated with the video image, if the period in which the steering angle is 0 degrees and the speed exceeds 0 km/h is from time t1 to time t2, Next, the section from time t1 to time t2 of the moving image is extracted. The extracted moving images are used as images for annotation processing. The image processing unit 102 stores the extracted sections of the moving image in the storage unit 110. If the moving image does not include a section photographed during straight forward travel, the image processing unit 102 does not extract the section of the moving image from the moving image. The image processing unit 102 may also extract a period in which the steering angle is 0 degrees and the speed is constant (constant speed) as a section of the moving image. The camera mounted on the vehicle moves at a constant speed, and the video images taken during a period when the steering angle is 0 degrees and the speed is constant are used. This makes it easier to track stationary objects and the like. If the moving image section is not extracted from the moving image, the operation flow in FIG. 3 ends.

In S103, the image processing unit 102 displays the video image extracted in S102 on the output unit 106, allows the user to select a range (region) including the object included in the image of the frame of the video image, and selects a range (area) of the object included in the image of the frame of the video image. Have the user enter the name. The range and names of multiple objects may be input for one image. The input unit 104 receives a user's input of a range including an object and the name of the object. That is, the image processing unit 102 allows the user to manually perform annotation processing on images included in the extracted moving images. The image processing unit 102 may display the image to be used when selecting the range including the object on the output unit 106 so that the image can be selected from the images of each frame of the moving image. Here, for example, the image processing unit 102 allows the user to select an image of a frame in which the size of the object is the largest for the same object included in each frame of the moving image, and Have a region (e.g., a rectangle) selected. The image processing unit 102 associates the image of the selected frame included in the moving image with the range of the object and the name of the object, and stores the image in the storage unit 110 . In addition, the image processing unit 102 extracts a range including a predetermined object from an image included in a moving image using well-known image recognition technology, associates the image, the range of the object, and the name of the object, and stores the extracted range. It may be stored in the section 110. Note that the image of the frame in which the size of the object is not necessarily the "maximum" size may be selected, but the image of the frame in which the size of the object is large enough for the user to accurately select the object may be selected.

In S104, the image processing unit 102 extracts feature points from the range of the object in the image of the frame included in the video input in S103. This feature point is called the first extracted feature point. For example, when the object is a rectangular sign, the feature point may be a corner or edge of the sign, or a point within a number, letter, or symbol within the sign. Further, the feature point is, for example, a license plate, a headlight, an emblem, a corner or edge of a windshield, a door mirror, etc. when the object is a car. One or more feature points are extracted for one object. In order to track an object more appropriately, it is desirable to extract multiple feature points for one object. Here, by selecting the image of the frame in which the object size is the largest in S103, it becomes easier for the image processing unit 102 to extract feature points.

Further, the image processing unit 102 extracts the same feature point from images of frames before and after the image of the frame including the extracted feature point. These feature points are later referred to as extracted feature points. Extraction of feature points from images of previous and subsequent frames is performed, for example, by local pattern matching near the feature points. In addition, as an algorithm for tracking feature points for images of previous and subsequent frames, we used a well-known feature point tracking algorithm (KLT method (Kanade-Lucas-Tomasi Feature Tracing).
ker) etc.) may be used. The difference between the positions of the same feature point in two frames of images is called a movement vector of the feature point. If the object is stationary, feature points are tracked in the direction of the image of the temporally previous frame. Furthermore, since the image of the frame where the object size is the largest is selected, the feature points are tracked in the direction of the image of the frame where the object size is smaller. The image processing unit 102 stores the feature points of each object in each extracted image in the storage unit 110. Note that, for example, if the camera is attached to the rear of the vehicle rather than the front, the size of the object in the image becomes smaller as time passes. In this case, the feature points may be tracked in the direction of the image of the temporally later frame. In short, it is sufficient to track the feature points in the direction in which the size of the object in the image becomes smaller.

In S105, the image processing unit 102 determines whether the object including the feature points extracted in S104 is a stationary object (an object that is stationary) or a rectilinear moving object (an object that is moving in a straight line), based on the movement vector of the feature point of each object. (object).

Generally, in a moving image captured by a camera that moves in a straight line, a stationary object moves along a predetermined straight line within the moving image. This predetermined straight line is called an epipolar constraint line. In a moving image captured by a camera that moves in a straight line, if an object including a feature point is stationary, the feature point moves on an epipolar constraint line within the image. As the vehicle travels straight, a stationary object in the image moves on the epipolar constraint line. For example, if the traveling direction of the vehicle is the center of the image (the vehicle is traveling straight toward the center of the image), the epipolar constraint line will be a straight line passing through the center of the image and the feature point. Therefore, if the vehicle is traveling straight toward the center of a moving image captured by a camera, the epipolar constraint line of each first extracted feature point will be between the first extracted feature point and the center of the image. It becomes a straight line passing through. The image processing unit 102 determines that the object including the feature point is a stationary object when there is a feature point extracted after the object on the epipolar constraint line of the feature point extracted first of the object. judge. The image processing unit 102 may determine that the object including the feature point is a stationary object when the movement vector of the feature point is on the epipolar constraint line. The image processing unit 102 stores information indicating that the object is a stationary object in the storage unit 110 in association with the range of the object. By predicting that a stationary object will move on an epipolar constraint line within an image, the accuracy of tracking the object can be improved.

Further, if the object including the feature point is a rectilinear moving object, the feature point moves on a straight line within the image. The object is, for example, another vehicle traveling straight. In a moving image captured by a camera moving in a straight line, an object moving in a straight line moves in a straight line within the image. The straight line is different from the epipolar constraint line. When the first extracted feature point and the later extracted feature point of an object exist on one straight line, the image processing unit 102 determines that the object including the feature point is a straight-moving object. . The image processing unit 102 stores information indicating that the object is a rectilinear moving object in the storage unit 110 in association with the range of the object. By predicting that a rectilinearly moving object will move in a straight line within an image, the accuracy of tracking the object can be improved.

If the object including the feature points extracted in S104 is neither a stationary object nor a rectilinear moving object, the image processing unit 102 does not perform any further processing on the object. The image processing unit 102 performs subsequent processing on the object if some of the feature points extracted after the object do not exist on the epipolar constraint line of the first extracted feature point of the object. You don't have to.

In S106, the image processing unit 102 tracks (calculates, detects) the position (range) of the object in each image, which is determined to be a stationary object or a rectilinear moving object in S105. When the object is stationary, the image processing unit 102 calculates the position of the object by assuming that the position of the object (the vertex of the rectangle containing the object) also moves on the epipolar constraint line like the feature points.

FIG. 4 is a diagram illustrating an example of tracking a stationary object. As shown in the example of FIG. 4, it is assumed that a later extracted feature point is located at a point that divides the line segment connecting the first extracted feature point and the center of the image into a:b. At this time, the image processing unit 102 calculates the position of the object including the subsequently extracted feature point by a line segment a connecting each vertex of the rectangle including the object including the first extracted feature point and the center of the image. : Calculated as a rectangle connecting the points divided into b.

Furthermore, when the object is a linearly moving object, the image processing unit 102 calculates the position of the object by assuming that the position of the object (each vertex of the rectangle containing the object) also moves on a straight line in the same way as the feature points. do. For example, assume that the position of the object including the subsequently extracted feature point is translated to the position of the later calculated feature point. Further, when a plurality of feature points are included in one object, the rectangle containing the object is enlarged or reduced in accordance with the enlargement or reduction of the interval between the feature points. The image processing unit 102 stores the position of the object in each tracked image in the storage unit 110 in association with the image.

In S107, the image processing unit 102 displays a region (rectangle) indicating the position of the object tracked in S106 on the output unit 106, overlapping the image containing the object. A user (inspector) or the like compares the object included in the image displayed on the output unit 106 and the area (rectangle) indicating the position of the object to check whether or not they are misaligned. The image processing unit 102 receives input of confirmation results (inspection results) from the user through the input unit 104 . The image processing unit 102 deletes from the storage unit 110 the image for which the input unit 104 has confirmed that the image is misaligned. The image processing unit 102 continues to store in the storage unit 110 the image for which the input unit 104 has confirmed that there is no shift. The image processing unit 102 may store in the storage unit 110 information indicating that it has been confirmed that there is no deviation, in association with the image for which the confirmation result that there is no deviation has been input by the input unit 104. . Thereby, the image processing unit 102 stores, in the storage unit 110, an image used as training data for machine learning and the position and name of an object included in the image.

(Modified example)
In the above example, in S102, the image processing unit 102 extracts a section of the video image in which the vehicle is traveling in a straight line (traveling straight ahead), but it may also extract a section of the same image in which the vehicle is stationary. . At this time, in S102, the image processing unit 102 extracts a section of the moving image captured when the speed of the driving information indicating the driving state of the vehicle associated with the moving image is 0 km/h. A speed of 0 km/h indicates that the vehicle is stopped. The extracted moving images are used as images for annotation processing. The image processing unit 102 stores the extracted sections of the moving image in the storage unit 110. At this time, in S105, the image processing unit 102 determines whether the object including the feature points extracted in S104 is a rectilinear moving object (an object moving in a straight line) based on the movement vector of the feature point of each object. Determine if there is. If the object including the feature point is a straight-moving object, the feature point moves on a straight line within the image. In a moving image captured by a stationary camera, a rectilinear moving object moves in a straight line within the image. When the first extracted feature point and the later extracted feature point of an object exist on one straight line, the image processing unit 102 determines that the object including the feature point is a straight-moving object. . By doing so, the image processing unit 102 stores in the storage unit 110 an image to be used as training data and the position and name of an object included in the image from a moving image taken from a stationary vehicle. can do.

(Actions and effects of embodiments)
The image processing device 100 stores in the storage unit 110 a moving image captured by a camera mounted on a vehicle, etc., and driving information indicating the driving state of the vehicle at the time the moving image was captured. There is. The image processing device 100 extracts, from the moving images stored in the storage unit 110, sections of moving images taken while the vehicle is traveling straight or standing still, based on the driving information of the vehicle. The section of the moving image is a section of the moving image that includes images suitable for automatic annotation processing. The image processing apparatus 100 can extract a section of a moving image that includes images suitable for automatically performing annotation processing. The image processing device 100 displays the extracted moving image on the output unit 106, and allows the user to select a range (region) that includes an object included in the frame image of the moving image, and input the name of the object. At this time, for the same object included in each frame of the moving image, the image processing device 100 selects an object area in the image of the frame in which the object size is the largest. By having the object area selected in the frame where the object size is maximum, the error between the selected area and the object can be minimized. When the object is a stationary object, the size of the object in the image becomes larger as the image is in a later frame in time. The image processing device 100 extracts feature points of the object in the selected object region (range). When the object is stationary, the image processing device 100 tracks the feature points of the object in the direction of the image of the temporally previous frame. The image processing device 100 determines that the object is stationary when the feature points of the object move on the epipolar constraint line. When determining that the object is a stationary object, the image processing apparatus 100 tracks the range of the object, assuming that the range (region, position) of the object moves on the epipolar constraint line, similar to the feature points. The image processing apparatus 100 can easily determine whether an object is a stationary object by regarding an object moving on an epipolar constraint line as a stationary object. The image processing device 100 performs learning to perform annotation processing by deleting the image and not using it as training data when the actual position of the object in the image differs from the tracked (calculated) position of the object as a result of tracking. The accuracy of the estimated model can be improved. The image processing device 100 can generate highly accurate training data for a trained estimation model that performs annotation processing from a moving image captured by a camera mounted on a vehicle.

<Computer-readable recording medium>
A program that causes a computer or other machine or device (hereinafter referred to as a computer or the like) to realize any of the above functions can be recorded on a computer-readable recording medium. Then, by causing a computer or the like to read and execute the program on this recording medium, the function can be provided.

Here, a computer-readable recording medium is a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc. means. Elements constituting a computer, such as a CPU and a memory, may be provided in such a recording medium, and the program may be caused to be executed by the CPU.

Furthermore, among such recording media, those that can be removed from computers and the like include flexible disks, magneto-optical disks, CD-ROMs, CD-R/Ws, DVDs, DATs, 8 mm tapes, and memory cards.

In addition, there are hard disks, ROMs, etc. as recording media fixed to computers and the like.

(others)
Although the embodiments of the present invention have been described above, these are merely examples, and the present invention is not limited thereto. Various modifications are possible based on the knowledge of those skilled in the art.

100: Image processing device 102: Image processing unit 104: Input unit 106: Output unit 108: Communication unit 110: Storage unit 90: Information processing device 91: Processor 92: Memory 93: Storage unit 94: Input unit 95: Output unit 96: Communication control unit

Claims (9)

An image processing device that extracts, as a processing target image, an image that undergoes annotation processing that associates the position and name of an object appearing in the image with the image,
Obtaining a moving image captured by a camera attached to a moving object and travel information of the moving object at the time the moving image was captured;
Based on the acquired traveling information, selecting a moving image of a period in which the mobile object is in a straight running state or a stopped state from among the acquired moving images,
extracting a processing target image from the selected moving image;
An image processing device including a processor. An annotation processing device that associates the position and name of an object appearing in an image with the image,
Obtaining a moving image captured by a camera attached to a moving object and travel information of the moving object at the time the moving image was captured;
Based on the acquired traveling information, selecting a moving image of a period in which the mobile object is in a straight running state or a stopped state from among the acquired moving images,
When an object in a first frame image included in the selected moving image can be tracked in a second frame image different from the first frame image, the object is associated with the first frame image. An annotation processing device comprising a processor that associates the position and name of the object with the image of the second frame. The processor includes:
performing the tracking of the position of the object in the first frame image with respect to a rectilinearly moving object in the moving image;
The annotation processing device according to claim 2. The processor includes:
When the position of the object in the second frame image based on the tracking result is different from the actual position of the object in the second frame image , the second frame image is compared to the object position and the object position in the second frame image. Delete names from stored images by associating them with each other,
The annotation processing device according to claim 2. An image processing method that extracts, as a processing target image, an image that undergoes annotation processing that associates the position and name of an object appearing in the image with the image, the method comprising:
Extracting images to be processed from moving images taken by a camera attached to a moving object, during a period in which the moving object is running in a straight line or in a stopped state;
Image processing method. An image processing method that extracts, as a processing target image, an image that undergoes annotation processing that associates the position and name of an object appearing in the image with the image, the method comprising:
Obtaining a moving image shot by a camera attached to a moving object and travel information of the moving object at the time of shooting the moving image,
Based on the acquired traveling information, selecting a moving image of a period in which the moving object is in a straight-line traveling state or a stopped state from among the acquired moving images,
extracting a processing target image from the selected video image;
Image processing method. An annotation processing method for associating the position and name of an object appearing in an image with the image, the method comprising:
Obtaining a moving image captured by a camera attached to a moving object and travel information of the moving object at the time the moving image was captured;
Based on the acquired traveling information, selecting a moving image of a period in which the mobile object is in a straight running state or a stopped state from among the acquired moving images,
When an object in a first frame image included in the selected moving image can be tracked in a second frame image different from the first frame image, the object is associated with the first frame image. An annotation processing method that associates the position and name of an object with the image of the second frame. An image processing program that extracts, as a processing target image, an image that performs annotation processing that associates the position and name of an object that appears in the image with the image,
Obtaining a moving image shot by a camera attached to a moving object and travel information of the moving object at the time of shooting the moving image,
Based on the acquired traveling information, selecting a moving image of a period in which the mobile object is in a straight running state or a stopped state from among the acquired moving images,
extracting an image to be processed from the selected video image;
An image processing program for the processor to execute. An annotation processing program that associates the position and name of an object appearing in an image with the image,
Obtaining a moving image captured by a camera attached to a moving object and travel information of the moving object at the time the moving image was captured;
Based on the acquired traveling information, selecting a moving image of a period in which the mobile object is in a straight running state or a stopped state from among the acquired moving images,
When an object in a first frame image included in the selected moving image can be tracked in a second frame image different from the first frame image, the object is associated with the first frame image. storing the position and name of the object in association with the image of the second frame;
An image processing program for the processor to execute.

Images