US10303712B2 - Image retrieval apparatus that retrieves desired image from consecutive images, control method therefor, and storage medium storing control program therefor - Google Patents

Image retrieval apparatus that retrieves desired image from consecutive images, control method therefor, and storage medium storing control program therefor Download PDF

Info

Publication number
US10303712B2
US10303712B2 US15/608,435 US201715608435A US10303712B2 US 10303712 B2 US10303712 B2 US 10303712B2 US 201715608435 A US201715608435 A US 201715608435A US 10303712 B2 US10303712 B2 US 10303712B2
Authority
US
United States
Prior art keywords
transition direction
transition
image
images
limit point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US15/608,435
Other languages
English (en)
Other versions
US20170351707A1 (en
Inventor
Yosato Hitaka
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Assigned to CANON KABUSHIKI KAISHA reassignment CANON KABUSHIKI KAISHA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HITAKA, YOSATO
Publication of US20170351707A1 publication Critical patent/US20170351707A1/en
Application granted granted Critical
Publication of US10303712B2 publication Critical patent/US10303712B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
    • G06K9/00765—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/005—Reproducing at a different information rate from the information rate of recording
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows

Definitions

  • the present invention relates to an image retrieval apparatus, a control method therefor, and a storage medium storing a control program therefor, and in particular, relates to an image retrieval apparatus that retrieves a desired image from consecutive images, such as a video image.
  • the image transition process may be performed outside the target frame. That is, the transition process may be performed in a direction away from the target frame (target image), which requires time until the user retrieves the target image.
  • the present invention provides an image retrieval apparatus, a control method therefor, and a storage medium storing a control program therefor that are capable of retrieving an image that a user desires in a short time.
  • a first aspect of the present invention provides an image retrieval apparatus that retrieves a predetermined image from a plurality of images shot continuously, the image retrieval apparatus including a display control unit configured to display the plurality of images on a display unit in order in either of a first transition direction and a second transition direction that is opposite to the first transition direction according to a user's operation, and a setting unit configured to set up a limit point in the first transition direction when the direction in which the images are displayed in order is switched from the first transition direction to the second transition direction based on an image displayed at a time of switching concerned, and to set up a limit point in the second transition direction when the direction in which the images are displayed in order is switched from the second transition direction to the first transition direction based on an image displayed at a time of switching concerned.
  • the display control unit stops displaying the images in order in the first transition direction when a displayed image reaches an image corresponding to the limit point in the first transition direction during the images are displayed in order in the first transition direction, and stops displaying the images in order in the second transition direction when a displayed image reaches an image corresponding to the limit point in the second transition direction during the images are displayed in order in the second transition direction.
  • a second aspect of the present invention provides a control method for an image retrieval apparatus that retrieves a predetermined image from a plurality of images shot continuously, the control method including a display control step of displaying the plurality of images on a display unit in order in either of a first transition direction and a second transition direction that is opposite to the first transition direction according to a user's operation, a first setting step of setting up a limit point in the first transition direction when the direction in which the images are displayed in order is switched from the first transition direction to the second transition direction based on an image displayed at a time of switching concerned, a second setting step of setting up a limit point in the second transition direction when the direction in which the images are displayed in order is switched from the second transition direction to the first transition direction based on an image displayed at a time of switching concerned, a first stop step of stopping displaying the images in order in the first transition direction when a displayed image reaches an image corresponding to the limit point in the first transition direction during the images are displayed in order in the first transition direction, and
  • a third aspect of the present invention provides a non-transitory computer-readable storage medium storing a control program causing a computer to execute the control method of the second aspect.
  • the limit point is set up for each transition direction when a plurality of images are displayed sequentially.
  • the display of images in the current transition direction stops.
  • an image distant from the desired image does not displayed, which enables the user to retrieve the desired image easily in a short time.
  • FIG. 1 is a block diagram schematically showing a configuration of an image retrieval apparatus according to a first embodiment of the present invention.
  • FIG. 2A and FIG. 2B are views for describing an example of an operation screen displayed on a display device of the image retrieval apparatus shown in FIG. 1 .
  • FIG. 3A is a flowchart showing a part of an image transition process for retrieving a target image frame performed by the image retrieval apparatus shown in FIG. 1 .
  • FIG. 3B is a flowchart showing the remaining part of the image transition process for retrieving the target image frame performed by the image retrieval apparatus shown in FIG. 1 .
  • FIG. 4 is a view for describing a process for setting up limit points in a forwarding direction and rewinding direction that has been described with reference to FIG. 3A and FIG. 3B .
  • FIG. 5 is a view for describing a process to stop image transition at the limit points that has been described with reference to FIG. 3A and FIG. 3B .
  • FIG. 6A , FIG. 6B , and FIG. 6C are views for describing update of the limit points that has been described with reference to FIG. 3A and FIG. 3B .
  • FIG. 7 is a view showing an example of data used in calculation of a transition speed in the image transition process described with reference to FIG. 3A and FIG. 3B .
  • FIG. 8A , FIG. 8B , and FIG. 8C are views for describing setting of limit points in an image retrieval apparatus according to a second embodiment of the present invention.
  • FIG. 9A is a flowchart showing a part of an image transition process performed by the image retrieval apparatus according to the second embodiment.
  • FIG. 9B is a flowchart showing the remaining part of the image transition process performed by the image retrieval apparatus according to the second embodiment.
  • FIG. 1 is a block diagram schematically showing a configuration of an image retrieval apparatus according to a first embodiment of the present invention.
  • the illustrated image retrieval apparatus is used when reproducing consecutive images, such as a video image, obtained by an image pickup apparatus, such as a digital camera.
  • the image retrieval apparatus has a CPU 100 that controls the entire image retrieval apparatus.
  • a hard disk drive (hereinafter, referred to as an HDD) 101 stores application programs, data, an image library, etc.
  • a ROM 102 stores operation process procedures (for example, programs for a startup process, a basic input/output process, etc. of the image retrieval apparatus) of the CPU 100 .
  • a RAM 103 is used as a main memory of the CPU 100 .
  • various programs including a control program for performing a below-mentioned process are loaded to the RAM 103 from the hard disk 101 etc., and the CPU 100 runs the programs concerned.
  • the RAM 103 is used as a work area at the time of execution of various processes by the CPU 100 .
  • An input device 105 recognizes an operation process by an operation on a screen. That is, the input device 105 is equipped with a touch panel etc. Furthermore, a pointing device, a keyboard, etc. are used as the input device 105 .
  • a detachable storage medium which stores image data etc., is attached to a storage-medium loading unit (media drive) 106 .
  • a network interface (I/F) 107 is connected with a network 110 via a communication line 109 .
  • the image retrieval apparatus receives data like image data from an external apparatus and sends data to the external apparatus via the I/F 107 .
  • the above-mentioned units are mutually connected via a system bus (an address bus, a data bus, and a control bus) 108 .
  • the image retrieval apparatus shown in FIG. 1 processes consecutive images, such as a video image including a plurality of moving image frames.
  • the consecutive images may be a plurality of static images taken by continuous shooting, and may be image data that consists of a plurality of static images taken by normal shooting and arranged in order of shot time.
  • FIG. 2A and FIG. 2B are views for describing an example of an operation screen displayed on the display device 104 of the image retrieval apparatus shown in FIG. 1 . Then, FIG. 2A is a view showing the operation screen through which an operation of instructing the image transition is received. FIG. 2B is a view showing a transition direction of the operation screen by an image transition operation.
  • the display device 104 is provided with a display panel 201 on which images, such as a video image, are displayed.
  • the display panel 201 is used as a touch panel that detects a touch operation, such as a finger operation.
  • An image such as a video image frame or a static image, is displayed in a transitional image display area 202 of the display panel 201 , for example.
  • a time line 203 is a line along which moving image frames or static images are arranged in time series.
  • a current position cursor 206 indicates a position of a video image frame displayed in the transitional image display area 202 within the entire video image. It should be noted that the current position cursor 206 indicates a position of a static image displayed in the transitional image display area 202 among all the static images that are arranged in order of shot time, for example, in the case where the processing target is a series of static images.
  • An operating body 204 is a finger or a pen, for example.
  • the CPU 100 detects transition operations, such as a flick operation, swipe operation, and drag operation, by the operating body 204 on the display panel 201 .
  • transition operations such as a flick operation, swipe operation, and drag operation
  • the CPU 100 performs transition of images displayed in the transitional image display area 202 according to a moving amount, operation frequency, and moving speed of the operating body 204 .
  • the CPU 100 performs fast transition of images in response to a flick operation.
  • the CPU 100 performs slow transition of images in response to a swipe operation or a drag operation. It should be noted that the CPU 100 changes an image transition speed corresponding to the number and speed of transition operations.
  • An arrow 205 indicates an image transition direction in the transitional image display area 202 .
  • a video image stream 207 shows a time series sequence of moving image frames that constitute a video image or static images.
  • a displayed image mark 208 indicates a position of an image displayed in the transitional image display area 202 within the video image stream 207 . Then, the displayed image mark 208 indicate an image (current position image) that is indicated by the current position cursor 206 in the time line 203 . Moreover, an arrow 209 shows a moving direction of the displayed image mark 208 . When images displayed in the transitional image display area 202 transit from right to left in FIG. 2A as shown by the arrow 209 , the displayed image mark 208 moves from a left position indicating an old image to a right position indicating a new image in time series.
  • the CPU 100 displays images in the transitional image display area 202 so as to flow from right to left.
  • the operation direction in this case is referred to as a “forwarding direction”.
  • the displayed image mark 208 on the video image stream 207 moves from left to right as shown by an arrow 212 , i.e., from a position indicating an old image toward a position indicating a new image in time series.
  • the CPU 100 displays images in the transitional image display area 202 so as to flow from left to right.
  • the operation direction in this case is referred to as a “rewinding direction”.
  • the displayed image mark 208 on the video image stream 207 moves from right to left as shown by an arrow 213 , i.e., from a position indicating a new image toward a position indicating an old image in time series.
  • the rewinding direction is opposite to the forwarding direction.
  • a user When retrieving a target image, a user operates with a flick operation so as to perform image transition at high speed. When a rough position of the target image is known, the user operates with a swipe operation so as to perform slow image transition near the target image. By transiting the images displayed in the transitional image display area 202 at high speed, the position near the target image can be reached quickly. On the other hand, when the images are transited at high speed, the user is difficult to recognize the image displayed in the transitional image display area 202 . Accordingly, since the images displayed in the transitional image display area 202 are transited slowly after reaching the rough position of the target image (predetermined image), the target image can be easily retrieved.
  • images are transited in the forwarding direction and the rewinding direction first at high speed to find the rough position of the target image. Subsequently, when the target image is approached, images are transited at low speed and the target image is easily retrieved.
  • FIG. 3A and FIG. 3B are flowcharts showing an image transition process for retrieving the target image frame performed by the image retrieval apparatus shown in FIG. 1 .
  • a limit point is set to an image frame at which a transition direction by a transition operation is reversed as mentioned later in order to retrieve a target image frame in the illustrated image transition process.
  • the CPU 100 determines whether the image transition operation was detected (step S 301 ). In the description, the CPU 100 determines whether the input device 105 received any image transition operation, for example. When an image transition operation is not detected (NO in the step S 301 ), the CPU 100 determines whether the image transition is in a temporary stop state (step S 302 ). In the description, the CPU 100 determines whether no image transition operation was received during a predetermined time period and the image displayed in the transitional image display area 202 stops (temporary stop state) without moving. That is, the CPU 100 determines whether the image displayed in the transitional image display area 202 is in the stop state during the predetermined time period.
  • the CPU 100 When the image transition is in the temporary stop state (YES in the step S 302 ), the CPU 100 records the position of the image displayed in the transitional image display area 202 within the video image to the RAM 103 as a stop position (step S 303 ). Then, the CPU 100 proceeds with the process to step S 322 mentioned later. When the image transition is not in the temporary stop state (NO in the step S 302 ), the CPU 100 proceeds with the process to step S 322 .
  • the CPU 100 determines whether the operation direction of the image transition operation (an image transition direction) is a forwarding direction (step S 304 ).
  • the CPU 100 records that the image transition direction is the forwarding direction to the RAM 103 (step S 305 ).
  • the CPU 100 determines whether the previous image transition direction is the forwarding direction (step S 306 ).
  • the CPU 100 determines that the image transition direction was changed from the rewinding direction to the forwarding direction.
  • the CPU 100 sets the stop position recorded in the step S 303 as a limit point in the rewinding direction and records it to the RAM 103 (step S 307 ).
  • the CPU 100 re-calculates transition speed (step S 308 ) as mentioned later.
  • the CPU 100 calculates a moving amount by the present image transition operation according to the number of frames of the video image (step S 309 ). It should be noted that the CPU 100 proceeds with the process to step S 309 when the previous image transition direction is the forwarding direction (YES in the step S 306 ).
  • the CPU 100 determines whether the limit point in the forwarding direction is given between the position (position within the video image) of the image that is currently displayed in the transitional image display area 202 and a position of an image that is a destination of transition according to the moving amount (step S 310 ).
  • the CPU 100 designates the image at the position corresponding to the limit point as an image displayed in the transitional image display area 202 (step S 311 ). That is, the CPU 100 moves the position of the current image to the limit point in the forwarding direction.
  • the image transition is stopped at the limit point and is brought into the temporary stop state, and the image corresponding to the limit point concerned is displayed on the transitional image display area 202 .
  • the CPU 100 brings the image transition into the stop state when the current image position is the limit point in the forwarding direction.
  • the CPU 100 When there is no limit point in the forwarding direction (NO in the step S 310 ), the CPU 100 performs transition of images in the forwarding direction according to the moving amount calculated in the step S 309 (step S 312 ). That is, the CPU 100 moves the position of the current image to the image corresponding to the image transition operation. The CPU 100 returns the process to the step S 301 after the process in the step S 311 or S 312 .
  • the CPU 100 determines whether the image transition direction is the rewinding direction (step S 313 ).
  • the CPU 100 records that the image transition direction is the rewinding direction to the RAM 103 (step S 314 ). Then, the CPU 100 determines whether the previous image transition direction is the rewinding direction (step S 315 ).
  • the CPU 100 determines that the image transition direction was changed from the rewinding direction to the forwarding direction. Then, the CPU 100 sets the stop position recorded in the step S 303 as a limit point in the forwarding direction and records it to the RAM 103 (step S 316 ). After that, the CPU 100 re-calculates transition speed (step S 317 ) as mentioned later.
  • the CPU 100 calculates a moving amount by the present image transition operation according to the number of frames of the video image (step S 318 ). It should be noted that the CPU 100 proceeds with the process to step S 318 when the previous image transition direction is the rewinding direction (YES in the step S 315 ).
  • the CPU 100 determines whether the limit point in the rewinding direction is given between the position (position within the video image) of the image that is currently displayed in the transitional image display area 202 and a position of an image that is a destination of transition according to the moving amount (step S 319 ).
  • the CPU 100 designates the image at the position corresponding to the limit point as an image displayed in the transitional image display area 202 (step S 320 ). That is, the CPU 100 moves the position of the current image to the limit point in the rewinding direction.
  • the image transition is stopped at the limit point and is brought into the temporary stop state, and the image corresponding to the limit point concerned is displayed on the transitional image display area 202 .
  • the CPU 100 brings the image transition into the stop state when the current image position is the limit point in the rewinding direction.
  • the CPU 100 When there is no limit point in the rewinding direction (NO in the step S 319 ), the CPU 100 performs transition of images in the rewinding direction according to the moving amount calculated in the step S 318 (step S 321 ). That is, the CPU 100 moves the position of the current image to the image corresponding to the image transition operation. The CPU 100 returns the process to the step S 301 after the process in the step S 320 or S 321 .
  • the CPU 100 determines whether the image transition operation has finished (step S 322 ). When there is no image transition operation within a period that is longer than the above-mentioned period for determining the temporary stop state, the CPU 100 determines that the image transition operation has finished. When the image transition operation has not finished (NO in the step S 322 ), the CPU 100 returns the process to the step S 30122 . On the other hand, when the image transition operation has finished (YES in the step S 322 ), the CPU 100 finishes the image transition process.
  • FIG. 4 is a view for describing a process for setting up limit points in the forwarding direction and rewinding direction that has been described with reference to FIG. 3A and FIG. 3B .
  • the CPU 100 shall perform the transition process for images (image frames) in the forwarding direction.
  • the CPU 100 determines that the image transition direction is the rewinding direction that is the opposite direction.
  • the CPU 100 temporary stops the transition process.
  • the CPU 100 temporary stops the image transition at the position of the displayed image mark 208 at the time when the direction of the transition operation is reversed.
  • the position of this displayed image mark 208 is a stop position recorded in the RAM 103 in the step S 303 shown in FIG. 3A .
  • the CPU 100 After the temporary stop, the CPU 100 performs the transition process of images by setting the image transition direction to the rewinding direction.
  • the CPU 100 sets up the limit point 402 in the forwarding direction in the step S 316 shown in FIG. 3B .
  • the CPU 100 determines that the image transition direction is the forwarding direction that is the opposite direction. In this case, the CPU 100 temporary stops the transition process. In the illustrated example, the CPU 100 temporary stops the image transition at the position of the displayed image mark 208 at the time when the direction of the transition operation is reversed. The position of this displayed image mark 208 is a stop position recorded in the RAM 103 in the step S 303 shown in FIG. 3A .
  • the CPU 100 After the temporary stop, the CPU 100 performs the transition process of images by setting the image transition direction to the rewinding direction.
  • the CPU 100 sets up the limit point 405 in the rewinding direction in the step S 307 shown in FIG. 3A .
  • the CPU 100 sets up the limit points 402 and 405 in the forwarding direction and the rewinding direction according to the user's image transition operations. Then, the CPU 100 performs the transition process within the range between the limit points 402 and 405 .
  • FIG. 5 is a view for describing a process to stop the image transition at the limit points that has been described with reference to FIG. 3A and FIG. 3B .
  • the CPU 100 stops the image transition process, when the image transition reaches the limit point 402 in the forwarding direction or the limit point 405 in the rewinding direction. Then, when the image transition process is stopped, the CPU 100 displays the image corresponding to the limit point 402 or 405 in the transitional image display area 202 . Then, when the image transition process is stopped at the limit point 402 in the forwarding direction, the CPU 100 does not perform the image transition in the forwarding direction beyond the image corresponding to the limit point 402 concerned henceforth. Similarly, when the image transition process is stopped at the limit point 405 in the rewinding direction, the CPU 100 does not perform the image transition in the rewinding direction beyond the image corresponding to the limit point 405 concerned henceforth.
  • FIG. 6A , FIG. 6B , and FIG. 6C are views for describing update of the limit points that has been described with reference to FIG. 3A and FIG. 3B . Then, FIG. 6A , FIG. 6B , and FIG. 6C show the update of the limit points in the video image stream.
  • the limit point 402 in the forwarding direction and the limit point 405 in the rewinding direction shall be set to the video image stream.
  • the CPU 100 performs the image transition process within a transition permission range 605 defined by the limit points 402 and 405 .
  • the transition permission range 605 means a transition range in a case where an image transition operation is given without reversing the image transition direction in the description.
  • the image transition direction shall be reversed to the rewinding direction before reaching the limit point 402 in the forwarding direction during the image transition process in the forwarding direction as shown by a reversal arrow 601 of the transition operation in FIG. 6B .
  • the CPU 100 sets up a new limit point 602 in the forwarding direction at the position of the image at which the image transition direction is reversed.
  • the CPU 100 defines a transition permission range 606 by the limit point 602 in the forwarding direction and the limit point 405 in the rewinding direction.
  • the image transition direction shall be reversed to the forwarding direction before reaching the limit point 405 in the rewinding direction during the image transition process in the rewinding direction as shown by a reversal arrow 603 of the transition operation in FIG. 6C .
  • the CPU 100 sets up a new limit point 604 in the rewinding direction at the position of the image at which the image transition direction is reversed.
  • the CPU 100 defines a transition permission range 607 by the limit point 602 in the forwarding direction and the limit point 604 in the rewinding direction.
  • the limit points will be set up and updated when the image transition direction is reversed by a user's transition operation.
  • the image transition process stops at the limit points, and the transition permission range becomes narrow by an image transition operation. That is, when the user switches the image transition direction from the forwarding direction to the rewinding direction or from the rewinding direction to the forwarding direction by a transition operation, the transition permission range is set up by the transition operation concerned. Accordingly, the image transition process that the user does not intend is not performed, and the target image will be retrieved easily.
  • FIG. 7 is a view showing an example of data used in calculation of the transition speed in the image transition process described with reference to FIG. 3A and FIG. 3B .
  • the CPU 100 calculates the transition speed according to the number of frames between the limit points. As shown in FIG. 7 , “100%” is set up as the transition speed (ratio) corresponding to a predetermined initial value (1000 or more, for example) of the number of frames between the limit points. That is, “100%” is set up as the first transition speed. Then, the transition speed ratio is set up according to the number of frames between the limit points.
  • the transition speed becomes slow as the number of frames between the limit points decreases.
  • the limit point in the rewinding direction is set at the position of the first image of the video image stream, and the limit point in the forwarding direction is set at the position of the last image of the video image stream.
  • the number of frames between the limit points i.e., the number of frames of the video image stream
  • the transition speed corresponding to the initial value concerned is set to “100%”.
  • the CPU 100 sets up the transition speed to 25% of the transition speed corresponding to the initial value.
  • transition speed is determined with using the number of frames between the limit points in the above-mentioned example, shooting time between limit points, or a ratio between a length (the number of frames or shooting time) of the entire video image stream and a length (the number of frames or shooting time) of a video image between limit points may be used.
  • the image transition speed becomes slower as the transition permission range becomes narrower, which enables the user to retrieve a desired image easily.
  • the transition permission range of the image transition is set up according to a user's transition operation in the first embodiment of the present invention, a user is able to retrieve a desired image in a short time.
  • FIG. 8A , FIG. 8B , and FIG. 8C are views for describing setting of limit points in the image retrieval apparatus according to the second embodiment of the present invention.
  • FIG. 8A is a view showing an example of limit point information.
  • FIG. 8B is a view showing an example of a video image stream.
  • FIG. 8C is a view showing another example of the limit point information.
  • the image retrieval apparatus in the second embodiment also sets up and updates limit points in the same manner as the first embodiment.
  • the CPU 100 stops the image transition process, and sets up a new limit point at the reversed position concerned. That is, the CPU 100 sets up and records a limit point in the forwarding direction and a limit point in the rewinding direction independently. Then, the number of the recorded limit points in the forwarding direction may differ from the number of the recorded limit points in the rewinding direction.
  • the limit point information includes a history of the limit points in the rewinding direction and a history of the limit points in the forwarding direction.
  • Initial values of the limit points are set up according to the number of frames of a video image stream. For example, when the number of frames of a video image stream is 20000 , the frame number 1 is set as the initial value of the limit point in the rewinding direction, and the frame number 20000 is set as the initial value of the limit point in the forwarding direction.
  • the numbers of frames at which the rewinding operations were changed to the forwarding operations are recorded as the limit points in the rewinding direction.
  • the effective limit point in the rewinding direction is updated from the frame number 3000 to the frame number 3100 .
  • the numbers of frames at which the forwarding operations were changed to the rewinding operations are recorded as the limit points in the forwarding direction.
  • the effective limit point in the forwarding direction is updated in order of the frame number 3450 , the frame number 3200 , the frame number 3125 , and the frame number 3110 .
  • a plurality of limit points are set to the video image stream (see FIG. 8B ) according to the limit point information shown in FIG. 8A .
  • the transition process stops at this limit point 803 and the transition process in the forwarding direction will not be performed beyond the frame corresponding to the limit point 803 .
  • the CPU 100 deletes the limit point 803 corresponding to the frame number 3110 and sets a limit point 804 corresponding to a last frame number 3125 as the effective limit point in the forwarding direction in the limit point information as shown in a lower video image stream in FIG. 8B .
  • the limit point 803 is deleted as mentioned above, the limit point corresponding to the frame number 3110 is deleted and the frame number 3125 is set up as the effective limit point in the limit point information as shown in FIG. 8C .
  • FIG. 9A and FIG. 9B are flowcharts showing an image transition process performed by the image retrieval apparatus according to the second embodiment of the present invention. It should be noted that the steps in the illustrated flowcharts that are identical to the steps in FIG. 3A and FIG. 3B are indicated by the same reference numbers and their descriptions are omitted.
  • the CPU 100 After the process in the step S 307 described with reference to FIG. 3A , the CPU 100 additionally records the limit point in the rewinding direction to the limit point information (step S 901 ). Then, the CPU 100 proceeds with the process to the step S 308 .
  • the CPU 100 determines whether the user's operation was given after the image transition stopped at the limit point in the forwarding direction (step S 902 ).
  • the CPU 100 deletes the limit point in the forwarding direction at which the image transition stopped last time from the limit point information (step S 903 ).
  • the CPU 100 sets the previous limit point of the limit point in the forwarding direction deleted from the limit point information (i.e., history information) as a new limit point in the forwarding direction (step S 904 ). Then, the CPU 100 proceeds with the process to the step S 308 .
  • the CPU 100 proceeds with the process to the step S 309 described with reference to FIG. 3A .
  • the CPU 100 re-calculates the transition speed in the step S 308 .
  • the CPU 100 After the process in the step S 316 , the CPU 100 additionally records the limit point in the forwarding direction to the limit point information (step S 905 ). Then, the CPU 100 proceeds with the process to the step S 317 .
  • the CPU 100 determines whether the user's operation was given after the image transition stopped at the limit point in the rewinding direction (step S 906 ).
  • the CPU 100 deletes the limit point in the rewinding direction at which the image transition stopped last time from the limit point information (step S 907 ).
  • the CPU 100 sets the previous limit point of the limit point in the rewinding direction deleted from the limit point information (i.e., history information) as a new limit point in the rewinding direction (step S 908 ). Then, the CPU 100 proceeds with the process to the step S 317 .
  • the CPU 100 proceeds with the process to the step S 318 described with reference to FIG. 3B .
  • the CPU 100 re-calculates the transition speed in the step S 317 .
  • the image retrieval apparatus is provided with the limit point information that shows the history of the limit points, and re-calculates the transition speed when the number of frames between the limit points is changed.
  • the transition speed becomes fast as the number of frames between the limit points increases, and the transition speed becomes slow as the number of frames between the limit points decreases, which enables the user to retrieve a desired image easily.
  • the CPU 100 functions as the display control unit and the setting unit in the example shown in FIG. 1 .
  • the functions of the above-mentioned embodiments may be achieved as a control method that is executed by the image retrieval apparatus.
  • the computer with which the image retrieval apparatus is provided may execute a program that has the functions of the above-mentioned embodiments.
  • the control program is recorded into a computer-readable storage medium, for example.
  • Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s).
  • computer executable instructions e.g., one or more programs
  • a storage medium which may also be referred to more fully as a
  • the computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions.
  • the computer executable instructions may be provided to the computer, for example, from a network or the storage medium.
  • the storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM), a flash memory device, a memory card, and the like.
US15/608,435 2016-06-01 2017-05-30 Image retrieval apparatus that retrieves desired image from consecutive images, control method therefor, and storage medium storing control program therefor Active US10303712B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016110034A JP6704797B2 (ja) 2016-06-01 2016-06-01 画像検索装置、その制御方法、およびプログラム
JP2016-110034 2016-06-01

Publications (2)

Publication Number Publication Date
US20170351707A1 US20170351707A1 (en) 2017-12-07
US10303712B2 true US10303712B2 (en) 2019-05-28

Family

ID=60482832

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/608,435 Active US10303712B2 (en) 2016-06-01 2017-05-30 Image retrieval apparatus that retrieves desired image from consecutive images, control method therefor, and storage medium storing control program therefor

Country Status (2)

Country Link
US (1) US10303712B2 (ja)
JP (1) JP6704797B2 (ja)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020089994A1 (en) * 2001-01-11 2002-07-11 Leach, David J. System and method of repetitive transmission of frames for frame-based communications
US6618375B2 (en) * 1999-09-13 2003-09-09 Qualcomm, Incorporated Radio link protocol frame sorting mechanism for dynamic capacity wireless data channels
US7508366B2 (en) * 2002-10-07 2009-03-24 Sharp Kabushiki Kaisha Method of driving a display, display, and computer program for the same
JP2009177507A (ja) * 2008-01-24 2009-08-06 Canon Inc 動画像再生装置及び方法、プログラム、並びに記憶媒体
US20100225952A1 (en) * 2005-09-29 2010-09-09 Fujifilm Corporation Image retrieval apparatus and image retrieval program storage medium
US8718444B2 (en) * 2010-06-16 2014-05-06 Panasonic Corporation Video search device, video search method, recording medium, program, and integrated circuit
US20160119576A1 (en) * 2014-10-22 2016-04-28 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US9454303B2 (en) * 2012-05-16 2016-09-27 Google Inc. Gesture touch inputs for controlling video on a touchscreen

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003348505A (ja) * 2002-05-27 2003-12-05 Sony Corp 情報処理装置及び情報処理方法、記憶媒体、並びにコンピュータ・プログラム
JP6299245B2 (ja) * 2014-02-05 2018-03-28 コニカミノルタ株式会社 表示装置、制御プログラム、スクロール表示方法、及び記録媒体

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6618375B2 (en) * 1999-09-13 2003-09-09 Qualcomm, Incorporated Radio link protocol frame sorting mechanism for dynamic capacity wireless data channels
US20020089994A1 (en) * 2001-01-11 2002-07-11 Leach, David J. System and method of repetitive transmission of frames for frame-based communications
US7508366B2 (en) * 2002-10-07 2009-03-24 Sharp Kabushiki Kaisha Method of driving a display, display, and computer program for the same
US20100225952A1 (en) * 2005-09-29 2010-09-09 Fujifilm Corporation Image retrieval apparatus and image retrieval program storage medium
JP2009177507A (ja) * 2008-01-24 2009-08-06 Canon Inc 動画像再生装置及び方法、プログラム、並びに記憶媒体
US8718444B2 (en) * 2010-06-16 2014-05-06 Panasonic Corporation Video search device, video search method, recording medium, program, and integrated circuit
US9454303B2 (en) * 2012-05-16 2016-09-27 Google Inc. Gesture touch inputs for controlling video on a touchscreen
US20160119576A1 (en) * 2014-10-22 2016-04-28 Canon Kabushiki Kaisha Image processing apparatus and image processing method

Also Published As

Publication number Publication date
JP2017215839A (ja) 2017-12-07
US20170351707A1 (en) 2017-12-07
JP6704797B2 (ja) 2020-06-03

Similar Documents

Publication Publication Date Title
JP5954751B2 (ja) テープ上のデータをロールバックする方法及びファイルシステム
KR102257620B1 (ko) 디스플레이 디바이스 및 그 제어 방법
KR100708135B1 (ko) 항목의 블록화를 이용한 스크롤 방법 및 장치
US9081491B2 (en) Controlling and editing media files with touch gestures over a media viewing area using a touch sensitive device
US7437674B2 (en) Video processing methods
JP6226721B2 (ja) 再生制御装置、再生制御方法、プログラム及び記憶媒体
US10356359B2 (en) Information processing apparatus, method for controlling the information processing apparatus, and recording medium
US8947464B2 (en) Display control apparatus, display control method, and non-transitory computer readable storage medium
CN105335039A (zh) 在可编辑文档的页面之间移动对象的方法及装置
JP6617041B2 (ja) プログラム、作業分析装置及び作業分析方法
US9632697B2 (en) Information processing apparatus and control method thereof, and non-transitory computer-readable medium
US10303712B2 (en) Image retrieval apparatus that retrieves desired image from consecutive images, control method therefor, and storage medium storing control program therefor
US9786317B2 (en) Recording apparatus and method of controlling recording apparatus
CN108475166B (zh) 信息处理装置及其控制方法和程序
US20200258550A1 (en) Moving image reproduction apparatus, moving image reproduction method, moving image reproduction system, and storage medium
JP2013090134A (ja) 記録装置及び記録方法
US10416869B2 (en) Information processing apparatus that scrolls and displays contents, control method therefor, and storage medium storing control program therefor
CN113687922A (zh) 任务切换的控制方法、装置及相关设备
JP2009141851A (ja) 画像記録再生装置及び画像表示方法
US10321089B2 (en) Image preproduction apparatus, method for controlling the same, and recording medium
JP6494275B2 (ja) 記録装置及び記録装置の制御方法
JP5419783B2 (ja) 画像再生装置及び画像再生装置の制御方法
JP6859061B2 (ja) 情報処理装置およびその制御方法およびプログラム
JP5284113B2 (ja) 情報処理装置、情報処理方法、プログラム及び記憶媒体
CN112567324A (zh) 一种图像速记方法、终端及计算机存储介质

Legal Events

Date Code Title Description
AS Assignment

Owner name: CANON KABUSHIKI KAISHA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HITAKA, YOSATO;REEL/FRAME:043799/0406

Effective date: 20170522

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4