JP2009231988A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To view and listen to a specific object while expanding it without increasing the operation burden on a user. <P>SOLUTION: A display section 22 displays a moving image having a frame image including a specific object. An operating section 26 receives an indication from a user. During or immediately after rewind reproduction of the moving image, an image enlargement section 28 expands the specific object in a frame image after a frame position where the operating section 26 received a normal indication of reproduction from the user, and displays an expanded specific image at the display section 22. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that processes a moving image.

  Digital movie cameras that allow general users to easily shoot moving images have become widespread. At the same time, players that play back moving images captured by such cameras have become widespread.

Patent Document 1 discloses an operation history display device. The operation history display device includes a reproducing unit that reproduces data in accordance with a predetermined operation mode, a time measuring unit that measures an elapsed time from the start of operation, a reproduction position of data by the reproducing unit, and a progress from the start of the operation. Display means for plotting time in a table with XY axes.
JP 2006-252678 A

  A general user can use a player to view a moving image captured by himself or another person with a digital movie camera. When viewing a moving image shot with a specific object as a target of interest, the main purpose is of course to view that object.

  Among the sections where the object is shown, especially when there is a favorite scene, some users rewind to the front of the scene, enlarge a specific object in the scene, and watch it again. Yes. In order to perform this processing, first, a rewinding of the moving image being reproduced is instructed, then a normal reproduction of the moving image being rewound is instructed, and then an enlarged reproduction is instructed.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing apparatus capable of enlarging and viewing a specific object while reducing a user's operation burden.

  An image processing apparatus according to an aspect of the present invention includes a display unit that displays a moving image having a frame image including a specific object, an operation unit that receives an instruction from a user, during rewinding reproduction of the moving image, or immediately after the stop. And an image enlargement unit that enlarges a specific object and displays it on the display unit in a frame image after the frame position at which the operation unit accepts a normal reproduction instruction from the user.

  Another embodiment of the present invention is also an image processing apparatus. This apparatus includes a display unit that displays a moving image having a frame image including a specific object, an operation unit that receives an instruction from a user, and an operation unit that is enlarged from the user during or immediately after the reproduction of the moving image. The operation history management unit that manages the frame position at which the playback instruction, slow playback instruction, or frame advance playback instruction has been received, and after the frame position is recorded in the operation history management unit, a moving image is transmitted from a frame image in the past from the frame position. And an image enlargement unit that enlarges a specific object in a frame image after the frame position and displays the enlarged image on the display unit.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, a specific object can be expanded and viewed, reducing a user's operation burden.

FIG. 1 is a configuration diagram of an imaging apparatus 500 according to the first embodiment. The imaging apparatus 500 according to Embodiment 1 includes an imaging unit 50 and an image processing apparatus 100.
The imaging unit 50 acquires a moving image and supplies it to the image processing apparatus 100. The imaging unit 50 includes a solid-state imaging device (not shown) such as a CCD (Charge Coupled Devices) sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a signal processing circuit (not shown) that processes a signal output from the solid-state imaging device. . The signal processing circuit can convert the analog three primary color signals R, G, and B output from the solid-state imaging device into digital luminance signals Y and color difference signals Cr and Cb.

  The image processing apparatus 100 processes a moving image acquired by the imaging unit 50. The image processing apparatus 100 includes an encoding unit 10, an object detection unit 12, an object registration unit 14, and an object tracking unit 16. The configuration of the image processing apparatus 100 can be realized in terms of hardware by a CPU, memory, or other LSI of an arbitrary computer, and in terms of software, it is realized by a program loaded in the memory. Describes functional blocks realized through collaboration. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding unit 10 encodes the moving image acquired by the imaging unit 50 to generate an encoded stream. More specifically, the moving image is compressed and encoded according to a predetermined standard to generate an encoded stream. For example, H.M. H.264 / AVC, MPEG-2, MPEG-4, etc. are compressed and encoded.

  The object detection unit 12 detects a specific object from the frame image included in the moving image acquired by the imaging unit 50. The object registration unit 14 registers a specific object in the object detection unit 12. For example, a child's face can be imaged and registered using the imaging unit 50. Examples of objects include people, pets such as dogs and cats, and moving objects such as cars and trains. Hereinafter, a case where the object is a person will be described as an example.

  The person as the object may be a person first detected from within the frame image after the start of moving image capturing, or may be a specific person registered by the object registration unit 14. In the former case, dictionary data for detecting an entire person is used, and in the latter case, dictionary data for detecting a registered specific person is used. The first detected person or the registered specific person becomes a tracking target in the subsequent frame image.

  The object detection unit 12 can identify a person by detecting a face in the frame image. The object detection unit 12 sets the body region below the face region including the detected face. The size of the body region is proportional to the size of the face region. Further, a person region including the whole body of a person may be set to be a tracking target.

  The face detection process may be performed by a known method and is not particularly limited. For example, a face detection method based on an edge detection method, a boosting method, a hue extraction method, or a skin color extraction method can be used.

  The edge detection method extracts various edge features from the face area including the face, eyes, nose, mouth, face outline, etc. of the face image that has been normalized in advance for face size and gray value, and identifies whether it is a face A face discriminator is constructed by learning feature quantities that are effective for this purpose based on statistical methods. For the face of a specific person registered from the object registration unit 14, a face discriminator is constructed from the face image.

  In order to detect a face from the input image, similar feature amounts are extracted while performing raster scanning from the end of the input image with the face size normalized during learning. From the feature amount, the discriminator determines whether the region is a face. As the feature amount, for example, a horizontal edge, a vertical edge, a right diagonal edge, a left diagonal edge, or the like can be used. If no face is detected, the input image is reduced at a certain rate, and the face is searched for the reduced image while performing raster scanning in the same manner as described above. By repeating such processing, a face of any size can be found in the image.

  The object tracking unit 16 tracks a specific object detected by the object detection unit 12, and generates tracking information based on the tracking state. Then, the generated tracking information is supplied to the encoding unit 10. The encoding unit 10 adds the tracking information generated by the object tracking unit 16 to the encoded stream.

  The object tracking unit 16 can track a specific object detected by the object detection unit 12 in a subsequent frame image, and can determine whether the tracking is successful in units of frame images. In this case, the encoding unit 10 adds the success / failure of the tracking as the tracking information to the header area or the area where user writing is permitted (hereinafter referred to as the user area) of each frame image. The success or failure of tracking of each frame image may be described collectively in a sequence header area or a GOP (Group Of Picture) header area instead of the picture header area.

  The object tracking unit 16 can track a specific object based on the color information of the object. In the example described above, tracking is performed by searching for a color area similar to the color of the body area in the subsequent frame image. If the result of face detection by the object detection unit 12 in the subsequent frame image is taken into account, the tracking accuracy can be improved.

  The success or failure of the tracking is determined as follows. That is, if the object to be tracked is included in the frame image, the object tracking unit 16 determines that the frame image has been successfully tracked. If the object to be tracked is not included in the frame image, the object tracking unit 16 The image is determined to be a tracking failure. Here, the tracking unit of the object may be the face area unit or the person area unit.

  The object tracking unit 16 specifies the position and size of a specific object in the frame image that has been successfully tracked. The object tracking unit 16 can specify the position of a specific object at a predetermined position in its detection area, for example, the face area or the person area. For example, the center point can be specified as the predetermined position. The object tracking unit 16 can specify the size of a specific object by the length and width of the detection area.

  The object tracking unit 16 can include the position and size of a specific object in the tracking information. As will be described later, when redundant data for super-resolution processing is added to an encoded stream, an area represented by the redundant data indicates the position and size of a specific object. In that case, a configuration in which the position and size of a specific object are not included in the tracking information is also possible. Instead, the tracking information may include the presence or absence of redundant data for super-resolution processing in units of frame images.

  The encoding unit 10 encodes redundant data in a detection area in order to perform super-resolution processing on a detection area of a specific object, here, either the face area or the person area. The encoding unit 10 can acquire the position and size of the detection area from the object tracking unit 16.

  Super-resolution processing is a technique for generating a high-resolution image from a plurality of low-resolution images with positional deviation. Details of the super-resolution processing (Shin Aoki, “Super-resolution processing using a plurality of digital image data”, Ricoh Technical Report No. 24, NOVEMBER, 1998), JP-A-2005-197910, JP-A-2007- No. 205, Japanese Patent Application Laid-Open No. 2007-193508, and the like. For example, when providing redundant data in the time direction, the encoding unit 10 causes the imaging unit 50 to capture an image at a high frame rate, encodes the detection region at a high frame rate, and normal frames rate for the other regions. Encode with The image data that the detection area has more redundantly than the other areas can be used as a plurality of low-resolution images with positional deviation for super-resolution processing.

  The encoding unit 10 generates an encoded stream CS to which the tracking information is added, and records the encoded stream CS on a recording medium (not shown) such as a memory card, a hard disk, or an optical disk, or sends it to a network.

  As described above, according to the first embodiment, the position and size of a specific object detected in the frame image are added to the encoded stream, so that the detection area of the specific object is set on the playback side. A reproduction method for tracking the area while expanding can be easily performed. The same applies to the case where redundant data for super-resolution processing of a detection area of a specific object is added.

  FIG. 2 is a configuration diagram of the image processing apparatus 200 according to the second embodiment. The image processing apparatus 200 according to the second embodiment may be mounted as one function of the imaging apparatus 500 or may be configured as a single device. The image processing apparatus 200 according to Embodiment 2 includes a decoding unit 20, a display unit 22, an acquisition unit 24, an operation unit 26, an image enlargement unit 28, and a control unit 30.

  The decoding unit 20 decodes the encoded stream CS in which the moving image is encoded. This encoded stream CS may be generated by the image processing apparatus 100 according to Embodiment 1. The display unit 22 displays a moving image having a frame image including a specific object. Here, the moving image decoded by the decoding unit 20 is displayed. It is assumed that the moving image includes the frame image.

  The operation unit 26 receives an instruction from the user. For example, a normal playback instruction, a normal stop instruction, a pause instruction, a fast-forward playback instruction, a rewind playback instruction, a slow playback instruction, a frame-by-frame playback instruction, an enlarged playback instruction, and the like of a moving image are accepted.

  The image enlargement unit 28 enlarges a specific object in a frame image after the frame position at which the operation unit 26 has received a normal reproduction instruction from the user during or after the moving image is rewound and displayed. To display. The rewind playback is stopped when the operation unit 26 receives a stop instruction or a pause instruction from the user. Immediately after the stop of rewind playback is a concept including the timing of the stop instruction or pause instruction and the subsequent normal playback instruction timing including a series of operations by the user and the end of the time interval considered by the designer. It is.

  The control unit 30 includes a reproduction mode control unit 32, a reproduction position specifying unit 34, and a reproduction start position moving unit 36. The reproduction mode control unit 32 sets a reproduction mode corresponding to the instruction input from the operation unit 26 in the decoding unit 20. When the rewind playback instruction is input from the operation unit 26, the playback mode control unit 32 sets the rewind playback mode in the decoding unit 20. The rewind playback here is a mode in which rewinding is performed in units of frames, not in units of chapters. Therefore, the user can comprehensively view the contents of the moving image during rewinding reproduction. Note that this rewind playback instruction is generally issued when there is a scene that the user wants to view again.

  The reproduction position specifying unit 34 specifies a frame position in the moving image when the operation unit 26 receives a normal reproduction instruction from the user during the rewinding reproduction of the moving image or immediately after the stop. This frame position indicates the position of the frame image when the instruction is accepted among the frame images continuous in the time direction. The normal reproduction instruction here is generally issued when the user discovers a scene that the user wants to view again.

  The image enlarging unit 28 acquires the frame position specified by the reproduction position specifying unit 34, enlarges a specific object in the frame image after the frame position, and causes the display unit 22 to display it. As a matter of course, the image enlargement unit 28 does not enlarge a frame image that does not include a specific object.

  When the position and size of the specific object detected and tracked in the frame image included in the moving image is added to the encoded stream CS, the acquisition unit 24 determines the position and size of the specific object. To get. The image enlarging unit 28 refers to the position and size of the specific object acquired by the acquisition unit 24 and expands the specific object.

  More specifically, the image enlargement unit 28 enlarges the detection region specified by the position and size by spatial pixel interpolation processing. As this pixel interpolation processing, simple linear interpolation processing or interpolation processing using an FIR filter can be employed. Although the accuracy is lower than in the case of using the redundant data described above, the detection area is subjected to super-resolution processing using image data of corresponding areas of a plurality of frame images adjacent in the time direction in the moving image. May be.

  When redundant data for super-resolution processing of a specific object detected and tracked in a frame image included in a moving image is added to the encoded stream CS, the acquisition unit 24 uses the redundant data. To get. The image enlargement unit 28 uses the redundant data acquired by the acquisition unit 24 to enlarge a specific object by super-resolution processing.

  When the specific image is not included in the frame image at the frame position, the reproduction start position moving unit 36 advances the reproduction start position according to the normal reproduction instruction to the frame position of the frame image including the specific object. Can do. When the success / failure of tracking for each frame image is added to the encoded stream CS, the reproduction start position moving unit 36 can advance the reproduction start position with reference to the success / failure of the tracking.

  Further, the reproduction start position moving unit 36 sets the reproduction start position according to the normal reproduction instruction when the redundant data for super-resolution processing for the frame image at the frame position is not added to the encoded stream. It is possible to proceed to the frame position of the frame image to which redundant data for resolution processing is added. When the presence or absence of redundant data for super-resolution processing is added for each frame image in the encoded stream CS, the reproduction start position moving unit 36 advances the reproduction start position with reference to the presence or absence. Can do.

  Further, the reproduction start position moving unit 36 can return the reproduction start position according to the normal reproduction instruction to the frame position of the frame image where the specific object included in the frame image at the frame position first appears. When the success / failure of tracking for each frame image is added to the encoded stream CS, the reproduction start position moving unit 36 can return the reproduction start position with reference to the success / failure of the tracking.

  The frame image in which the specific object first appears may be a frame image when a series of movements of the specific object included in the frame image at the frame position starts. That is, it may be the oldest frame image when a specific object is continuously included in the past direction from the frame image at the frame position. In this case, when a specific object is removed from the screen in the middle frame image, it is considered that a series of movements are interrupted there.

  Further, the reproduction start position moving unit 36 performs the normal reproduction up to the frame position of the first frame image to which redundant data for super-resolution processing of a specific object included in the frame image at the frame position is added. The reproduction start position according to the instruction can be returned. When the presence or absence of redundant data for super-resolution processing is added for each frame image in the encoded stream CS, the reproduction start position moving unit 36 refers to the presence or absence and returns the reproduction start position. Can do. The first frame image to which redundant data for super-resolution processing is added is a frame image when a series of movements of a specific object included in the frame image at the frame position is started. Also good.

  FIG. 3 is a diagram illustrating an example of a moving image displayed on the display unit 22 of the image processing apparatus 200 according to the second embodiment. FIG. 3A shows a moving image during rewind playback. FIG. 3B shows a moving image when normal reproduction is started. This moving image includes a person 60 as a specific object.

  When there is a scene that the user wants to view again during playback of a moving image, the user inputs a rewind playback instruction from the operation unit 26. The playback mode control unit 32 controls to rewind and play back the moving image according to the rewind playback instruction. Next, the user inputs a normal playback instruction from the operation unit 26 when he / she finds a scene he / she wants to watch during rewind playback of the moving image. The reproduction mode control unit 32 performs control so that the moving image is normally reproduced in accordance with the normal reproduction instruction. The playback position specifying unit 34 specifies the frame position in the moving image when the normal playback instruction is input.

  The image enlargement unit 28 enlarges the person 60 in the frame image after the frame position. When the person 60 is not included in the frame image specified by the reproduction position specifying unit 34, the reproduction start position moving unit 36 advances the reproduction start position to the frame position of the frame image including the person 60.

  As described above, according to the second embodiment, by automatically enlarging a specific object in a frame image after the frame position when a normal playback instruction is input during rewind playback, A specific object can be enlarged and viewed while reducing the operation burden. That is, the user can enlarge a specific object without rewinding to a desired frame position and inputting an enlargement reproduction instruction.

  Also, the timing at which the user inputs the normal playback instruction and stops the rewind playback is often the timing at which a specific object appears in the frame image. Even if the user tries to stop at that timing, the user may rewind too much. In that case, the specific object is not included in the frame image at the frame position when the normal playback instruction is input, but by moving the playback start position to the frame image at the frame position including the specific object, Unnecessary playback sections that do not include a specific object can be skipped.

  On the other hand, even if the user tries to stop at that timing, rewinding may be insufficient. In this case, viewing can be started from the first frame image in which the specific object appears by returning the reproduction start position to the frame position where the specific object appears in the frame image.

  In addition, when redundant data for super-resolution processing for the frame image at the frame position when the normal reproduction instruction is input is not added in the encoded stream, redundant data for super-resolution processing is added. By advancing the reproduction start position to the frame position of the frame image, it is possible to view from a frame image that can be enlarged with high image quality. Also, by returning the reproduction start position according to the normal reproduction instruction to the frame position of the first frame image to which redundant data for super-resolution processing is added, the first image that can be enlarged with high image quality is obtained. It can be viewed from the frame image.

  FIG. 4 is a configuration diagram of an image processing apparatus 300 according to the third embodiment. The image processing apparatus 300 according to Embodiment 3 may be mounted as a function of the imaging apparatus 500 or may be configured as a single device. The configuration of the image processing apparatus 300 according to the third embodiment is a configuration in which an operation history management unit 40 is added to the configuration of the image processing apparatus 200 according to the second embodiment. Further, the control unit 30 may include an enlargement ratio setting table 38.

In the following, the configuration and operation of the operation history management unit 40 and components that operate differently from the second embodiment will be described.
The operation history management unit 40 manages the frame position at which the operation unit 26 receives a normal reproduction instruction from the user during or after the moving image rewinding reproduction. In addition, the operation history management unit 40 manages the frame position at which the operation unit 26 receives an enlargement reproduction instruction, a slow reproduction instruction, or a frame advance reproduction instruction from the user during or immediately after the moving image is reproduced. More specifically, the reproduction position specifying unit 34 specifies the frame position when the operation unit 26 receives those instructions, and records the frame position in the operation history management unit 40. The frame-by-frame playback here is forward-frame-by-frame playback that is activated when a fast-forward playback instruction is input while a moving image is paused.

  Further, the operation history management unit 40 may manage the frame position at which the operation unit 26 receives a stop instruction, a pause instruction, or a normal playback instruction from the user during fast-forward playback of a moving image. Further, the operation history management unit 40 may manage the frame position at which the operation unit 26 has received a stop instruction, a pause instruction, or a normal playback instruction from the user during reverse frame-by-frame playback. A scene when an instruction such as an enlargement reproduction instruction, a slow reproduction instruction, or a frame advance reproduction instruction is input from the user can be estimated as a scene with high user attention.

  After the frame position is recorded in the operation history management unit 40, the image enlargement unit 28, when a moving image is reproduced from a frame image that is past the frame position, the specific object in the frame image after the frame position Is enlarged and displayed on the display unit 22. Of course, this enlargement is not due to a user operation. Note that the moving image reproduction here may be normal reproduction, slow reproduction, or forward frame advance reproduction.

  The operation history management unit 40 is configured so that the operation unit 26 issues a normal playback instruction from the user at the frame position or a frame position existing within a predetermined range adjacent to the frame position during or after rewinding playback of a moving image. The operation unit 26 increments the number of operation histories at the frame position at which an enlarged playback instruction, slow playback instruction, or frame advance playback instruction has been received from the user immediately after the received frame position or moving image is being played or immediately after it is stopped. The predetermined range adjacent to the frame position is set to a range considered by the designer that the user's instruction is for the same scene.

  The image enlargement unit 28 changes the enlargement rate of a specific object according to the number of operation histories managed by the operation history management unit 40. Specifically, the enlargement ratio is improved as the number of operation histories increases. Here, the image enlargement unit 28 can determine the enlargement rate with reference to the enlargement rate setting table 38. Of course, there is an upper limit to the improvement of the enlargement rate, and the enlargement rate is made constant when the number of operation histories exceeds a predetermined number. Further, when a specific object does not fit in the screen due to enlargement, the magnification is reduced to an enlargement rate that fits in the screen.

  FIG. 5 is a diagram illustrating a first example of a moving image displayed on the display unit 22 of the image processing apparatus 200 according to the third embodiment. FIG. 5A shows a moving image during rewind playback. FIG. 5B shows a moving image when the first normal reproduction is started. FIG. 5C shows a moving image when the second normal reproduction is started. This moving image includes a person 60 as a specific object.

  When there is a scene that the user wants to view again during playback of a moving image, the user inputs a rewind playback instruction from the operation unit 26. The playback mode control unit 32 controls to rewind and play back the moving image according to the rewind playback instruction. Next, the user inputs a normal playback instruction from the operation unit 26 when he / she finds a scene he / she wants to watch during rewind playback of the moving image. The playback position specifying unit 34 specifies the frame position when the normal playback instruction is input, and records it in the operation history management unit 40. The playback mode control unit 32 controls to perform normal playback from the frame position.

  When the user inputs a normal playback instruction from the operation unit 26 at a different opportunity from the normal playback, the playback mode control unit 32 performs control so that the moving image is normally played back in accordance with the normal playback instruction. The image enlargement unit 28 enlarges the person 60 in the frame image after the frame position recorded in the operation history management unit 40.

  FIG. 6 shows an example of the enlargement ratio setting table 38 of the image processing apparatus 200 according to the third embodiment. The enlargement ratio setting table 38 associates the number of operation histories at the frame position A with the enlargement ratio of a specific object. The operation at the frame position A includes an operation at a frame position existing within a predetermined range adjacent to the frame position A. In the example of FIG. 6, the enlargement ratio is fixed to 4.0 times after the number of operation histories is 3 or more.

  FIG. 7 is a diagram illustrating a second example of a moving image displayed on the display unit 22 of the image processing apparatus 200 according to the third embodiment. FIG. 7A shows a moving image during rewind playback. FIG. 7B shows a moving image when the first normal reproduction is started. FIG. 7C shows a moving image when the second normal reproduction is started. This moving image includes a person 60 as a specific object.

  When there is a scene that the user wants to view again during playback of a moving image, the user inputs a rewind playback instruction from the operation unit 26. The playback mode control unit 32 controls to rewind and play back the moving image according to the rewind playback instruction. Next, the user inputs a normal playback instruction from the operation unit 26 when he / she finds a scene he / she wants to watch during rewind playback of the moving image. The playback position specifying unit 34 specifies the frame position when the normal playback instruction is input, and records it in the operation history management unit 40. The reproduction mode control unit 32 performs control so that a moving image is normally reproduced according to the normal reproduction instruction. The image enlargement unit 28 enlarges the person 60 by a factor of 1.5 in the frame image after the frame position.

  When the user inputs a normal playback instruction from the operation unit 26 at a different opportunity from the normal playback, the playback mode control unit 32 performs control so that the moving image is normally played back in accordance with the normal playback instruction. The image enlargement unit 28 enlarges the person 60 by 2.0 times in the frame image after the frame position recorded in the operation history management unit 40.

  As described above, according to the third embodiment, by recording the frame position when the normal reproduction instruction is input during the rewind reproduction, when the moving image is reproduced next time, A specific object can be automatically enlarged in the frame image. Therefore, a specific object can be enlarged and viewed while reducing the operation burden on the user.

  In addition, the larger the number of operations at the frame position, the higher the magnification rate when a specific object is magnified. Thereby, it is possible to display a specific object in the screen in proportion to the degree of attention of the user. Note that not the frame position when a normal playback instruction is input during rewind playback, but an enlarged playback instruction, slow playback instruction, or frame advance playback instruction is input during playback of a moving image (excluding rewind playback). The same argument applies to the frame position when it is done.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  As a modification, the image processing apparatuses 200 and 300 according to the second and third embodiments are configured by an object detection unit 12, an object registration unit 14, and an object tracking unit that are components of the image processing apparatus 100 according to the first embodiment. 16 may be provided as a component. In this case, it is not necessary for the encoding device to track a specific object and include the tracking information in the encoded stream. The processing according to the second and third embodiments can be performed even on an existing encoded stream that does not include the tracking information.

1 is a configuration diagram of an imaging apparatus according to Embodiment 1. FIG. 3 is a configuration diagram of an image processing apparatus according to Embodiment 2. FIG. 10 is a diagram illustrating an example of a moving image displayed on a display unit of an image processing apparatus according to Embodiment 2. FIG. FIG. 3A shows a moving image during rewind playback. FIG. 3B shows a moving image when normal reproduction is started. 6 is a configuration diagram of an image processing apparatus according to Embodiment 3. FIG. 10 is a diagram illustrating a first example of a moving image displayed on a display unit of an image processing device according to Embodiment 3. FIG. FIG. 5A shows a moving image during rewind playback. FIG. 5B shows a moving image when the first normal reproduction is started. FIG. 5C shows a moving image when the second normal reproduction is started. 10 is a diagram illustrating an example of an enlargement ratio setting table of an image processing apparatus according to Embodiment 3. FIG. 10 is a diagram illustrating a second example of a moving image displayed on the display unit of the image processing apparatus according to Embodiment 3. FIG. FIG. 7A shows a moving image during rewind playback. FIG. 7B shows a moving image when the first normal reproduction is started. FIG. 7C shows a moving image when the second normal reproduction is started.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 encoding part, 12 object detection part, 16 object tracking part, 14 object registration part, 20 decoding part, 22 display part, 24 acquisition part, 26 operation part, 28 image expansion part, 30 control part, 32 reproduction | regeneration mode control part , 34 playback position specifying unit, 36 playback start position moving unit, 38 enlargement ratio setting table, 40 operation history management unit, 50 imaging unit, 60 person, 100 image processing device, 200 image processing device, 300 image processing device, 500 imaging apparatus.

Claims (8)

A display unit for displaying a moving image having a frame image including a specific object;
An operation unit for receiving instructions from the user;
During the rewind playback of the moving image or immediately after it is stopped, the specific object is enlarged and displayed on the display unit in a frame image after the frame position where the operation unit has received a normal playback instruction from the user. An image magnifier,
An image processing apparatus comprising:   A reproduction start position moving unit that advances the reproduction start position according to the normal reproduction instruction to the frame position of the frame image including the specific object when the specific object is not included in the frame image at the frame position; The image processing apparatus according to claim 1, further comprising:   The image processing apparatus further includes a reproduction start position moving unit that returns the reproduction start position according to the normal reproduction instruction to the frame position of the frame image where the specific object included in the frame image at the frame position first appears. The image processing apparatus according to claim 1. A display unit for displaying a moving image having a frame image including a specific object;
An operation unit for receiving instructions from the user;
An operation history management unit that manages a frame position at which the operation unit has received a normal reproduction instruction from the user immediately after rewinding reproduction of the moving image or immediately after the stop,
After the frame position is recorded in the operation history management unit, when the moving image is reproduced from a frame image past the frame position, the specific object in the frame image after the frame position is enlarged. An image enlargement unit to be displayed on the display unit;
An image processing apparatus comprising: A display unit for displaying a moving image having a frame image including a specific object;
An operation unit for receiving instructions from the user;
An operation history management unit that manages a frame position at which the operation unit has received an enlarged reproduction instruction, a slow reproduction instruction, or a frame advance reproduction instruction from the user during or after the moving image is reproduced
After the frame position is recorded in the operation history management unit, when the moving image is reproduced from a frame image past the frame position, the specific object in the frame image after the frame position is enlarged. An image enlargement unit to be displayed on the display unit;
An image processing apparatus comprising: The operation history management unit performs normal playback from the user at the frame position or a frame position existing within a predetermined range adjacent to the frame position during the rewind playback of the moving image or immediately after the stop. The number of operation histories at the frame position at which the instruction is received, or at the frame position at which the operation unit receives an enlargement playback instruction, slow playback instruction, or frame advance playback instruction from the user during or immediately after the playback of the moving image Increment
The image processing apparatus according to claim 4, wherein the image enlargement unit changes an enlargement rate of the specific object according to the number of operation histories managed by the operation history management unit. A decoding unit that decodes an encoded stream in which the moving image is encoded;
An acquisition unit for acquiring a position and a size of a specific object detected and tracked in a frame image included in the moving image, added in the encoded stream;
The said image expansion part expands the said specific object with reference to the position and magnitude | size of the said specific object acquired by the said acquisition part, The Claim 1 characterized by the above-mentioned. Image processing apparatus. A decoding unit that decodes an encoded stream in which the moving image is encoded;
An acquisition unit for acquiring redundant data for super-resolution processing of a specific object detected and tracked in a frame image included in the moving image added in the encoded stream;
The image according to any one of claims 1 to 6, wherein the image enlargement unit enlarges the specific object by the super-resolution processing using redundant data acquired by the acquisition unit. Processing equipment.

Images