JP2007049634A - Reproducing apparatus and reproducing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reproducing technology whereby moving picture data during transfer can be confirmed in real time and no effect is imposed on a transfer time and a transfer amount by monitoring a processing capability at a receiver side. <P>SOLUTION: An apparatus for extracting still picture data from the moving picture data to reproduce the still picture data includes a reception means for receiving the moving picture data from a transmitter side apparatus; an extract means for extracting the still picture data from the moving picture data received by the reception means; a reproduction means for updating the still picture data extracted by the extract means at prescribed intervals to reproduce the updated data; a control means for controlling the reception means, the extract means, and the reproduction means; and a monitoring means for monitoring a processing load of the control means. The control means revises the update interval of the still picture data reproduced by the reproduction means in response to a degree of the processing load obtained by the monitoring means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for extracting still image data from moving image data being received and performing thinning reproduction.

  In recent years, not only still image data but also moving image data can be recorded, and a video camera, a digital camera, an imaging function mobile phone, etc. that can be connected to an information processing device such as a personal computer (hereinafter referred to as a PC) to perform data communication. Imaging devices are widely used. In addition, by connecting these imaging devices to an information processing apparatus such as a PC, a still image or a moving image that has already been shot is displayed on the display of the information processing apparatus, and the image data is transferred to the hard disk or DVD of the information processing apparatus. It is possible to record in a recording device such as.

By the way, when browsing still images and moving images that have been taken from the information processing device using these image pickup devices, each image data itself is transmitted from the image pickup device to the information processing device. The data is changed to browsing data and displayed. However, in this method, no matter what the image data is, the contents of the image data cannot be known until the reception on the information processing apparatus side is completed. For example, even though moving image data with a large file size is taken for a long time, it may be a useless file when played back. Accordingly, there is streaming reproduction (hair isochronous method) as a method for reproducing and displaying moving image data being received. This streaming reproduction is a technique for reproducing moving image data being received in real time when the moving image data is captured. In other words, a still image data group that is continuous in time series extracted from moving image data is transmitted by a digital camera or the like on the transmission side, and the received still image data group is streamed and reproduced on the reception side. Thus, the contents of the moving image data can be known before receiving the entire moving image data (for example, Patent Document 1).
JP2003-333464A

  However, in the above prior art, in addition to the moving image data, it is necessary to receive a still image data group for streaming reproduction, so that the entire transfer amount increases by the file size of the still image data group. Furthermore, as the file size of the moving image data increases, the file size of the still image data group for streaming playback also increases.

  Although it is possible to synchronize the display of the still image data group for streaming playback and the moving image data frame being received, the still image data group for streaming playback is created in consideration of the processing capability of the receiving side. Not. For this reason, the display of the still image data group for streaming playback may put a heavy load on the processing capability on the receiving side and delay the reception of moving image data. Furthermore, in an environment where high-speed transfer is possible, switching to still image data for streaming playback can be performed in a short time, which may place an unnecessary load on the processing capability of the receiving apparatus.

  The present invention has been made in view of the above-described problems, and is capable of confirming moving image data being transferred in real time and monitoring the processing capability on the receiving side so that the transfer time and the transfer amount are not affected. The purpose is to realize reproduction technology.

  In order to solve the above-mentioned problems and achieve the object, the present invention is an apparatus for extracting and reproducing still image data from moving image data, and receiving means for receiving moving image data from a transmitting-side device; Extraction means for extracting still image data from moving image data received by the means, reproduction means for updating and reproducing still image data extracted by the extraction means at predetermined intervals, the receiving means, the extraction means, and Control means for controlling the regeneration means, and monitoring means for monitoring the processing load of the control means, the control means by the regeneration means according to the degree of the processing load obtained by the monitoring means The update interval of the still image data to be reproduced is changed.

  The present invention also relates to a method for extracting and reproducing still image data from moving image data, a receiving step for receiving moving image data from a transmitting side device, and a still image from moving image data received by the receiving step. An extraction process for extracting data, a reproduction process for updating and reproducing still image data extracted in the extraction process at predetermined intervals, and a processing load for controlling the reception process, the extraction process, and the reproduction process And an update interval of the still image data reproduced by the reproduction step is changed according to the degree of processing load obtained by the monitoring step.

  According to the present invention, when moving image data is transferred from an imaging device such as a camera to an information processing device such as a PC, reproduction is performed by changing the update interval of still image data according to the processing load on the receiving side. Therefore, it is possible to easily recognize the reception status and data contents of the moving image data being received without extending the transfer time, and cancel reception of unnecessary moving image data.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

  FIG. 1 shows an example of the configuration of a thinning reproduction system as a platform on which the present invention can be implemented.

  In FIG. 1, 101 is a computer system main body, 102 is a display for displaying data, 103 is a mouse as a typical pointing device, 104 is a mouse button, and 105 is a keyboard. Reference numeral 205 denotes a digital camera, which is connected to the computer system main body 101 via a communication cable 107 such as a USB, and can exchange various data with each other via the communication cable 107.

  FIG. 2 shows software and hardware constituting the thinning reproduction system of this embodiment.

  In FIG. 2, 201 is hardware. An operating system (OS) 202 operates on the hardware 201. Reference numeral 203 denotes application software that runs on the OS 202. Reference numeral 204 denotes hardware, but a control unit that operates the system via the other hardware 201 and the OS 202. Of the blocks configuring the hardware 201 and the OS 202, the configuration requirements are included as a matter of course, but the blocks that are not directly necessary for explaining the present embodiment are not shown. Such a block (not shown) is, for example, an external memory such as a memory card in hardware, or a memory management system in OS.

  In FIG. 2, reference numeral 2021 denotes a file system constituting the OS, which has a function that allows application software to input and output files without being aware of hardware. Reference numeral 2011 denotes a disk IO interface for the file system 2021 to read / write the hard disk 2009. A drawing management system 2022 constitutes the OS 202 and has a function that allows the application software 203 to perform drawing without being aware of the hardware 201. Reference numeral 2012 denotes a video interface for the drawing management system 2022 to draw on the display 102. Reference numeral 2023 denotes an input device system that constitutes the OS 202, and has a function that allows the application software 203 to receive user input without being aware of the hardware 201. Reference numeral 2013 denotes an input device interface for the input device management system 2023 to receive an input from the keyboard 105 or the mouse 103. Reference numeral 205 denotes a digital camera, which is connected via a USB cable in this example. The external interface management system 2024 manages the USB interface 2014 of the digital camera 205. The communication module 2032 communicates with the external interface management system 2024. The thinning reproduction software 2031 uses the communication module 2032 to exchange image data and various information with the digital camera 205. The CPU 2041 is a main control unit for overall system operation. Reference numeral 2042 denotes a physical memory into which applications executed by the CPU 2041 and data are read.

  FIG. 3 shows the configuration of the digital camera 205 of this embodiment.

  In FIG. 3, reference numeral 301 denotes a USB interface, which is used for connection with a PC. Reference numeral 302 denotes a mode switch. When the switch is switched to 302a, the playback mode is set, and when the switch is switched to 302b, the shooting mode is set. Reference numeral 303 denotes a memory card slot. When transferring image data from the digital camera to the PC, the mode switch 302 is set to the playback mode. In the shooting mode, an image preview is displayed on the LCD 304, and an image is shot by pressing the release switch 305, and the shot image data is recorded on the memory card 306.

  In the present embodiment, digital image data of a moving image shot and recorded by the digital camera of FIG. 3 is AVI (Audio Video Interleaved). The digital image data of the still image is JPEG (Joint Photographic Experts Group). Furthermore, the audio digital audio data is WAV (Word Audio Video). In the playback mode, the last image shot and recorded is displayed on the LCD 304, and the images are sequentially sent and played by the left and right buttons 307a and 307b of the cross key 307. A menu button 308 displays a menu screen on the LCD 304. Menu items can be selected with the left and right buttons 307 a and 307 b and can be determined with the SET button 309.

  Note that the digital camera 205 is set to the PC connection mode by setting the mode switch 302 to the playback mode and connecting the USB interface 301 on the camera side and the USB interface 2014 (FIG. 2) on the PC side with a USB cable (107 in FIG. 1). Be changed. In this PC connection mode, the digital camera can be controlled from the PC side, and digital data recorded on the memory card can be transferred to the PC side in the shooting mode.

  Next, the operation and action of this embodiment will be described with reference to FIGS.

  FIG. 4 is a flowchart showing an operation of receiving multimedia data transmitted from the digital camera and displaying a part or related data on the display. FIG. 5 shows a screen transition state when still image data thinned out from the moving image data being received is displayed on the display 102 when the multimedia data being received is moving image data. FIG. 6 shows a display screen in a state in which still image data thinned out from moving image data is set to be hidden. FIG. 7 shows a screen on which thumbnail data is extracted from still image data being received and displayed on the display 102 when the multimedia data being received is still image data. FIG. 8 shows an audio icon displayed on the display 102 when the multimedia data being received is audio data.

  First, the operation when the received multimedia data is moving image data will be described using the flowchart of FIG.

  In this embodiment, before receiving multimedia data, the processing capability of the PC serving as the platform and the communication capability of the camera are acquired in S402. The processing capability of the PC is the specifications of the CPU 2041, the size of the physical memory 2042, etc., and the communication capability of the camera is whether communication with USB 2.0 is possible, or communication with only USB 1.1 is possible. When the acquisition of the performance of the platform is finished, the reception of multimedia data actually starts in S403.

  In S404, the type of received multimedia data is determined. In the determination, not only the extension of the received data but also the header is actually analyzed to determine the data type. Here, since the received multimedia data is moving image data, a thread A for measuring the timing for displaying the extracted still image data is started. Further, all the still image data is extracted from the received moving image data, and when a still image display request is received from the thread A, two threads called a thread B for displaying the extracted still image data are started.

  First, the thread A that performs timer monitoring will be described.

  The initial value of the timer that the thread A follows is determined by the platform performance acquired in S402. When the processing capability of the CPU 2041 is high and the communication capability of the camera is also high, the initial value of the fixed time waiting in S412 is short. On the other hand, when the processing capacity of the CPU 2041 is low or the communication capability of the camera is not sufficient, the initial value of the fixed time waiting in S412 becomes long. However, when the processing capacity of the CPU 2041 is very high and the communication speed is high, the displayed still image may be switched to the next image before it can be sufficiently confirmed. Has a value.

  Next, thread B that receives moving image data, extracts still image data from the received moving image data, and displays still image data in response to a still image display request from thread A will be described. .

  When moving image data is received from the digital camera 205 via the USB interface 301 on the digital camera side and the USB interface 2014 on the PC side in S421, all still image data constituting the moving image data is extracted in S422. Next, if the moving image data is captured in the vertical position in S423, the extracted still image data is rotated to the vertical position. If S424 receives a still image display request from the thread A, the still image data is displayed on the display 102 of the PC in S425. On the other hand, if there is no still image display request from the thread A in S424, the still image data extracted in S426 is discarded.

  In S406, the operating status of the CPU 2041 and the change in the transfer rate are constantly monitored, and the timer setting value is changed according to the monitoring result. Here, for example, it is assumed that a part of the processing capacity of the CPU 2041 is deprived due to the activation of another application. Then, not only is it difficult to immediately display still image data in response to a still image display request from the thread A, but there is a possibility that communication with the digital camera will be finally affected. Therefore, in S406, if an increase in CPU load or a decrease in transfer speed is detected, the interval between still image display requests by thread A is increased to the extent that the transfer speed does not decrease. Conversely, when the load on the CPU is reduced due to the termination of another application and the transfer speed is improved, the interval between still image display requests by the thread A is reduced.

  FIG. 5 shows still images extracted from the moving image data being received and still images that are thinned and reproduced on the display 102 of the PC when the multimedia data being received is moving image data.

  In FIG. 5, reference numerals 500 to 509 denote still images extracted from moving image data received within a certain period of time. Reference numerals 510 to 514 denote screens displayed on the display 102 of the PC when the extracted still image data is thinned and reproduced. Although 510 to 514 are shown side by side, any one of 510 to 514 is not displayed on the display 102 of the PC at the same time by being explicitly arranged.

In S422, still image data is extracted from the moving image data and all the still image data included in the received moving image data is extracted. However, all the extracted still image data is displayed on the display 102 of the PC. I don't mean. Which of the still image data extracted from the moving image data is to be displayed on the display 102 is determined according to the operating status of the CPU 2041 and the transfer speed. Reference numerals 501, 503, 505, 506 and 509 correspond to still images extracted from the moving image data but not displayed on the display 102. If the number of still image data included in the moving image data is A, the number of still image data extracted from the moving image data is B, and the number of still images displayed on the display 102 is C, In the present embodiment, the following relational expression is established.
A = B ≧ C
Here, the thinning process is performed at the stage of extracting and displaying all the still image data without thinning out the still images at the stage of extracting the still image data, for the following two reasons. That is,
The first reason is that processing for displaying still image data requires a large amount of computation such as decoding, but the processing for extracting still image data from moving image data is basically a copy of the data. This is because even if still image data is extracted, the load is not so much.

  The second reason is that the file size of the still image data to be extracted is different for each still image data, but the moving image data image does not have a database describing the file size of each still image data. For this reason, it is difficult to extract still image data in the middle unless the leading still image data is extracted in order.

  In the present embodiment, since the digital image data of the moving image shot and recorded by the digital camera is AVI, the description has been made on the assumption of AVI. However, since S422 has a mechanism for extracting all still image data as described above, it is possible to implement this method even when receiving MPEG image data.

  When displaying 504, the operating status and transfer rate of the CPU 2041 are the same as when displaying 502, so the interval for issuing a display request in thread A is not changed. Therefore, before displaying 504, the same number (503) as that before displaying 502 is thinned out.

  When 507 is displayed, the processing load on the CPU 2041 is increased compared to when 504 is displayed. As a result, before displaying 507, two sheets (505 and 506) more than before displaying 504 are thinned out.

  When displaying 508, the processing capacity of the CPU 2041 is greater than when displaying 507. In addition, since the transfer speed has improved so that there is no possibility of redisplaying the still image already displayed on the display 102 of the PC even if the interval for issuing the display request in the thread A is narrowed, the number of thinned out sheets becomes zero. Yes.

  As shown in FIG. 5, the adjustment of the interval at which the display request is issued by the thread A is repeated until the completion of reception of the entire moving image file is detected in S407.

  When the completion of file reception is detected in S407, a thread set for extracting and displaying still image data in the moving image data is terminated in S408. If all the files have not been received, the next file is received in S403.

  It is also possible to hide the display of part of the multimedia data being received or related image data. FIG. 6 shows a screen for displaying or hiding image data.

  A still image 602 that is multimedia data being transferred is displayed at the center of the screen 600. By pressing a button 601 present on the screen 600, the still image 602 can be hidden. Reference numeral 610 denotes a screen in which the image 602 is not displayed. On the screen 610, an image that has been hidden can be returned to the display state by pressing a button 611. In FIG. 6, the image 602 of the multimedia data being transferred and the button 601 for hiding the image 602 are displayed on the same window. However, the image 602 and the button 601 may be arranged on different windows. Good.

  Next, an operation when the received multimedia data is still image data will be described.

  In this embodiment, before the multimedia data is received, the processing capability of the PC serving as the platform and the communication capability of the camera are acquired in S402. When the acquisition of the platform performance is completed, the reception of multimedia data actually starts in S403.

  In S404, the type of received multimedia data is determined. In the determination, not only the extension of the received data but also the header is actually analyzed to determine the data type. Since the received multimedia data is still image data, it passes through the determination of S404 and S430. Until the reception of the entire file is completed in S432, thumbnail data is extracted from the still image data being received and displayed on the display 102 of the PC as shown in FIG. 7 extracts the thumbnail 702 included in the EXIF header 701 from the EXIF-JPEG 700 taken by the digital camera of FIG. 3, and displays it on the display 102 of the PC instead of the still image data being received. (710). This display is performed until the reception of EXIF-JPEG 700 is completed (S432).

  When receiving the JPEG 700 that does not have the thumbnail 702, the location where the pixel data 703 has been received may be displayed instead of the thumbnail 702.

  Finally, the operation when the received multimedia data is audio data will be described.

  In this embodiment, before the multimedia data is received, the processing capability of the PC serving as the platform and the communication capability of the camera are acquired in S402. When the acquisition of the platform performance is completed, the reception of multimedia data actually starts in S403.

  In S404, the type of received multimedia data is determined. In the determination, not only the extension of the received data but also the header is actually analyzed to determine the data type. Since the received multimedia data is audio data, it passes through the determination of S404, S430, and S440. Until reception of the entire file is completed in S442, an image icon 801 for audio data prepared by the system is displayed on the display 102 of the PC as shown in FIG. 8 in S441.

[Other Embodiments]
The embodiment according to the present invention has been described in detail using specific examples. However, the present invention can take an embodiment as a system, apparatus, method, program, storage medium (recording medium), or the like. is there. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  It goes without saying that the object of the present invention can be achieved even if any part of the illustrated functional blocks and operations is realized by a hardware circuit or by software processing using a computer.

  Note that the present invention includes a case where the present invention is achieved by supplying a software program for realizing the functions of the above-described embodiments directly or remotely to a system or apparatus. In that case, a computer such as a system reads and executes the program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the recording medium (storage medium) for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. In addition, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser can be used to connect to a homepage on the Internet, and the computer program itself of the present invention can be downloaded from the homepage. It can also be supplied by downloading a compressed file including an automatic installation function to a recording medium such as a hard disk. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and a user who satisfies predetermined conditions downloads key information for decryption from a homepage via the Internet. You can also. In this case, the downloaded key information is used to execute the encrypted program and install it on the computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instructions of the program may be part of the actual processing. Alternatively, it can be realized by performing all of them.

  Further, after the program read from the recording medium is written in the memory of a function expansion board inserted into the computer or a function expansion unit connected to the computer, the CPU of the board or the like performs a part of the actual processing. Alternatively, it can be realized by performing all of them.

1 is a diagram showing a configuration of a thinning reproduction system according to an embodiment of the present invention. It is a figure which shows the software and hardware which comprise the thinning | decimation reproduction | regeneration system of this embodiment. It is a figure which shows the structure of the digital camera of this embodiment. It is a flowchart which shows the operation | movement which receives the multimedia data which a digital camera transmits, and displays the part or related data on a display. It is a figure which shows the transition state of a screen at the time of displaying the still image data thinned out from the moving image data being received on the display when the multimedia data being received is moving image data. It is a figure which shows the display screen in the state set to hide the still image data thinned out from moving image data. FIG. 10 is a diagram illustrating a screen on which thumbnail data is extracted from still image data being received and displayed on the display 102 when the multimedia data being received is still image data. It is a figure which shows the audio | voice icon displayed on the display when the multimedia data under reception is an audio | voice.

Explanation of symbols

101 Computer System Main Body 102 Display 103 Mouse 104 Mouse Button 105 Keyboard 107 Communication Cable 201 Hardware 202 Software (OS)
203 Application software 204 Control unit 205 Digital camera 301 USB interface 302 Mode switch 303 Memory card slot 304 LCD
305 Release switch 306 Memory card 307 Four-way controller 308 Menu button 309 SET button

Claims (15)

A device that extracts and reproduces still image data from moving image data,
Receiving means for receiving moving image data from the transmission side device;
Extracting means for extracting still image data from moving image data received by the receiving means;
Reproducing means for updating and reproducing the still image data extracted by the extracting means at a predetermined interval;
Control means for controlling the receiving means, the extracting means, and the reproducing means;
Monitoring means for monitoring the processing load of the control means,
The control means changes the update interval of the still image data reproduced by the reproduction means in accordance with the degree of processing load obtained by the monitoring means.   When the processing load obtained by the monitoring unit is larger than a predetermined level, the control unit makes the update interval of the still image data reproduced by the reproduction unit longer than the current interval, and the processing load is smaller than the predetermined level. 2. The apparatus of claim 1, wherein the update interval is shorter than a current interval.   The control means selects, from all the still image data extracted by the extraction means, still image data to be reproduced by the reproduction means according to the degree of processing load obtained by the monitoring means. The apparatus according to claim 1 or 2.   The apparatus according to claim 1, further comprising a non-display unit configured to hide the still image data reproduced by the reproduction unit.   The apparatus according to claim 1, wherein a minimum value is set for the update interval.   The apparatus according to claim 1, further comprising a stopping unit that stops each process performed by the receiving unit, the extracting unit, and the reproducing unit. The receiving means receives multimedia data including moving image data, still image data, and audio data from the transmission side device,
7. The apparatus according to claim 1, further comprising recording means for recording the received multimedia data.   The control means immediately ends the extraction of the still image data by the extraction means and the reproduction of the extracted still image data by the reproduction means when the reception of the moving image data by the reception means is completed. Apparatus according to any one of claims 1 to 7.   The apparatus according to claim 1, wherein the extraction unit extracts reduced image data attached to the still image data, and the reproduction unit reproduces the reduced image data.   The apparatus according to claim 9, wherein when the reduced image data is not attached to the still image data, information associated with the still image data is displayed by the reproduction unit.   8. The apparatus according to claim 7, wherein when the multimedia data includes audio data, the reproduction means displays an icon for audio data. Detecting means for detecting a shooting position of the moving image data;
A rotation means for rotating the still image data extracted by the extraction means to a vertical position when the photographing position detected by the detection means is a vertical position;
The apparatus according to claim 1, wherein the reproduction unit reproduces the still image data rotated by the rotation unit. A method for extracting and reproducing still image data from moving image data,
A receiving step of receiving moving image data from the transmitting side device;
An extraction step of extracting still image data from the moving image data received by the reception step;
A reproduction step of updating and reproducing the still image data extracted by the extraction step at a predetermined interval;
A monitoring step of monitoring a processing load when controlling the reception step, the extraction step, and the regeneration step,
A method of changing an update interval of the still image data reproduced by the reproduction step according to a degree of processing load obtained by the monitoring step.   A program for causing a computer to execute the method according to claim 13.   A computer-readable storage medium storing the program according to claim 14.

Images