JP2008136079A - Storage reproducing device and storage reproducing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly control display of a subwindow based on the motion of the subscreen used for bidirectional communication and the sound state of the subscreen by performing reproduction operation of a device that distributes an image. <P>SOLUTION: The storage reproducing device comprises an image analyzing part 14 which determines whether communication is established or not based on the motion signal detected from received communication data, a sound analyzing part 15 which determines whether the communication is established or not based on the output value of sound data which is extracted from received communication data, a subscreen control part 16 which, if determined the communication is established, sets transparency and saturation of the image to be displayed on the subscreen and/or the volume of sound outputted as the sound of the subscreen, an operation delay measuring part 21 which measures response time, and a reproduction control part 24 which, if the device has a content corresponding to the operation signal, compares elapsed time with delay time, and outputs synthesized data according to the operation signal when the elapsed time exceeds the delay time. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a storage / reproduction device and a storage / reproduction program that perform distribution and bidirectional communication of image and audio contents with an electronic device connected via a network.

  Conventionally, image and audio content distribution has been performed by radio wave broadcasting such as terrestrial digital broadcasting and sanitary broadcasting, or by distribution of recording media storing image and audio content.

  However, in recent years, the rapid transition from analog content to digital content, promotion of network infrastructure represented by optical communication networks, the spread of electronic devices that can be connected to the Internet, and various services via the Internet Changes in image and audio content distribution methods are occurring due to an increase in provision and the like.

  For example, instead of receiving a unilaterally transmitted signal from the broadcast station and playing back the image and audio content, the user can select the image and audio content that the user wants to view on-demand via the network for streaming delivery. There is a method of playing with. In addition, a method for distributing images and audio contents by downloading them via a network rather than distributing them through a recording medium on which images and audio contents are recorded is generally well known.

  Furthermore, DLNA (Digital Living Network Alliance) is promoting the formulation of guidelines for connecting AV (Audio Visual) devices and personal computers used at home via a home network to share content. In line with this guideline, the number of household content recording / playback devices that can be connected to the network is increasing.

  There is also a system that enables users to communicate with each other while delivering image and audio content. For example, Patent Document 1 discloses an image communication system that enables voice and character communication via a network while simultaneously reproducing images and sounds distributed from a distribution server on a playback device located at a physically separated location. Has been proposed.

  However, in the image communication system described in Patent Document 1, when users on the content supply side and the content reception side view the same content at the same time, the content supply side always performs a reproduction operation and distributes the image to the content reception side. . For this reason, the user on the content receiving side cannot perform operations related to reproduction such as primary stop, fast forward, and rewind. Therefore, the user on the content receiving side has to ask the user on the content supplying side to perform an operation related to reproduction using the image communication system, and the operation is troublesome.

  In addition, videophones, network conference systems, and the like are generally known as devices that allow users at remote locations to communicate. When bidirectional communication is performed using such a general device, a child screen is overlaid on the main image, and the face of the other party is displayed in the child screen. For this reason, when the sub-screen is superimposed on the main image, an important part of the main image may be hidden behind the sub-screen and become invisible.

Therefore, for example, Patent Document 2 discloses a child that displays the contents of the main image hidden by the child screen simultaneously with the contents of the child screen by transparently displaying the child screen according to the movement of the image displayed on the child screen. Screen display devices have been proposed.
JP 2003-235027 A JP 7-336622 A

  However, in the sub-screen display device described in Patent Literature 2, the sub-screen is transparently displayed according to the movement of the image displayed on the sub-screen so that the main screen image hidden by the sub-screen is displayed as the sub-screen image. Since the images are displayed at the same time, even if there is a movement in the image displayed on the child screen, it is not always an important scene in both communication. For this reason, there has been a problem that the sub-screen display cannot be appropriately controlled for both users.

  The present invention has been made in view of the above problems, can perform image distribution operations from both directions, and can appropriately display a child screen based on the movement of the child screen and the sound state of the child screen. It is an object to provide a storage / reproduction device and a storage / reproduction program that can be controlled.

  In order to achieve the above object, the first feature of the storage / playback device according to the present invention is a content distribution function for reading and distributing content data including image data and audio data stored in the storage device, and a captured image. It is connected to an electronic device having a bidirectional communication function for transmitting and receiving communication data including data and recorded voice data. The content data from the electronic device is output as a main screen, and the communication data is overlapped on the main screen. A storage / reproduction device that outputs as a sub-screen, and when a selection signal indicating whether an image analysis mode or a sound analysis mode is selected by an external input, a motion signal is received from communication data received from an electronic device. And an image analysis unit that determines that communication is being performed when the detected motion signal exceeds a motion signal threshold, and the selection signal is a voice signal. An audio analysis unit that extracts audio data from communication data received from an electronic device when the mode is indicated, and determines that communication is in progress when an output value of the extracted audio data exceeds a threshold for audio data; If the analysis unit (14) and / or the voice analysis unit determines that a call is in progress, the transparency and saturation of the image displayed on the sub-screen and / or the volume of the audio output as the sub-screen audio are set. The main screen content data and the content data of the main screen based on the volume and volume of the audio output as the audio of the sub-screen, or the transparency and saturation of the image displayed on the sub-screen set by the sub-screen control unit And a combining unit that combines the communication data of the child screens.

  In order to achieve the above object, the second feature of the storage / playback apparatus according to the present invention is that a content storage unit that stores content data, an operation delay measurement unit that measures response time of an electronic device, and an operation signal from an external input Is received from the storage unit, index information data indicating the title index of the content corresponding to the content data stored in the content storage unit, and the title index of the content corresponding to the operation signal exists in the read index information data When the operation signal is received, the elapsed time from when the operation signal is received is compared with the delay time measured by the operation delay measurement unit, and when the elapsed time exceeds the delay time, the synthesis unit performs synthesis according to the operation signal. And a reproduction control unit for outputting the processed data.

  According to the storage / reproduction device and the storage / reproduction program according to the present invention, a reproduction control operation can be performed on an apparatus that distributes image and audio content connected via a network, and is used for bidirectional communication. The child screen can be appropriately controlled based on the movement of the image displayed on the screen and the sound state of the child screen.

  Embodiments of the present invention will be described below with reference to the drawings.

  In the present embodiment, a storage / reproduction device that performs content distribution and bidirectional communication with a storage / reproduction device connected via a network will be described as an example.

  FIG. 1 is a block diagram showing the configuration of an embodiment of a storage / playback apparatus according to the present invention.

  The storage / reproduction device 30 in this embodiment is connected to the storage / reproduction device 31 via the network 100. Note that the storage / reproduction device 30 and the storage / reproduction device 31 have the same configuration.

  Further, the storage / reproduction device 30 includes an analysis mode selection interface 1 that generates a selection signal indicating whether the image analysis mode or the sound analysis mode is selected by an external input, and a user on the storage / reproduction device 30 side, for example. A camera 2 that converts the captured image into an electrical signal, an acute directional microphone 3 that converts audio into an electrical signal, an external AV device 4 that transmits video and audio content data to the storage / playback device 30, and A reproduction operation interface 5 that receives a reproduction control operation signal by an external input and an output unit 26 that outputs output data from the storage / reproduction device 30 are connected.

  The storage / reproduction device 30 includes an image input unit 6, an audio input unit 7, a content storage unit 8, an external device connection unit 9, a reproduction operation reception unit 10, a sub-screen encoder unit 11, and a noise removal filter 12. , Network connection interface 13, image analysis unit 14, audio analysis unit 15, sub-screen control unit 16, sub-screen decoder unit 17, content decoder unit 18, image synthesis unit 19, and voice synthesis unit 20 , An operation delay measurement unit 21, an AV synchronization unit 22, a reproduction unit 23, a reproduction control unit 24, and a storage unit 25.

  The noise removal filter 12 receives an audio signal converted into an electric signal by the sharp directional microphone 3 via the audio input unit 7. Then, noise is removed from the received voice signal to enhance the dialogue voice, and the voice signal from which noise has been removed is transmitted to the sub-screen encoder unit 11.

  Here, the noise removal filter 12 and the above-described sharp directional microphone 3 are provided in order to remove excessive noise for the user.

  For example, when the audio of the content data being reproduced by either the storage / reproduction device 30 or the storage / reproduction device 31 is mixed with the audio of the communication data, the audio of the content data being reproduced is slightly The audio of the delayed content data is superimposed on the audio of the communication data. The voice superimposed with a slight delay causes the user to feel uncomfortable. Therefore, the sharp directional microphone 3 and the noise removal filter 12 remove excess noise due to such superimposition of sound.

  The sub-screen encoder unit 11 encodes each of the image data photographed by the camera 2 and converted into an electrical signal and the audio data received from the noise removal filter 12, and both are multiplexed by a multiplexer and transmitted to the network connection interface 13. To do.

  The sub-screen decoder unit 17 decodes image data and audio data for the sub-screen received from the storage / reproduction device 31 via the network 100.

  When the analysis signal is received by the analysis mode selection interface 1, the image analysis unit 14 detects a motion signal from the communication data received from the child screen decoder unit 17, and the detected motion signal exceeds the motion signal threshold It is determined that communication is in progress.

  When the selection signal received by the analysis mode selection interface 1 is the voice analysis mode, the voice analysis unit 15 extracts the voice data from the communication data received from the sub-screen decoder unit 17, and the output value of the extracted voice data is If the audio data threshold is exceeded, it is determined that communication is in progress.

  Depending on whether or not the determination result of the image analysis unit 14 and / or the voice analysis unit 15 is in communication, the sub-screen control unit 16 can transmit the transparency and saturation of the image displayed on the sub-screen and / or the voice of the sub-screen. Set the volume of output audio.

  The content decoder unit 18 receives image data and audio data for main screen received from the storage / reproduction device 31 via the network 100, image data and audio data for main screen received from the content storage unit 8 storing content data, Alternatively, the main screen image data and audio data received from the external AV device 4 via the external device connection unit 9 are decoded.

  The image composition unit 19 receives the main screen image data received from the content decoder unit 18 and the sub-screen decoder unit 17 based on the transparency and saturation of the image displayed on the sub-screen set by the sub-screen control unit 16. Synthesize the received sub-screen image data.

  The voice synthesizing unit 20 synthesizes the audio data of the main screen received from the content decoder unit 18 and the audio data of the sub screen received from the sub screen decoder unit 17 based on the sound volume output as the audio of the sub screen.

  The operation delay measurement unit 21 measures the response time of the storage / reproduction device 31.

  The storage unit 25 stores index information data, delay time data, and the like, and the content storage unit 8 stores content data corresponding to the index information data stored in the storage unit 25.

  When receiving an operation signal from the playback operation interface 5, the playback control unit 24 reads index information data indicating the title index of the content corresponding to the content data stored in the content storage unit 8 from the storage unit 25, and reads it. When the title index of the content corresponding to this operation signal exists in the index information data, the elapsed time from the reception of this operation signal is compared with the delay time measured by the operation delay measurement unit 21. When the time exceeds the delay time, the data synthesized by the image synthesizing unit 19 and the voice synthesizing unit 20 is output according to the operation signal received from the reproduction operation interface 5.

  The AV synchronization unit 22 synchronizes the image data received from the image synthesis unit 19 and the audio data received from the audio synthesis unit 20, and the reproduction unit 23 receives from the AV synchronization unit 22 according to the control signal of the reproduction control unit 24. The processed image data and audio data are output to the output unit 26.

≪Sub-screen display setting process≫
Next, the processing procedure of the sub-screen display setting process of the storage / reproduction device 30 in this embodiment will be described with reference to the flowchart of FIG.

  First, the analysis mode selection interface 1 generates a selection signal indicating the “image analysis mode” or “voice analysis mode” received by the selection operation by the user.

  When the mode indicated by the generated selection signal is “image analysis mode”, the analysis mode selection interface 1 transmits the selection signal of “image analysis mode” to the image analysis unit 14 and the sub-screen control unit 16. To do. On the other hand, when the mode indicated by the received selection signal is “voice analysis mode”, the analysis mode selection interface 1 sends the “voice analysis mode” to the image analysis unit 14, the voice analysis unit 15, and the sub-screen control unit 16. "" Selection signal is transmitted (step S1).

  Next, the sub-screen decoder unit 17 decodes communication data including captured image data and recorded audio data received from the storage / reproduction device 31 via the network 100, and image analysis is performed on the image data of the decoded communication data. The voice data of the decoded communication data is transmitted to the voice analysis unit 15 and the voice synthesis unit 20.

  Next, when the mode indicated by the selection signal received from the analysis mode selection interface 1 is “image analysis mode”, the image analysis unit 14 determines that the image data of the communication data received from the child screen decoder unit 17 is It is determined whether or not communication is suggested (step S5).

  Specifically, first, when the image analysis unit 14 receives a selection signal indicating the “image analysis mode”, the image data received from the small-screen decoder unit 17 is divided into blocks, and the most similar to the previous frame in units of blocks. A motion vector is extracted using a block matching method for searching for a block that is present. Then, when the extracted motion vector exceeds a predetermined motion signal threshold, it is determined that communication has started.

  Further, when the motion vector extracted for a certain time falls below the motion signal threshold, it is determined that the communication has ended.

  Here, if the threshold value for the motion signal is set too high, it will be difficult to detect the start of communication, and even if the user starts communication, the child screen will be difficult to see, or the sound for the child screen will not be displayed. It may be difficult to hear. Conversely, if the threshold value for the motion signal is set too low, it is easy to misdetect the start of communication, and even when the user does not start communication, the sub screen is displayed more clearly than necessary, or the sub screen is displayed. Audio may interfere with the main screen audio. For this reason, it is necessary for the provider or the like to calculate in advance an appropriate value based on the actual measurement, and for the provider or the user to set an appropriate value in advance.

  In addition to the block matching method described above, whether or not the communication is suggested is determined by implementing a gesture (sign language) recognition function and displaying it as subtitles. It may be determined whether or not.

  Next, when the image analysis unit 14 determines that the image data received from the sub-screen decoder unit 17 indicates that communication is being performed, the image analysis unit 14 transmits a communication status signal indicating that communication is being performed to the sub-screen control unit. 16 to send.

  The sub-screen control unit 16 that has received the communication status signal indicates that the mode indicated by the selection signal received from the analysis mode selection interface 1 is “image analysis mode” and that the communication status signal received from the image analysis unit 14 is Since it indicates that communication is in progress, the sub-screen control unit 16 finally determines that the user who is using this sub-screen is in communication (step S5).

  On the other hand, when the mode indicated by the selection signal received from the analysis mode selection interface 1 is “voice analysis mode”, the sub-screen control unit 16 indicates that communication is being performed by the image analysis unit 14 and the voice analysis unit 15. (Step S2).

  In this case, the analysis mode selection interface 1 transmits the “audio analysis mode” selection signal to both the image analysis unit 14 and the audio analysis unit 15, whereby both the image analysis unit 14 and the audio analysis unit 15 perform the analysis operation. To do.

  Specifically, for example, when receiving a selection signal indicating “audio analysis mode”, the image analysis unit 14 detects a face area from the image data received from the sub-screen decoder unit 17. In the detection of the face area, an average “reference face” template is prepared from a plurality of samples of face images, and template matching is performed. Here, in order to increase the detection accuracy, a motion vector is extracted in the same manner as the method used in the image analysis mode, and matching is performed by weighting a portion having a relatively large amount of motion in the image.

  This is based on the usage situation of the storage / reproduction device 30 according to the present embodiment. Even if the background is complicated, the background hardly moves, and the movement of the person performing communication is large in the image. Utilizing that it is a moving part.

  Then, after detecting the face area, pattern matching of only the mouth parts around the mouth area with the template of the “reference face” prepared in advance is performed to detect the exact position of the mouth. Then, while tracking the position of the mouth, the motion vector of the mouth region is extracted, and when the extracted motion vector exceeds a predetermined motion signal threshold, a communication status signal indicating that communication is in progress This is transmitted to the sub-screen control unit 16.

  On the other hand, when the voice analysis unit 15 receives the selection signal indicating “voice analysis mode”, the voice level of the voice data received from the sub-screen decoder unit 17 exceeds a predetermined voice data signal threshold. Then, a communication status signal indicating that communication is in progress is transmitted to the sub-screen control unit 16.

  Finally, the sub-screen control unit 16 uses the sub-screen only when it receives a communication status signal indicating that communication is in progress from both the image analysis unit 14 and the voice analysis unit 15. Determines that the person is communicating.

  If the child screen control unit 16 determines that the communication is suggested by the image analysis unit 14 and / or the voice analysis unit 15, the transparency of the image displayed on the child screen is set to “0”%, for example. By setting (step S3) and setting the saturation of the image displayed on the child screen to the original state, for example, the image displayed on the child screen is set to color display (step S4).

  As a result, when the user is communicating using the sub-screen image, the sub-screen image can be clearly displayed, so that the sub-screen display is appropriately controlled so that the user can easily see it. Can do.

  Further, when the sub-screen control unit 16 determines that the image analysis unit 14 and / or the voice analysis unit 15 does not suggest that communication is being performed, the transparency of the image displayed on the sub-screen is set to, for example, “60”%. By setting (step S3) and setting the saturation of the image displayed on the child screen to, for example, the minimum, the image displayed on the child screen is set to monochrome display (step S7).

  As a result, when the user is not communicating using the sub-screen image, the sub-screen image can be made transparent so that it does not interfere with the main screen display. Interference with the colors of the screen can be prevented, and the display of the sub-screen can be appropriately controlled so that it is easy for the user to see based on the movement of the sub-screen and the sound state of the sub-screen.

  Then, the image composition unit 19 synthesizes the image of the child screen on the main screen based on the transparency and saturation of the image displayed on the child screen set by the child screen control unit 16.

≪Bidirectional operation processing≫
Next, according to the flowchart of FIG. 3, the processing procedure of the bidirectional operation processing of the storage / reproduction device 30 in the present embodiment will be described.

  First, the reproduction operation interface 5 transmits a reproduction control operation signal received by a selection operation or the like by the user to the reproduction control unit 24 via the reproduction operation receiving unit 10, and the reproduction control unit 24 transmits the reproduction control operation signal. When received, the storage device 31 connected via the network 100 is inquired as to whether or not the reproduction control operation is being performed (step S11). If the storage / playback device 31 is under playback control, the playback control operation accepted by the playback operation interface 5 is prohibited in order to prioritize the playback control operation of the storage / playback device 31 (step S19).

  Here, during playback control operation, it means a state in which fast forward, rewind, pause, frame advance, etc. are being performed and it is not in the normal playback state, and playback control operation signals are fast forward, rewind, An operation signal that is not a normal playback signal such as pause or frame advance.

  Thereby, it is possible to prevent the reproduction control operation signal from being transmitted from the storage / reproduction device 31 during the reproduction control operation of the storage / reproduction device 31.

  If the storage / reproduction device 31 is not in a reproduction control operation, the reproduction control unit 24 determines whether the storage / reproduction device 30 is in a reproduction control operation (step S12), and determines that the reproduction control operation is not in progress. If so, the process proceeds to step S13.

  Next, when the reproduction control unit 24 determines in step S12 that the storage / reproduction device 30 is not performing a reproduction control operation, the target of the received reproduction control operation signal is stored in the content storage unit 8 of the storage / reproduction device 30. It is determined whether or not the content is the current content (step S13).

  Specifically, the playback control unit 24 reads index information data indicating the title index of the content corresponding to the image or audio content data stored in the content storage unit 8 from the storage unit 25, and adds the read index information data to the read index information data. The determination is made based on whether there is an image or audio content corresponding to the playback control operation signal received by the playback operation interface 5.

  Then, the reproduction control unit 24, when the index information data of the image or audio content data corresponding to the reproduction control operation signal received by the reproduction operation interface 5 does not exist in the read index information data, that is, the reproduction control operation signal When it is determined that the target is not content stored in the content storage unit 8 of the storage / reproduction device 30 (step S13), the reproduction control unit 24 stores the reproduction control operation signal received by the reproduction operation interface 5 in the storage / reproduction device. It transmits to 31 (step S18).

  On the other hand, when the reproduction control unit 24 determines that the target of the reproduction control operation signal is the content stored in the content storage unit 8 of the storage / reproduction device 30 in step S13, the process proceeds to step S14.

  Thereby, the reproduction control operation signal received by the reproduction operation interface 5 can be appropriately allocated according to the state of the storage / reproduction device 30 and the storage / reproduction device 31.

  Next, when the reproduction control unit 24 determines in step S13 that the target of the reproduction control operation signal is not the content stored in the content storage unit 8 of the storage / reproduction device 30, the reproduction control unit 24 sets the response time of the storage / reproduction device 31. A measurement instruction signal for measuring the indicated delay time is transmitted to the operation delay measurement unit 21.

  The operation delay measurement unit 21 that has received the measurement instruction signal calculates the delay time by measuring the response time when the packet is transmitted to the storage / reproduction device 30, for example, and stores the delay time in the storage unit 25 (step S14). ).

  Next, the reproduction control unit 24 starts measuring an elapsed time for comparison with the delay time calculated in step S14 (step S15), and whether or not a reproduction control operation signal from the storage / reproduction device 31 is transmitted. Is monitored (step S16).

  Then, the reproduction control unit 24 reads the delay time from the storage unit 25, and performs the process of step S18 when the elapsed time when the measurement is started in step S15 exceeds the read delay time (step S17).

  On the other hand, when the playback control unit 24 receives a playback control operation signal from the storage / playback device 31 without exceeding the read delay time (step S17), the playback control operation 24 starts the measurement in step S15. In order to give priority to the operation signal, the operation of the storage / reproduction device 30 is prohibited (step S19).

  As a result, the previously received reproduction control signal can be preferentially processed between the storage / reproduction device 30 and the storage / reproduction device 31, so that bidirectional operations are possible.

  As described above, according to the storage / reproduction device 30 in the present embodiment, interactive operations can be performed while viewing the same content with the storage / reproduction device 31 connected to the network, and the child screen can be displayed. Since the display of the child screen can be appropriately controlled based on the movement of the user and the sound state of the child screen, convenience for the user can be improved.

  For example, the same content can be shared with relatives, friends, lovers, and the like in a remote place in a family in a different room in the home or in a wider network environment. In addition, it is possible to realize a communication system that allows users to communicate while viewing the same content shared by users at remote locations, sharing time even if the physical distance is far away You can create a situation to do. For example, a couple having a child and a parent of a couple living far away from this couple can have a conversation while watching the photographed child together.

<Modification 1>
The storage / reproduction device 30 in the present embodiment appropriately controls the image display of the sub-screen based on the movement of the sub-screen and the audio state of the sub-screen from the communication data received from the storage / reproduction device 31 connected via the network 100. The configuration.

  The storage / reproduction device 40, which is the first modification of the present embodiment, is configured to appropriately control the image display of the child screen and the sound volume of the child screen.

  FIG. 4 is a block diagram illustrating a configuration of a storage / reproduction device 40 that is the first modification.

  As shown in FIG. 4, the sub-screen control unit 16 sets the transparency and saturation of the image displayed on the sub-screen, sets the sound volume output as the sub-screen audio, and sets the set value as the image. The data is transmitted to the synthesis unit 19 and the voice synthesis unit 20. Then, the image synthesizing unit 19 synthesizes the sub-screen image on the main screen based on the set transparency and saturation, and the voice synthesizing unit 20 determines the sound of the main screen based on the set sound volume. Synthesizes the voice of the sub-screen.

  Specifically, first, when the sub-screen control unit 16 determines that the image analysis unit 14 and / or the voice analysis unit 15 indicate that communication is being performed, the sub-screen control unit 16 sets the transparency of the image displayed on the sub-screen, for example, By setting “0”% and setting the saturation of the image displayed on the sub-screen to the original state, for example, the image displayed on the sub-screen is set to color display and is output as the audio of the sub-screen. For example, the sound volume is set to “60”% so that the user can easily hear it.

  Further, when the sub-screen control unit 16 determines that the image analysis unit 14 and / or the voice analysis unit 15 does not suggest that communication is being performed, the transparency of the image displayed on the sub-screen is set to, for example, “60”%. By setting the saturation of the image displayed on the sub screen to the minimum, for example, the image displayed on the sub screen is set to black and white display, and the volume of the audio output as the sub screen audio is set to the content on the main screen. For example, it is set to “0”% so as not to disturb the viewing user.

  Then, the image synthesis unit 19 synthesizes the sub-screen image on the main screen based on the transparency and saturation of the image displayed on the sub-screen set by the sub-screen control unit 16, and the voice synthesis unit 20 Then, based on the volume of the audio output as the audio of the sub-screen set by the sub-screen control unit 16, the sound of the main screen and the audio of the sub-screen are synthesized.

  As a result, the storage / reproduction device 40 according to the modified example 1 displays the image of the child screen based on the movement of the child screen and the sound state of the child screen from the communication data received from the memory / reproduction device 41 connected via the network 100. And the sound volume of the sub-screen can be appropriately controlled.

It is the block diagram which showed one Embodiment of the memory | storage / reproducing apparatus based on this invention. It is the flowchart which showed the process sequence of the small-screen display setting process of the storage / reproducing apparatus in this embodiment. It is the flowchart which showed the process sequence of the bidirectional | two-way operation processing of the storage / reproducing apparatus in this embodiment. It is a block diagram which shows the structure of the storage / reproducing apparatus which is the modification 1. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Analysis mode selection interface 2 ... Camera 3 ... Sharp directivity microphone 4 ... Apparatus 5 ... Reproduction operation interface 6 ... Screen input part 7 ... Audio | voice input part 8 ... Content storage part 9 ... External apparatus connection part 10 ... Reproduction operation reception part DESCRIPTION OF SYMBOLS 11 ... Sub-screen encoder part 12 ... Noise removal filter 13 ... Network connection interface 14 ... Image analysis part 15 ... Audio | voice analysis part 16 ... Sub-screen control part 17 ... Sub-screen decoder part 18 ... Content decoder part 19 ... Image composition part 20 ... Speech synthesis unit 21 ... operation delay measurement unit 22 ... AV synchronization unit 23 ... playback unit 24 ... playback control unit 25 ... storage unit 26 ... output unit 30 ... storage playback device 31 ... storage playback device 40 ... storage playback device 41 ... storage playback Device 100 ... Network

Claims (4)

An electronic device having a content distribution function for reading and distributing content data including image data and audio data stored in a storage device, and a bidirectional communication function for transmitting and receiving communication data including captured image data and recorded audio data A storage / reproduction device connected to a device, outputting content data from the electronic device as a main screen, and outputting communication data as a sub-screen so as to overlap the main screen,
When a selection signal indicating whether the image analysis mode or the voice analysis mode is selected by an external input, a motion signal is detected from the communication data received from the electronic device, and the detected motion signal is used for a motion signal. An image analysis unit that determines that communication is in progress when the threshold is exceeded;
When the selection signal indicates a voice analysis mode, voice data is extracted from communication data received from the electronic device, and it is determined that communication is being performed when the output value of the extracted voice data exceeds a voice data threshold value. A voice analysis unit to
If the image analysis unit and / or the voice analysis unit determines that a call is in progress, the transparency and saturation of the image displayed on the sub-screen, and / or the volume of the audio output as the sub-screen audio A sub-screen control unit for setting
The content data of the main screen and the communication data of the child screen based on the transparency and saturation of the image displayed on the child screen set by the child screen control unit or the sound volume output as the sound of the child screen And a synthesizing unit for synthesizing the data. A content storage unit for storing the content data;
An operation delay measuring unit for measuring a response time of the electronic device;
When an operation signal is received from an external input, index information data indicating the title index of content corresponding to the content data stored in the content storage unit is read from the storage unit, and the read index information data corresponds to the operation signal If there is a title index of the content to be processed, the elapsed time from when the operation signal is received is compared with the delay time measured by the operation delay measurement unit, and the elapsed time exceeds the delay time. The storage / reproducing apparatus according to claim 1, further comprising: a reproduction control unit that outputs data synthesized by the synthesizing unit according to the operation signal. An electronic device having a content distribution function for reading and distributing content data including image data and audio data stored in a storage device, and a bidirectional communication function for transmitting and receiving communication data including captured image data and recorded audio data A storage / reproduction program applied to a storage / reproduction device connected to a device, outputting content data from the electronic device as a main screen, and outputting communication data as a sub-screen so as to overlap the main screen,
Storage and playback device
When a selection signal indicating whether the image analysis mode or the voice analysis mode is selected by an external input, a motion signal is detected from the communication data received from the electronic device, and the detected motion signal is used for a motion signal. An image analysis unit that determines that communication is in progress when the threshold is exceeded;
When the selection signal indicates a voice analysis mode, voice data is extracted from communication data received from the electronic device, and it is determined that communication is being performed when the output value of the extracted voice data exceeds a voice data threshold value. A voice analysis unit to
If the image analysis unit and / or the voice analysis unit determines that a call is in progress, the transparency and saturation of the image displayed on the sub-screen, and / or the volume of the audio output as the sub-screen audio A sub-screen control unit for setting
The content data of the main screen and the communication data of the child screen based on the transparency and saturation of the image displayed on the child screen set by the child screen control unit or the sound volume output as the sound of the child screen A storage / playback program for functioning as a synthesis unit that synthesizes. Storage and playback device
A content storage unit for storing the content data;
An operation delay measuring unit for measuring a response time of the electronic device;
When an operation signal is received from an external input, index information data indicating the title index of content corresponding to the content data stored in the content storage unit is read from the storage unit, and the read index information data corresponds to the operation signal If there is a title index of the content to be processed, the elapsed time from when the operation signal is received is compared with the delay time measured by the operation delay measurement unit, and the elapsed time exceeds the delay time. The storage / reproducing program according to claim 3, further causing a reproduction control unit to output data synthesized by the synthesizing unit in accordance with the operation signal.

Images