JP7307306B2 - Information processing device, information processing system, control method, and program - Google Patents

Description

The present invention relates to a method of communication performed by exchanging data.

There is a system that enables communication by bidirectionally transmitting and receiving moving images (video, audio) (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2008-210417

Among such systems, there is a system that has a function of automatically adjusting the picture quality of the video according to the condition of the communication line and performing two-way communication.

Therefore, when the data transfer speed of the communication line is low and the communication band is narrow, the image quality of the video is degraded and block noise occurs, which may make it difficult to see the video being exchanged.

Therefore, between mutual terminals, the method of transmitting and receiving moving images (video and audio) is separated from video and audio, and video and audio are transmitted and received separately. There is a mechanism that selects and implements any of the transmission/reception methods.

If you want to communicate with high-definition video, you can use the latter method to communicate with high-definition video regardless of the condition of the communication line.

However, in the latter method, video and audio are separated and exchanged as still images, so audio is lost in the recorded data. Also, due to the exchange at a low frame rate, there is less information as video.

Therefore, there is a problem that it is difficult to confirm the content of communication.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a system capable of displaying a high-quality image to a user.

The present invention for achieving the above object is an information processing system comprising a device capable of communicating with a first information processing device and a second information processing device, wherein the first information processing device a first transmission means for transmitting image data at a predetermined frame rate to the second information processing device; a second transmission means for transmitting acquired audio data to the second information processing device and the device; and a third transmitting means for transmitting image data shot at a frame rate higher than the predetermined frame rate to the device, wherein the second information processing device transmits the image data by the first transmitting means. a first receiving means for receiving image data; and a second receiving means for receiving audio data transmitted by the second transmitting means; and generating means for synthesizing the audio data and the image data transmitted by the third transmitting means.

Advantageous Effects of Invention According to the present invention, it is possible to display a high-quality image to a user.

1 is a schematic configuration diagram of an information processing system; FIG. It is a block diagram showing an example of a hardware configuration. 1 is a functional block diagram of an information processing system; FIG. It is a flow chart which shows communication communication processing. It is a flow chart which shows communication communication processing. 4 is a flow chart showing processing of an operation at the start of recording. FIG. 10 is a flow chart showing processing of an operation at the end of recording; FIG. FIG. 10 is a flow chart showing processing of operations when combining recorded data; FIG. FIG. 3 is a configuration diagram showing an example of configuration of each API; 4 is a configuration diagram showing an example of configuration of each command; FIG. 4 is a configuration diagram showing an example of the configuration of recorded data; FIG. 4 is a configuration diagram showing the configuration of a login screen; FIG. 4 is a configuration diagram showing the configuration of a communication screen; FIG. 4 is a configuration diagram showing the configuration of a login screen; FIG. FIG. 4 is a configuration diagram showing the configuration of a room selection screen; 4 is a configuration diagram showing the configuration of a communication screen; FIG. FIG. 4 is a configuration diagram showing the configuration of a recorded data confirmation screen;

A schematic configuration of the information processing system according to the present embodiment will be described with reference to FIG.

As an example of the schematic configuration of an information processing system according to the present invention, the information processing system comprises a client terminal 100, a client terminal 101, a network 102, a communication server 103, and a data management server 104. and are communicatively connected.

In this information processing system, the client terminal 100 displays a login screen 1200 (details will be described later) and a communication screen 1300 (details will be described later), and provides client functions for performing various inputs and outputs.

Note that these screens can be provided using, for example, a web browser, but are not limited to such a mode, and can be provided by a generated application.

In this information processing system, the client terminal 101 displays a login screen, a room selection screen, or a communication screen, and provides client functions for performing various inputs and outputs.

These screens can be provided using, for example, a Web browser as described above, but are not limited to such a mode, and can be provided by a generated application.

The communication server 103 is an external platform, and provides a communication function using technology (for example, WebRTC) that enables real-time communication of audio, video, and the like.

The data management server 104 stores data such as images, and depending on the case, synthesizes audio data (hereinafter referred to as audio data) and image data (hereinafter referred to as video data) and provides them to client terminals.

The data management server 104 includes a recording device 105 and a storage device 106. The recording device 105 synthesizes audio data and video data, and the storage device 106 combines video data and audio data and video data. Store the synthesized data.

Next, a schematic hardware configuration of the client terminal 100 and the client terminal 101 will be described with reference to the flowchart shown in FIG.

As shown in FIG. 2, each device includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input controller 205, a video controller 206, a memory A controller 207 and a communication I/F controller 208 are connected.

The CPU 201 comprehensively controls each device and controller connected to the system bus 204 .

ROM 202 or external memory 211 stores BIOS (Basic Input/Output System) and OS (Operating System), which are control programs executed by CPU 201, computer-readable executable programs for realizing this information processing method, and necessary various types of programs. Holds data (including data tables).

A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 . The CPU 201 loads programs and the like necessary for executing processing from the ROM 202 or the external memory 211 to the RAM 203, and executes the loaded programs to realize various operations.

An input controller 205 controls input from an input device 209 such as a keyboard and a pointing device such as a mouse (not shown).

If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger or the like) an icon, cursor, or button displayed on the touch panel.

Also, the touch panel may be a touch panel, such as a multi-touch screen, capable of detecting positions touched by multiple fingers.

Video controller 206 controls display on an external output device such as display 210 .

The display shall include the display of a notebook computer integrated with the main body.

Note that the external output device is not limited to a display, and may be a projector, for example. In addition, an input device is also provided for the device capable of receiving the above-described touch operation.

The video controller 206 can control a video memory (VRAM) for performing display control, and can use a part of the RAM 203 as a video memory area, or provide a separate dedicated video memory. is also possible.

A memory controller 207 controls access to the external memory 211 . External memory can be connected to an external storage device (hard disk), flexible disk (FD), or PCMCIA card slot for storing boot programs, various applications, font data, user files, edit files, and various data via an adapter. A compact flash (registered trademark) memory or the like can be used.

A communication I/F controller 208 connects and communicates with an external device via a network, and executes communication control processing in the network. For example, communication using TCP/IP, telephone lines such as ISDN, and communication using 3G lines for mobile phones are possible.

The CPU 201 enables display on the display 210 by, for example, rasterizing an outline font to a display information area in the RAM 203 . The CPU 201 also allows the user to issue instructions using a mouse cursor (not shown) on the display 210 .

A camera controller 212 controls photographing by a camera 214 . An image obtained by photographing with the camera 214 is stored in a storage area such as the external memory 211 or the ROM 202 .

The voice controller 213 controls sound generation and sound collection by the microphone/speaker 215 . A sound emitted to the microphone/speaker 215 can be acquired and stored as voice data in a storage area such as the external memory 211 or the ROM 202, or can be produced by adjusting the volume.

Note that the communication server 103 and the data management server 104 also have substantially the same configuration, so description thereof will be omitted.

Next, an outline of each function in the information processing system will be described with reference to FIG. 3, and each function will be described together with a flowchart described later.

The client terminal 100 includes a login section 300 , a room management section 301 , a still image acquisition section 302 , a still image display section 303 , a recording start/end section 304 and a transmission/reception section 305 .

The login unit 300 has a function of logging in to the information processing system in the client terminal 100, outputs a login screen 1200 (see FIG. 12), performs authentication using the input user ID and password, and logs in the user. .

The room management unit 301 has functions of entering and exiting a space (hereinafter referred to as a room) in which voice data, video data, and the like are exchanged in real time using the communication server 103 for communication.

It also has a function of distributing audio data, such as the voice uttered by the user who activated the client terminal 100, to the room into which the client terminal 100 has entered. Further, it has a function of generating a recorder activation API 900 (see FIG. 9) and transmitting it to the data management server 104 .

The still image acquisition unit 302 has a function of generating a still image acquisition API 930 (see FIG. 9) and acquiring from the storage device 106 a still image captured by the client terminal 101 and uploaded using the still image acquisition API 930. .

The still image display unit 303 has a function of outputting the still image acquired from the storage device 106 to the communication screen 1300 (see FIG. 13).

The recording start/end unit 304 has a function of generating a recording request command 1000 (see FIG. 10) and transmitting it to the client terminal 101 .

The transmission/reception unit 305 has a function of exchanging various data with other devices.

The client terminal 101 includes a login section 310 , a room management section 311 , a still image upload section 312 , a recording start/end section 313 , a video data storage section 314 , a video data upload section 315 and a transmission/reception section 316 .

The login unit 310 has a login function to the information processing system in the client terminal 101, outputs a login screen 1400 (see FIG. 14), performs authentication using the input user ID and password, and logs in the user. .

The room management unit 311 has a function of entering and leaving a room, and also has a function of distributing voice data such as the voice uttered by the user who activated the client terminal 101 to the room into which the user entered the room.

The still image upload unit 312 generates a still image upload API 920 (see FIG. 9), and uses the still image upload API 920 to upload still image data captured by the camera of the device on which the client terminal 101 operates (hereinafter referred to as still image data) to the storage device 106.

The recording start/end unit 313 has a function of generating a recording start/recording end command 1001 (see FIG. 10) and transmitting it to the recording device 105 .

The image data storage unit 314 has a function of storing image data captured by the camera of the device on which the client terminal 101 operates.

The video data upload unit 315 has a function of generating the video data upload API 950 (see FIG. 9) and transmitting the video data saved in the video data storage unit 314 to the storage device 106 using the video data upload API 950. .

The transmission/reception unit 316 has a function of exchanging various data with other devices.

The recording device 105 includes a login unit 32, a room management unit 321, an audio data storage unit 322, an audio data upload unit 323, a video/audio data acquisition unit 324, a video/audio data synthesis unit 325, a synthetic data upload unit 326, and a transmission/reception unit. A portion 327 is provided.

The login unit 320 has a login function to the information processing system in the recording device 105 .

The room management unit 321 has a function of entering and leaving a room, and the voice data storage unit 322 has a function of acquiring and storing voice data distributed to the room.

The audio data upload unit 323 has a function of generating the audio data upload API 940 (see FIG. 9) and using the audio data upload API 940 to transmit the audio data saved in the audio data storage unit 322 to the storage device 106. there is

The video/audio data acquisition unit 324 has a function of acquiring video data and audio data stored in the storage device 106 from the storage device 106 .

The video/audio data synthesizing unit 325 has a function of synthesizing the video data and audio data acquired by the video/audio data acquiring unit 324 .

The composite data upload unit 326 has a function of generating a composite data upload API 970 (see FIG. 9) and using the composite data upload API 970 to transmit the composite data generated by the video/audio data synthesizer 325 to the storage device 106. .

The transmission/reception unit 327 has a function of exchanging various data with other devices.

The data management server 104 includes an activation unit 330 , a data storage unit 331 , a data synthesis instruction unit 332 and a transmission/reception unit 333 .

Upon receiving the recorder activation API 900 transmitted from the room management unit 301 of the client terminal 100, the activation unit 330 generates the recording preparation API 910 (see FIG. 9) and transmits the recording preparation API 910 to the recorder 105. I have.

The data storage unit 331 has a function of storing still images, audio data, video data, and synthesized data generated in the information processing system.

The data synthesizing instruction unit 332 detects that the video data upload unit 315 has saved the video data in the storage device 106, and generates the video data/audio data synthesizing API 960 (see FIG. 9).

It has a function of transmitting the generated video/audio data synthesizing API 960 to the recording device 105 .

The transmission/reception unit 305 has a function of exchanging various data with other devices.

Communication processing according to the present embodiment will be described with reference to flowcharts shown in FIGS. 4 and 5. FIG.

In step S400, the login unit 300 outputs the login screen 1200 (see FIG. 12), acquires user information such as the user ID and password entered on the login screen 1200, and performs authentication. Then, login to the information processing system is performed.

FIG. 12 shows the configuration of a login screen 1200 output to the client terminal 100. The login screen 1200 includes a user ID entry field 1201 for entering a user ID and a password entry field for entering a password. 1202, and a login button 1203 that is pressed to issue a login instruction.

When the user presses the login button 1203, authentication is performed using the user ID and password entered in the user ID input field 1201 and password input field 1202, and when the user is authenticated, the information processing system is logged in. .

In step S401, room management unit 301 outputs communication screen 1300 (see FIG. 13).

FIG. 13 shows the configuration of a communication screen 1300 output to the client terminal 100. The communication screen 1300 includes a remote photographing button 1301, a recording start/end button 1302, a room selection list box 1303, and a room entry button 1304. , and a room exit button 1305 .

When a remote shooting button 1301 is pressed, a remote shooting instruction is issued to the client terminal 101 of the communication partner.

Recording can be started by pressing a recording start/end button 1302, and when recording starts, the recording start/end button 1302 turns red.

On the other hand, recording can be ended by pressing the recording start/end button 1302 while recording is in progress, and the button turns green when recording ends.

In the room selection list box 1303, it is possible to select a room to be entered. By pressing a room entry button 1304, the selected room can be entered. You can leave from

In this step, when it is detected that the room entry button 1304 is pressed while the room output to the room selection list box 1303 is selected, the user is allowed to enter the selected room.

In step S402, the room management unit 301 distributes the voice acquired by the client terminal 100 to the room.

In step S<b>403 , the room management unit 301 generates the recorder activation API 900 and transmits it to the data management server 104 via the transmission/reception unit 305 .

FIG. 9 shows the configuration of the recorder activation API 900, and the recorder activation API 900 is composed of a user ID and a room ID for identifying a room. Note that the room ID of the room selected in step S401 is set as the room ID.

The recorder activation API 900 is an API for requesting the data management server 104 to activate the recorder 105 from the client terminal 100 .

At step S404, the transmitter/receiver 333 receives the recorder activation API 900 transmitted from the client terminal 100 at step S403.

In step S<b>405 , the activation unit 330 generates the recording preparation API 910 and transmits it to the recording device 105 via the transmission/reception unit 333 .

FIG. 9 shows the configuration of the recording preparation API 910. The recording preparation API 910 includes domains (user ID domains,

subsequent character string) and the room ID.

The recording preparation API 910 is an API for issuing a recording preparation request to the recording device 105. The recording preparation refers to the process of logging into the information processing system and entering the same room as the client terminal 100. Recording preparation processing can be executed by using the domain and room ID information.

At step S406, the transmitter/receiver 327 receives the recording preparation API 910 transmitted from the data management server 104 at step S405.

In step S407, the login unit 320 uses the parameters of the recording preparation API 910 received in step S406 to log in to the information processing system.

In step S408, the room management unit 321 causes the recording device 105 to enter the same room as the room entered in step S401.

In step S<b>409 , the room management unit 321 generates a recording preparation API response 911 and returns it to the data management server 104 via the transmission/reception unit 327 .

FIG. 9 shows the configuration of the recording preparation API response 911, and the recording preparation API response 911 is composed of a message indicating that recording preparation has been completed.

In step S410, the transmission/reception unit 333 receives the recording preparation API response 911 transmitted from the recording device 105 in step S409.

In step S<b>411 , the transmission/reception unit 333 returns the recorder activation API response 901 to the client terminal 100 .

FIG. 9 shows the configuration of the recorder activation API response 901, and the recorder activation API response 901 is composed of a message indicating that recording preparations have been made.

In step S412, the transmission/reception unit 305 receives the recorder activation API response 901 transmitted from the data management server 104 in step S411.

Step S413 indicates that the processing from step S414 to step S519 is repeated at predetermined time intervals (for example, one second intervals).

In step S<b>414 , the still image acquisition unit 302 generates the still image acquisition API 930 (see FIG. 9 ), and the transmission/reception unit 305 transmits it to the data management server 104 .

FIG. 9 shows the configuration of the still image acquisition API 930, and the still image acquisition API 930 includes a tenant ID (user ID domain,

character string below), a room ID, and a file ID (details will be described later).

The still image acquisition API 930 is an API for requesting the data management server 104 from the client terminal 100 to acquire a still image stored in the storage device 106, and uses the tenant ID, room ID, and file ID. , a specific still image can be acquired from the storage device 106 .

In step S415, the transmission/reception unit 333 receives the still image acquisition API 930 transmitted from the client terminal 100 in step S414.

In step S416, still image acquisition unit 334 acquires a still image from storage device 106 according to the parameters of still image acquisition API 930 received in step S415.

In step S417, the transmission/reception unit 333 returns the still image acquired from the storage device 106 in step S416 to the client terminal 100 as the still image acquisition API response 931 (see FIG. 9).

FIG. 9 shows the configuration of the still image acquisition API response 931, and the still image acquisition API response 931 is composed of a status code (such as an error code in HTTP) and a still image acquisition URL.

When the client terminal 100 receives the still image acquisition API response 931, the client terminal 100 can acquire the still image by using the still image acquisition URL.

In step S418, the transmission/reception unit 305 receives the still image acquisition API response 931 returned in step S417, and in step S419, the still image display unit 303 displays the Display a still image on the screen.

In step S500, the login screen 1400 (see FIG. 14) is output by the login unit 310, and the user information such as the user ID and password entered on the login screen 1400 is acquired and authenticated. Then, login to the information processing system is performed.

FIG. 14 shows the configuration of a login screen 1400 output to the client terminal 101. The login screen 1400 includes a user ID entry field 1401 for entering a user ID and a password entry field for entering a password. 1402, and a login button 1403 to be pressed to issue a login instruction.

When the user presses the login button 1403, the user performs authentication using the user ID and password entered in the user ID input field 1401 and password input field 1402, and is logged into the information processing system after being authenticated. .

In step S501, room management unit 311 outputs room selection screen 1500 (see FIG. 15).

FIG. 15 shows the configuration of the room selection screen 1500. On the room selection screen 1500, the user selects a room to enter from the room selection list box 1501, presses the room entry button 1504, and enters the selected room. I can enter.

Also, by pressing a recorded data upload button 1503, video data can be uploaded.

Also, when the return button 1502 is pressed, the previously displayed login screen 1400 is displayed.

When the user enters the room, a communication screen 1600 (see FIG. 16) is displayed.

FIG. 16 shows the configuration of the communication screen 1600. The communication screen 1600 includes a room exit button 1601 to be pressed when leaving the room, a photographing button 1602 for photographing, and a recording start/end button 1603. is pressed, recording can be started, and recording is terminated by pressing the recording start/end button 1603 again.

In step S502, the room management unit 311 distributes the voice acquired by the client terminal 101 to the room.

In step S503, the image data storage unit 314 stores the image captured by the camera operating on the client terminal 101 in the buffer area at a predetermined frame rate (eg, 30 fps). The video saved here is used as video data for subsequent recording processing.

By saving at approximately 30 fps, high frame rate video data can be left, and by saving in the buffer area of the client terminal 101, high quality video data can be left.

Step S504 indicates that the processing from steps S415 to S530 is repeated at intervals of one second.

In step S505, the still image upload unit 312 acquires one still image per second from the video data saved in step S503. The still image acquired here is used for communication with the client terminal 100 .

In step S506, the still image upload unit 312 generates the still image upload API 920 (see FIG. 9). Send to

FIG. 9 shows the configuration of the still image upload API 920, and the still image upload API 920 consists of a tenant ID, room ID, file ID, and still image binary data.

The still image upload API 920 is an API for requesting the data management server 104 from the client terminal 101 to upload a still image to the storage device 106, and includes a tenant ID, room ID, and file ID (for communicating with the other party). Therefore, a unique ID is issued.), the still images saved in the storage device 106 can be uniquely managed.

In step S420, the transmission/reception unit 333 receives the still image upload API 920 transmitted from the client terminal 101 in step S506, and in step S421, the data storage unit 331 acquires the still image included in the still image upload API 920, Save in storage device 106 .

In step S<b>422 , the transmission/reception unit 333 returns the still image upload API response 921 (see FIG. 9 ) to the client terminal 101 .

FIG. 9 shows the configuration of the still image upload API response 921, and the still image upload API response 921 is composed of a status code and a message (message indicating the processing result).

In step S430, the transmission/reception unit 316 receives the still image upload API response 921 returned in step S429.

The processing of the operation at the start of recording in this embodiment will be described with reference to the flowchart shown in FIG.

In step S600, recording start/end unit 304 receives that recording start/end button 1302 on communication screen 1300 has been pressed, and generates recording request command 1000 (see FIG. 10). It is transmitted to the client terminal 101 (room in which the room is entered).

FIG. 10 shows the configuration of the recording request command 1000. The recording request command 1000 includes a command for instructing the start/end of recording, a recording flag (which indicates whether recording is started or stopped). ), and a recording ID (issued so as to be unique in order to communicate with the other party). In this case, instruction information relating to an instruction to start recording is included as a command.

A recording request command 1000 is a command from the client terminal 100 to instruct the client terminal 101 to start/end recording.

The recording ID is information for uniquely identifying video data, audio data, and synthesized data generated by recording processing.

By using the recording ID as the file name of the video data, audio data, and synthesized data, the video data and the audio data can be linked. As a result, even if the video data and the audio data are stored separately, it is possible to generate synthesized data in which the video data and the audio data are integrated by the synthesizing process.

At step S601, the transmitting/receiving unit 316 receives the recording request command 1000 transmitted at step S600.

In step S602, the recording start/end unit 313 acquires the recording start date and time (for example, system date and time).

In step S603, the recording start/end unit 313 generates a recording start/recording end command 1001, and transmits the recording start/recording end command 1001 (see FIG. 10) to the client terminal 100 (the room in which the room is entered). do.

FIG. 10 shows the configuration of the recording start/recording end command 1001. The recording start/recording end command 1001 is composed of a command, a recording flag, a recording ID, and a recording start date and time.

A recording start/recording end command 1001 is a command for instructing the recording start/end from the client terminal 101 to the recording device 105 . Here, a command for instructing the start of recording is included.

In step S604, the transmission/reception unit 327 receives the recording start/recording end command 1001 transmitted from the client terminal 101 in step S603.

In step S605, the room management unit 321 acquires the recording ID and the recording start date and time from the recording start/recording end command 1001 received in step S604.

Step S606 indicates that the process of step S607 is repeated until the recording start/recording end command 1001 is received again.

In step S607, the voice data saving unit 322 saves the voice data delivered to the room in which the user has entered in the buffer area of the web browser.

In step S608, the video data storage unit 314 generates a file name of video data with the structure of the video data 1101 (see FIG. 11).

FIG. 11 shows the configuration of the video data 1101. The video data 1101 is video data generated by the client terminal 101, and the file name is the recording start date and time included in the recording start/recording end command 1001. , and the recording ID, and the file name is composed of the recording start date and time and the recording ID included in the recording start/recording end command 1001 . Since the file name is composed of the recording ID and the recording start date and time, it is possible to link the video data and the audio data in the subsequent synthesizing process.

Step S609 indicates that the process of step S610 is repeated until the recording request command 1000 is received again.

In step S610, the video data stored in the buffer area of the client terminal 101 in step S503 is written to a file.

Operation processing at the end of recording in this embodiment will be described with reference to the flowchart shown in FIG.

In step S700, the recording start/end unit 304 generates a recording request command 1000, and the transmitting/receiving unit 305 transmits the recording request command 1000 to the client terminal 101 (the room the user is in). At this time, instruction information relating to an instruction to end recording is included as a command.

In step S701, the transmission/reception unit 316 receives the recording request command 1000. In step S702, the recording start/end unit 313 generates the recording start/end recording command 1001 (see FIG. 11), A recording start/recording end command 1001 is transmitted to the . Here, the command includes a command to end recording.

At step S703, the transmission/reception unit 327 receives the recording start/recording end command 1001 transmitted at step S702.

In step S704, the audio data storage unit 322 ends the audio data storage process started in step S607.

In step S705, the audio data upload unit 323 generates the audio data upload API 940 (see FIG. 9), and the transmission/reception unit 327 uses the audio data upload API 940 to transmit the saved audio data to the data management server 104. .

FIG. 9 shows the configuration of the voice data upload API 940, and the voice data upload API 940 consists of a tenant ID, room ID, and file ID.

The audio data upload API 940 is an API for requesting the data management server 104 to upload audio data to the storage device 106 from the recording device 105 .

By using the tenant ID, room ID, and file ID, the voice data stored in the storage device 106 can be uniquely managed, and the file name of the voice data to be transmitted is voice data 1100 (see FIG. 11). With this configuration, since the file name is composed of the recording ID and the recording start date and time, it is possible to link the video data and the audio data in the subsequent synthesizing process.

FIG. 11 shows the configuration of audio data 1100. The audio data 1100 is audio data generated by the recording device 105. The file name is the recording start date and time included in the recording start/recording end command 1001. and a recording ID.

In step S706, the transmission/reception unit 333 receives the audio data upload API 940 transmitted from the recording device 105 in step S705.

In step S<b>707 , the data storage unit 331 acquires audio data from the audio data upload API received in step S<b>706 and stores it in the storage device 106 .

In step S<b>708 , the data storage unit 331 generates an audio data upload API response 941 (see FIG. 9 ), and transmits it to the recorder 105 via the transmission/reception unit 333 .

FIG. 9 shows the configuration of the audio data upload API response 941, and the audio data upload API response 941 is composed of a status code and upload URL. By using the upload URL, the recording device 105 can upload audio data.

At step S709, the transmission/reception unit 327 receives the audio data upload API response 941 transmitted from the data management server 104 at step S708.

In step S710, the video data saving unit 314 ends the file writing process of the video data started in step S610, and in step S711, the video data saving unit 314 saves the written video data in the storage area.

Next, the processing of operations when combining recorded data in this embodiment will be described with reference to the flowchart shown in FIG.

At step S800, the video data upload unit 315 generates the video data upload API 950 (see FIG. 9).

FIG. 9 shows the configuration of the video data upload API 950, and the video data upload API 950 consists of a tenant ID, room ID, and file ID.

The video data upload API 950 is an API for requesting the data management server 104 from the client terminal 101 to upload video data to the storage device 106 . By using the tenant ID, room ID, and file ID, the video data saved in the storage device 106 can be uniquely managed.

Using the video data upload API 950, the video data saved in the storage area of the client terminal 101 in step S711 is transmitted to the data management server 104. FIG.

In step S801, the transmission/reception unit 333 receives the video data upload API 950 transmitted from the client terminal 101 in step S800.

In step S802, the data storage unit 331 acquires the video data from the video data upload API 950 received in step S801, and stores it in the storage device .

In step S803, the data storage unit 331 generates a video data upload API response 951 (see FIG. 9), and the transmission/reception unit 333 transmits it to the client terminal 101. FIG.

The video data upload API response 951 is composed of a status code and an upload URL, and the client terminal 101 can upload video data by using the upload URL.

In step S804, the transmission/reception unit 316 receives the video data upload API response 951 transmitted from the data management server 104 in step S803.

In step S<b>805 , the data synthesizing instruction unit 332 generates the video/audio data synthesizing API 960 (see FIG. 9 ), and the transmitting/receiving unit 333 transmits it to the recorder 105 .

FIG. 9 shows the configuration of the video/audio data synthesizing API 960. The video/audio data synthesizing API 960 consists of a video data storage path, an audio data storage path, and a synthesized data storage path.

The video/audio data synthesizing API 960 is an API for requesting the recording device 105 from the data management server 104 to synthesize video data and audio data.

By using the video data storage path and the audio data storage path, the recording device 105 can acquire the video data and audio data to be synthesized from the storage device 106 .

At step S806, the transmission/reception unit 327 receives the video/audio data synthesizing API 960 transmitted from the data management server 104 at step S805.

In step S807, the video/audio data acquisition unit 324 acquires video data from the storage device 106 based on the information of the video/audio data synthesizing API 960 received in step S806.

In step S808, the video/audio data acquisition unit 324 acquires audio data from the storage device 106 based on the information of the video/audio data synthesizing API 960 received in step S806.

In step S809, the video/audio data synthesizing unit 325 performs synthesizing processing of the video data and audio data acquired from the storage device 106 in steps S807 and S808.

In step S810, the video/audio data synthesizing unit 325 saves the synthetic data 1102 obtained by synthesizing in step S809 as a file.

FIG. 11 shows the configuration of the composite data 1102. The composite data 1102 is composite data generated by the recording device 105. The file name is the recording start date and time included in the recording start/recording end command 1001. and a recording ID.

, in step S811, the combined data upload unit 326 generates a combined data upload API 970 (see FIG. 9), and transmits the created data to the data management server 104 via the transmission/reception unit 327. FIG.

FIG. 9 shows the configuration of the composite data upload API 970, and the composite data upload API 970 consists of a tenant ID, room ID, and file ID.

The composite data upload API 970 is an API for requesting the data management server 104 to upload composite data to the storage device 106 from the recording device 105 . By using the tenant ID, room ID, and file ID, the composite data saved in the storage device 106 can be uniquely managed. Synthetic data is transmitted to the data management server 104 using the synthetic data upload API 970 .

In step S812, the transmission/reception unit 333 receives the combined data upload API 970 transmitted from the recording device 105 in step S811. In step S814, the data synthesis instruction unit 332 generates a synthesized data upload API response 971 (see FIG. 9), and the transmission/reception unit 333 transmits it to the recording device 105. FIG.

FIG. 9 shows the configuration of the synthetic data upload API response 971, and the synthetic data upload API response 971 consists of a status code and an upload URL. By using the upload URL, the recording device 105 can upload the synthesized data.

In step S815, the transmission/reception unit 327 receives the combined data upload API response 971 transmitted from the data management server 104 in step S814.

FIG. 17 shows the configuration of a recorded data confirmation screen 1700 for playing back the composite data stored in the data management server 104. The recorded data confirmation screen 1700 includes a screen for identifying the composite data. By displaying a thumbnail image or the like and pressing a play button 1701, synthesized data can be played.

Although the embodiments have been described above, the present invention can be embodied as, for example, a system, method, program, recording medium, or the like.

Specifically, it may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

In addition, the program in the present invention is a program that allows a computer to execute the processing methods of the flow charts shown in FIGS. It is The program according to the present invention may be a program for each processing method of each device shown in FIGS.

As described above, a recording medium recording a program for realizing the functions of the above-described embodiments is supplied to a system or device, and the computer (or CPU or MPU) of the system or device reads the program stored in the recording medium. Needless to say, the object of the present invention can also be achieved by reading and executing.

In this case, the program itself read from the recording medium implements the novel functions of the present invention, and the recording medium recording the program constitutes the present invention.

Examples of recording media for supplying programs include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon A disk or the like can be used.

Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program, the OS (operating system) and the like running on the computer are actually executed. Needless to say, a case where part or all of the processing is performed and the functions of the above-described embodiments are realized by the processing are included.

Furthermore, after the program read from the recording medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read according to the instruction of the program code. It goes without saying that a case where a CPU or the like provided in a function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Moreover, the present invention may be applied to a system composed of a plurality of devices or to an apparatus composed of a single device.

Moreover, it goes without saying that the present invention can also be applied to a case in which a program is supplied to a system or apparatus.

In this case, by loading a recording medium storing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

Furthermore, by downloading and reading out the program for achieving the present invention from a server, database, etc. on the network using a communication program, the system or device can enjoy the effects of the present invention.

It should be noted that the present invention also includes all configurations obtained by combining each of the above-described embodiments and modifications thereof.

100 client terminal 100
101 client terminal 102 network 103 communication server 104 data management server

Claims (5)

An information processing system comprising devices capable of communicating with a first information processing device and a second information processing device,
The first information processing device is
a first transmitting means for transmitting captured image data of a predetermined frame rate to the second information processing apparatus;
a second transmitting means for transmitting the acquired voice data to the second information processing device and the device;
a third transmission means for transmitting image data captured at a frame rate higher than the predetermined frame rate to the device;
with
The second information processing device is
a first receiving means for receiving the image data transmitted by the first transmitting means;
a second receiving means for receiving the audio data transmitted by the second transmitting means;
with
The device comprises:
generating means for synthesizing the audio data transmitted by the second transmission means and the image data transmitted by the third transmission means;
An information processing system comprising: A first information processing device for transmitting image data captured at a predetermined frame rate to a second information processing device and a device capable of communicating with the second information processing device,
a first receiving means for receiving the audio data acquired by the first information processing device;
a second receiving means for receiving image data captured at a frame rate higher than the predetermined frame rate;
generating means for synthesizing the image data received by the second receiving means and the audio data received by the first receiving means;
A device comprising: A control method for an information processing system comprising devices capable of communicating with a first information processing device and a second information processing device,
The first information processing device is
a first transmission step of transmitting captured image data at a predetermined frame rate to the second information processing device;
a second transmission step of transmitting the acquired voice data to the second information processing device and the device;
a third transmission step of transmitting image data captured at a frame rate higher than the predetermined frame rate to the device;
and run
The second information processing device is
a first receiving step of receiving the image data transmitted by the first transmitting step;
a second receiving step of receiving the audio data transmitted by the second transmitting step;
and run
The device comprises:
a generation step of synthesizing the audio data transmitted in the second transmission step and the image data transmitted in the third transmission step ;
A control method for an information processing system, characterized by executing A control method for a first information processing device for transmitting image data captured at a predetermined frame rate to a second information processing device and a device capable of communicating with the second information processing device, comprising:
The device comprises:
a first receiving step of receiving the audio data acquired by the first information processing device;
a second receiving step of receiving image data captured at a frame rate higher than the predetermined frame rate;
a generating step of synthesizing the image data received in the second receiving step and the audio data received in the first receiving step ;
A device control method characterized by executing the computer,
a first receiving means for receiving audio data obtained by a first information processing device that transmits image data captured at a predetermined frame rate to a second information processing device;
a second receiving means for receiving image data captured at a frame rate higher than the predetermined frame rate;
generating means for synthesizing the image data received by the second receiving means and the audio data received by the first receiving means;
A program to function as

Images