JP2007104281A - Data transmission/reception method, multimedia data processing system, terminal, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent multimedia data of the same contents from being repeatedly transmitted and received between terminals in a short time, in a multimedia data processing system. <P>SOLUTION: The cache compatible request transmitter 113 of a terminal 110 transmits an image flag, picked-up image data, an image ID, and parameter data to a terminal 120, on the occasion of the image flag=0 (new); and transmits the image flag, the image ID, and the parameter data, on the occasion of the image flag=1 (reuse). The cache compatible request receiver 121 of a terminal 120 stores picked-up image data and an image ID on a data memory 124 newly, on the occasion of the image flag=0 (new), acquires a corresponding picked-up image from the data memory 124 using the image ID as a search key; and sends the picked-up image data, an image ID, and parameter data to an image processing portion 122, on the occasion of the image flag=1 (reuse). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention provides a first terminal capable of inputting and outputting multimedia data, a second processing for processing the multimedia data transmitted from the first terminal and transmitting the result to the first terminal. The present invention relates to a multimedia data processing system having a terminal.

  Here, the prior art of an image composition system will be described as an example of a multimedia data processing system.

  When a user captures an image of a real space around the user using a terminal having a photographing function and inputs the captured image to the image composition system, the image composition system synthesizes a virtual object or character, and the result is obtained. Conventionally, a system for outputting and presenting it to a user has been proposed.

  In particular, when a captured image includes a marker to be recognized, that is, a specific printed matter or an object having a specific shape, the system performs an image recognition process to determine the position of the marker existing in the real space. The direction can be calculated, and the size and orientation of the object to be combined can be reflected in the combining process, and the image can be combined as if something that does not exist in the real space exists. Such an example is proposed in Non-Patent Document 1.

  Furthermore, when the processing capability of the terminal used by the user is small, the above object can be achieved by causing another server terminal connected on the network to perform processing with a large load. Such an example is proposed in Patent Document 1 and Patent Document 2.

  FIG. 9 shows the configuration of a conventional image composition system as described above.

  The terminal 210 is a camera-equipped mobile phone or PC (Personal Computer) held by the user, and includes a request acquisition unit 211, a data storage unit 212, a request transmission unit 213, a request reception unit 214, and a result display unit 215. The terminal 220 is a server terminal or the like connected to the terminal 210 via a network (not shown), and includes a request reception unit 221, an image processing unit 222, and a result transmission unit 223.

  The user captures an image of his / her surroundings by the image capturing function 211A of the request acquisition unit 211. At this time, the user performs imaging so that the marker to be recognized is included in the captured image. The image capturing function 211A generates captured image data and an image ID for uniquely identifying the captured image data. In addition, the user acquires “what he wants to synthesize” from the candidate list by the parameter acquisition function 211B of the request acquisition unit 211 and decides it. The composite ID of the one selected by the user is set as parameter data. The captured image data acquired by the image capturing function 211A and the parameter data acquired by the parameter acquisition function 211B are sent from the request acquisition unit 211 to the data storage unit 212 and stored therein.

  When the user completes image capturing and parameter setting, the request acquisition unit 211 outputs captured image data and parameter data to the request transmission unit 213. The request transmission unit 213 transmits the captured image data and parameter data to the terminal 220 according to a communication protocol.

  The request receiving unit 221 of the terminal 220 receives captured image data and parameter data, and sends them to the image processing unit 222. The image processing unit 222 performs image processing based on the captured image data and parameter data sent thereto. In this example, the marker included in the captured image data is recognized by the image recognition process, the position and direction of the marker are calculated, the composite specified by the parameter data is combined with the obtained position and direction, and the composite is processed. Result image data is generated. The generated result image data is sent to the result transmission unit 223. The result transmission unit 223 transmits the result image data to the terminal 210 according to the communication protocol.

  The result receiving unit 214 of the terminal 210 receives the result image data and sends it to the result display unit 215. The result display unit 215 displays the sent image data on a device such as a display, thereby presenting the result obtained by the user's request to the user. As a result, the user can view an image as if a virtual object that does not exist in the real space exists.

After the series of operations so far, when the user performs the image composition process again after changing the conditions, the request acquisition unit 211 reads out the previous captured image data and the previous setting parameter data data storage unit 212 and reuses them. For this reason, when the same captured image data as the previous one is used, it is not necessary to perform another imaging, and only the parameters need be updated. Similarly, when the same parameter data as the previous time is used, it is not necessary to set parameters again, and only the captured image needs to be updated.
H. Matsuoka et al. , “Regeneration of Real Objects in the Real World”, SIGGRAPH 2002, San Antonio, 21-26 JULY, 2002. JP 2002-111998 A JP 2004-341642 A

  In the conventional image composition system described above, captured image data and parameter data are transmitted from the terminal 210 to the terminal 220 every time the user requests processing. Of these, multimedia data such as photographed image data generally has a large amount of data. After the user obtains the first composite result, the same captured image data as the previous one is used, and even when only the composite is changed, the conventional system uses the same captured image data as the previous one. Transmission from the terminal 210 to the terminal 220 was necessary.

  As described above, in the conventional image composition system, it is necessary to transmit multimedia data that is identical in content many times, which increases the amount of communication data, resulting in the disadvantage of increased communication cost and communication time. Was bringing to the user.

An object of the present invention is from the user side terminal, already free method of transmitting and receiving data without contents to transmit the same multimedia data, the multimedia data processing system, to provide terminal, and a program.

  In order to achieve the above object, according to the data transmission / reception method of the present invention, the terminal on the user side does not transmit the multimedia data for the multimedia data once transmitted, and processes the processing of the multimedia data without transmitting the multimedia data. The terminal that transmits to the terminal that performs the processing of the multimedia data stores the received multimedia data in the data storage unit, does not receive the multimedia data, and receives the multimedia data ID, the data storage The multimedia data corresponding to the ID is retrieved from the section.

In addition, the multimedia data processing system of the present invention,
When the terminal on the user side is new multimedia data, the multimedia data and its ID are processed. In the case of multimedia data that has been transmitted in the past, the multimedia data is processed into multimedia data. It has a cache-compatible request transmitter that transmits to the terminal,
When a terminal that processes multimedia data receives a data storage unit that locally stores the multimedia data transmitted from the user terminal together with its ID, and receives the multimedia data and its ID from the user terminal. These are stored in the data storage unit, and the multimedia data is sent to the multimedia data processing unit for processing the multimedia data, and the multimedia data is not received from the terminal on the user side. When the ID is received, the data storage unit retrieves the multimedia data of the ID and has a cache correspondence request receiving unit that sends it to the multimedia data processing unit.

  Thereby, it is possible to avoid the multimedia data having the same content being repeatedly transmitted and received in a short time.

  In addition, when the terminal for processing multimedia data is a server used by a plurality of users, the storage capacity of the data storage unit is limited or a trouble occurs during server maintenance. There is a possibility that multimedia data already sent to the server may be lost and data inconsistency may occur. In such a case, the cache request receiving unit of the terminal that processes the multimedia data is The system can operate normally by requesting retransmission of multimedia data to the cache correspondence request transmission unit of the terminal.

  ADVANTAGE OF THE INVENTION According to this invention, it can avoid that the multimedia data of the same content is repeatedly transmitted / received in a short time, can reduce communication data amount, and can suppress the increase in communication cost and communication time as a result.

  Next, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a diagram showing the configuration of an image composition system according to the first embodiment of the present invention.

  The terminal 110 is a camera-equipped mobile phone or a PC owned by the user, and includes a request acquisition unit 111, a data storage unit 112, a cache-compatible request transmission unit 113, a request reception unit 114, and a result display unit 115. The terminal 120 is a server terminal or the like connected to the terminal 110 via a network (not shown), and includes a cache correspondence request reception unit 121, an image processing unit 122, a result transmission unit 123, and a data storage unit 124.

  The request acquisition unit 111 provides a human machine interface (HMI), is responsible for acquiring a user request, and provides an image capturing function 111A and a parameter acquisition function 111B. The user may shoot a new photographic image and may reuse data captured in the past, and an “image flag” is introduced to distinguish them. When taking a new image, the user takes an image around him / her with the image shooting function 111 </ b> A of the request acquisition unit 111. At this time, the user performs imaging so that the marker to be recognized is included in the captured image. The image capturing function 111A generates captured image data and an image ID for uniquely identifying the captured image data, and sets the image flag = 0 (new). When reusing data captured in the past, the request acquisition unit 111 displays the captured image data stored in the data storage unit 112 as a selection candidate, causes the user to determine, acquires the captured image data and the image ID, Set image flag = 1 (reuse). In addition, the user acquires and determines “what he wants to synthesize” from the candidate list by the parameter acquisition function 111B of the request acquisition unit 111. The composite ID of the one selected by the user is set as parameter data. When the user completes image capturing and parameter setting, the image flag, captured image data, image ID, and parameter data are sent from the request acquisition unit 111 to the data storage unit 112. The data storage unit 112 stores captured image data and an image ID when the image flag = 0 (new). When the image flag = 1 (reuse), the data storage unit 112 does not store these data because they already exist. In any case, the data storage unit 112 adds the image ID and parameter data to the stored content as the most recent user request, and is used as a default value when the terminal application is activated next time. When the storage process to the data storage unit 112 is completed, the request acquisition unit 111 transmits the image flag, the captured image data, the image ID, and the parameter data to the cache correspondence request transmission unit 113. The cache correspondence request transmission unit 113 transmits the image flag, captured image data, image ID, and parameter data to the terminal 120 when the image flag = 0 (new), and the image flag when the image flag = 1 (reuse). The image ID and parameter data are transmitted to the terminal 120.

  The cache correspondence request reception unit 121 of the terminal 120 newly stores the captured image data and the image ID in the data storage unit 124 when the image flag = 0 (new), and the image flag = 1 (reuse) when the image flag = 1 (reuse). Using the ID as a search key, the corresponding captured image is acquired from the data storage unit 124, and the captured image data, the image ID, and the parameter data are sent to the image processing unit 122. Similar to the conventional system of FIG. 9, the image processing processing unit 122 recognizes the received captured image data, synthesizes the composite specified by the parameter data at the marker position, and generates result image data. To do. The generated result image data is sent to the result transmission unit 123. The result transmission unit 123 transmits the result image data to the terminal 110 according to the communication protocol.

  The result receiving unit 114 of the terminal 110 receives the result image data and sends it to the result display unit 115. The result display unit 115 displays the transmitted image data on a device such as a display, and presents the result obtained by the user's request to the user. As a result, as in the conventional case, the user can view an image as if a virtual object that does not exist in the real space exists there.

  Thereafter, in response to a request to change only the composite while the captured image data remains the same, the image flag = 1 (reuse), image data is not transmitted, and communication cost and communication time are reduced. be able to.

  FIGS. 2 to 8 show display examples of screens when a mobile phone application program is installed in the user terminal 110. FIG. 2 shows a screen when the application program is first started. The photographed image data and parameter data are unconfirmed. FIG. 3 shows a screen during image data acquisition by the image capturing function 111A. FIG. 4 shows that the captured image data is confirmed and the parameter data is unconfirmed. FIG. 5 shows a state in which a composite to be parameter data is selected from the candidate list. FIG. 6 shows a screen when both the captured image data and the parameter data are determined and a request for the composition process is made to the server side terminal 120. At this time, the screen flag = 0 (new), and the image data is transmitted to the server-side terminal 120. FIG. 7 shows a state in which the result image by the image processing unit 122 of the server side terminal 120 is displayed by the result display unit 115 of the user side terminal 110. FIG. 8 shows a screen when only the composite is changed and the server side terminal 120 is requested again for the synthesis process. At this time, the image flag = 1 (reuse), and the image data is not transmitted to the server side terminal 120.

[Second Embodiment]
In the first embodiment, it is assumed that the consistency of image data stored in the data storage unit 112 of the user side terminal 110 and the data storage unit 124 of the server side terminal 120 is maintained. However, when the system is actually operated, the consistency of the image data stored in both data storage units 112 and 124 is lost due to the intentional erasure due to the limitation of the respective data storage capacity or the maintenance reason. there is a possibility.

As a situation where consistency is broken,
1) Data existing in the data storage unit 112 of the user side terminal 110 does not exist in the data storage unit 124 of the server side terminal 120 2) Data existing in the data storage unit 124 of the server side terminal 120 is data of the user side terminal 110 There are two possible cases that do not exist in the storage unit 112.

  In the former case, that is, in the case where data existing in the data storage unit 112 of the user side terminal 110 does not exist in the data storage unit 124 of the server side terminal 120, the cache correspondence request reception unit 121 sets the image flag = 1 (re- In other words, it is detected that the captured image data corresponding to the image ID does not exist in the data storage unit 124. Therefore, the cache correspondence request reception unit 121 requests the cache correspondence request transmission unit 113 to retransmit the captured image data corresponding to the image ID. The cache response request transmission unit 113 acquires corresponding captured image data from the data storage unit 112 using the image ID as a search key, and transmits the acquired captured image data to the cache response request reception unit 121. The cache correspondence request receiving unit 121 that has received the captured image data stores it in the data storage unit 124. By the operation so far, it becomes possible to continue the processing with the contents of the normal operation shown in the first embodiment.

  In the latter case, that is, when data existing in the data storage unit 124 of the server-side terminal 120 does not exist in the data storage unit 112 of the user-side terminal 110, it is impossible to refer to past captured image data in the request acquisition unit 111. The user inevitably takes a new photographed image. Therefore, it is possible to continue the processing with the contents of the normal operation shown in the first embodiment as a result of taking new image data. However, in the data storage unit 124 of the server-side terminal 120, the captured image data that is not reused may remain unsupported. In this case, the server operator can unilaterally delete the captured image data that has passed for a fixed time in the data storage unit 124 during maintenance. Even if unilateral erasure is performed, the processing can be continued without the user being aware of data inconsistency due to the above-described retransmission request to the cache correspondence request transmission unit 113 by the cache correspondence request reception unit 121. Become.

[Third embodiment]
In the first and second embodiments, a captured image is taken as an example of multimedia data with a large amount of data, but the multimedia data handled in the present invention may be a moving image or sound.

  As a service example that handles moving images, a service that captures moving image data at the user-side terminal 110, synthesizes a virtual object to be animated at the server-side terminal 120, and outputs a moving image can be considered. In addition, as a service example that handles voice, a service in which the user's terminal 110 records the user's speech and the server side terminal 120 performs voice color conversion to output the voice can be considered.

  Further, the input and output need not be in the same format, and can be applied to a case where the content is converted into a content format (PDF, Flash content, etc.) that can be displayed by a commonly distributed viewer.

[Fourth Embodiment]
The functions of the user-side terminal 110 and the server-side terminal 120 are recorded on a computer-readable recording medium in addition to those realized by dedicated hardware. A program recorded on a medium may be read into a computer system and executed. Here, the computer-readable recording medium refers to a recording medium such as a flexible disk, a magneto-optical disk, and a CD-ROM, and a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short time, as in the case of transmitting a program via the Internet, and a server in that case. Some of them hold programs for a certain period of time, such as volatile memory inside computers.

It is a figure which shows the structure of the image composition system by the 1st Embodiment of this invention. It is a figure of a screen when starting an application program for the first time. It is a figure of the screen which shows during image data acquisition by the image photographing function 111A. It is a figure of a screen which shows that photographic image data is confirmed and parameter data is unconfirmed. It is a figure of a screen which shows a mode that the compound used as parameter data is selected from a candidate list. It is a figure of a screen when both photographed image data and parameter data are confirmed and a request for composition processing is made to the server side terminal 120. It is a figure of the screen which shows a mode that the result image by the image process part 122 of the server side terminal 120 is displayed by the result display part 115 of the user side terminal 110. FIG. It is a figure of a screen when changing only a compound and making a request for composition processing to the server side terminal 120 again. It is a figure which shows the structure of the prior art example of an image composition system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 110 User side terminal 111 Request acquisition part 111A Image photographing function 111B Parameter acquisition function 112 Data storage part 113 Cache corresponding | compatible request receiving part 114 Result receiving part 115 Result display part 120 Server side terminal 121 Cache corresponding | compatible request receiving part 122 Image processing part 123 Result transmission unit 124 Data storage unit

Claims (8)

A first terminal capable of inputting and outputting multimedia data; a second terminal that processes the multimedia data transmitted from the first terminal and transmits the result to the first terminal; In a data transmission / reception method in a multimedia data processing system having
The first terminal transmits the ID to the second terminal without transmitting the multimedia data with respect to the multimedia data transmitted once.
When the second terminal stores the received multimedia data in the data storage unit and does not receive the multimedia data but receives the ID of the multimedia data, the multimedia corresponding to the ID is received from the data storage unit. A data transmission / reception method in a multimedia data processing system, characterized by searching for data.   The second terminal requests retransmission of multimedia data corresponding to the received ID to the first terminal when multimedia data corresponding to the received ID does not exist in the data storage unit. 2. The data transmission / reception method according to 1. A first terminal capable of inputting and outputting multimedia data; a second terminal that processes the multimedia data transmitted from the first terminal and transmits the result to the first terminal; In a multimedia data processing system having
In the case of new multimedia data, the first terminal sends the multimedia data and its ID to the second terminal in the case of multimedia data that has been transmitted in the past. A cache-compatible request transmitter for transmitting,
When the second terminal receives the multimedia data transmitted from the first terminal together with its ID, the data storage unit and the multimedia data and its ID from the first terminal, Is stored in the data storage unit and the multimedia data is sent to a multimedia data processing unit for processing the multimedia data, and multimedia data is not received from the first terminal. A multimedia data processing system having a cache correspondence request receiving unit that retrieves multimedia data of the ID from the data storage unit and sends the multimedia data to the multimedia data processing unit.   The cache correspondence request transmission unit transmits a multimedia data flag that identifies new multimedia data or multimedia data that has been transmitted in the past to the second terminal, and the cache correspondence request reception unit The multimedia data processing system according to claim 3, wherein the processing is performed based on a multimedia data flag.   The cache correspondence request receiving unit requests the first terminal to retransmit the multimedia data corresponding to the received ID when the multimedia data corresponding to the received ID does not exist in the data storage unit. Item 5. The multimedia data processing system according to Item 3 or 4. A data storage unit;
Acquiring multimedia data, storing it in the data storage unit together with a multimedia data ID for identifying the multimedia data, and acquiring multimedia data from the data storage unit in the case of multimedia data acquired in the past And
In the case of new multimedia data, the multimedia data and its ID. In the case of multimedia data that has been transmitted in the past, this multimedia data is transmitted to a terminal that performs processing of the multimedia data. A request transmitter;
A result receiving unit for receiving processed multimedia data from the terminal;
And a result display unit for displaying the received multimedia data. A multimedia data processing unit for processing multimedia data transmitted from the user terminal;
A result transmitting unit for transmitting the processed multimedia data to the user terminal;
A data storage unit for locally storing multimedia data transmitted from the user terminal together with its ID;
When multimedia data and its ID are received from the user side terminal, they are stored in the data storage unit, and the multimedia data is sent to the multimedia data processing unit, and the multimedia data is sent from the first terminal. When the multimedia data ID is received without receiving the data, the terminal having the cache correspondence request receiving unit that searches the multimedia data of the ID from the data storage unit and sends the multimedia data to the multimedia data processing unit.   The program for performing the function of the terminal of Claim 6 or 7 with a computer.