JP2016174289A - Server device, moving image data reproduction method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To distribute moving image data to a mobile terminal so that a moving image is reproduced in overlap with a printed matter by imaging the print matter, output from an image printer, by the mobile terminal.SOLUTION: When a mobile terminal 4 registers in a server device 1 moving image data and image data which are preserved in a memory 41, the server device 1 notifies the mobile terminal 4 of data preservation location information. The mobile terminal 4 transfers the image data and the preservation location information to an image printer 2. The image printer 2 prints on a printing paper a composite image obtained by composing a code image, having coded preservation location information, with the image data, to output a printed matter 3. The mobile terminal 4 reads the preservation location information from the code image of the printed matter 3, to reproduce the moving image data acquired by accessing the server device 1 when communicable with the server device 1, or reproduce the moving image data preserved in the memory 41 when not communicable with the server device 1, in overlap with the image of the printed matter 3 imaged by a camera 42.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a server device, a moving image data reproducing method, and a program for reproducing a moving image by AR technology in a mobile terminal.

  Image printing apparatuses that read digital data of images taken with a digital camera, a smartphone, or the like and print them out are installed in various places. For example, the user brings a storage medium in which image data taken with a digital camera is stored, and causes the image printing apparatus to read the image data in the storage medium. The image printing apparatus displays an image of the image data read from the storage medium and presents it to the user. When the user selects an image to be printed and designates the print size and the number of prints, the image printing apparatus prints out the image at the designated size. In such an image printing apparatus, user needs are diversified, and new services are always required.

  Conventionally, augmented reality (AR) technology is known in which an image of a virtual object or the like is superimposed and displayed on an image of a real space photographed by a mobile terminal such as a tablet terminal or a smartphone (for example, Patent Document 1). 2).

JP 2011-8752 A Special table 2012-503228 gazette

  The present invention has been made in view of the above-described conventional situation, and by photographing a printed matter output from the image printing apparatus with a portable terminal, the portable terminal is configured so that a moving image is superimposed and reproduced on the printed matter. It is an object to provide a server device, a moving image data reproduction method, and a program for distributing moving image data to a computer.

  A program according to an aspect of the present invention includes a transmission unit that transmits moving image data and image data stored in a storage unit to a server device, a reception unit that receives data storage location information from the server device, and the image data And a transfer unit that transfers the storage location information to an image printing apparatus, a reading unit that reads the storage location information from a code image printed on a printed matter created by the image printing device, and a reading of the storage location information Thereafter, when communication with the server device is possible, an acquisition unit that accesses the server device based on the storage location information to acquire the image data, and when communication with the server device is possible, from the server device Compare the feature points of the acquired image with the image on the printed matter acquired using the camera, and communicate with the server device A comparison unit that compares feature points of an image stored in the storage unit and an image on the printed matter acquired using the camera, an image acquired from the server device, and the camera When the feature point matches the image on the printed matter acquired using the server, the moving image data is received from the server device, the received moving image data is reproduced, and the image stored in the storage unit A function of a computer of a portable terminal as a playback unit that plays back moving image data stored in the storage unit when a feature point matches an image on the printed matter acquired using the camera It is.

  The program according to one aspect of the present invention may be configured such that when the moving image data or the image data is not stored in the storage unit when the reproducing unit reproduces the moving image received from the server device, the moving image is transmitted from the server device. The computer is caused to function so as to acquire data or the image data and store the data or the image data in the storage unit.

  A moving image data reproduction method according to an aspect of the present invention is a moving image data reproduction method using a portable terminal, an image printing apparatus, and a server apparatus, wherein the portable terminal stores moving image data and image data stored in a storage unit. To the server device, the server device stores the moving image data and image data received from the mobile terminal, transmits data storage location information to the mobile terminal, and the mobile terminal transmits the image data. And the storage location information is transferred to the image printing device, and the image printing device prints a composite image obtained by synthesizing the code image obtained by encoding the storage location information and the image data on a photographic printing paper. Output, the portable terminal uses a camera to read the code image printed on the printed matter to obtain the storage location information, When communication with the server device is possible, the image data is acquired by accessing the server device based on the storage location information, the image acquired from the server device, and the printed matter acquired using the camera When the feature point is compared with the above image, and the feature point matches, the moving image data is received from the server device, the received moving image data is reproduced, and the server device cannot be communicated, the storage unit The feature points of the image data stored in the image and the image on the printed matter acquired using the camera are compared, and when the feature points match, the moving image data stored in the storage unit is reproduced. It is characterized by doing.

  A server device according to an aspect of the present invention is a server device that stores image data and moving image data corresponding to the image data, and transmits the moving image data to a mobile terminal, wherein the image data and the server device The moving image data is transmitted to the mobile terminal that has been accessed by reading the storage location information from the code image of the printed matter on which the code image obtained by encoding the storage location information of the moving image data is printed. When the moving image data is reproduced for the first time, the image data and the moving image data are transmitted so as to be stored in the portable terminal.

  A program according to an aspect of the present invention includes a reading unit that reads storage location information of moving image data in a server device from a code image printed on a printed matter, and an image obtained by accessing the server device based on the storage location information. An acquisition unit that acquires data, an image acquired from the server device, a comparison unit that compares feature points of an image on the printed matter acquired using a camera, and the feature points match, A program that causes the mobile terminal computer to function as a playback unit that receives the moving image data from the server device and plays back the received moving image data, the image data received from the server device during the first playback of the moving image data And the moving image data is stored in the storage unit, and in the second and subsequent reproductions, when communication with the server device is possible, Image data is acquired from the server device, the feature point comparison is performed by the comparison unit, the moving image data is received from the server device, the reproduction unit reproduces, and communication with the server device is not possible. The comparison unit performs feature point comparison using stored image data, and the playback unit causes the computer to function so as to play back the moving image data stored in the storage unit.

  A moving image data reproducing method according to an aspect of the present invention is a moving image data reproducing method using a mobile terminal, an image printing apparatus that stores image data, and a server apparatus that stores the image data and moving image data. When the image printing device accepts the purchase order of the image data, the image data storage location information of the moving image data in the server device is encoded to generate a code image, and a composite image obtained by combining the image data and the code image is generated. Printing on photographic paper and outputting a printed matter, the portable terminal reads the storage location information from the code image printed on the printed matter using a camera, and sends it to the server device based on the storage location information Access to obtain the image data, the image obtained from the server device and the camera obtained using the camera When the feature points of the image on the lint are compared, and the feature points match, the moving image data is received from the server device, the received moving image data is reproduced, and the server device When the moving image data is reproduced for the first time, the image data and the moving image data are transmitted so as to be stored in the storage unit of the portable terminal.

  In the moving image data reproduction method according to one aspect of the present invention, when the portable terminal cannot communicate with the server device, the image data stored in the storage unit and the image on the printed matter acquired using the camera And when the feature points match, the moving image data stored in the storage unit is reproduced.

  A program according to an aspect of the present invention includes a transmission unit that transmits moving image data and image data stored in a storage unit to a server device, a reception unit that receives data storage location information from the server device, and the image data And a transfer unit that transfers the storage location information to an image printing apparatus, a reading unit that reads the storage location information from a code image printed on a printed matter created by the image printing device, and a reading of the storage location information Then, when the moving image data and the image data are stored in the storage unit, the image data is taken out from the storage unit, and when the moving image data or the image data is not stored in the storage unit, An acquisition unit that accesses the server device based on the storage location information to acquire the image data, and the acquisition unit A comparison unit that compares feature points of an image extracted from the storage unit or an image acquired from the server device with an image on the printed matter acquired using a camera, an image extracted from the storage unit, and When the feature point matches the image on the printed matter acquired using the camera, the moving image data stored in the storage unit is reproduced, and the image acquired from the server device and the camera are used. When the feature point matches the acquired image on the printed matter, the moving image data is received from the server device, and the computer of the portable terminal is caused to function as a reproducing unit that reproduces the received moving image data. .

  The program according to an aspect of the present invention is based on the storage location information when reading the storage location information of the moving image data in the server device from the code image printed on the printed matter, and when the storage location information is read for the first time. An acquisition unit that accesses the server device to acquire image data, an image acquired from the server device, a comparison unit that compares feature points of an image on the printed matter acquired using a camera, A program that causes the mobile terminal computer to function as a playback unit that receives the video data from the server device when the feature points match and plays back the received video data, and at the time of the first playback of the video data The image data and the moving image data received from the server device are stored in a storage unit, and the storage location information is stored for the second time. When reading down, the comparison unit compares feature points between the image stored in the storage unit and the image on the printed matter acquired using a camera, and the reproduction unit stores the image in the storage unit. The computer is caused to function so as to reproduce the stored moving image data.

  According to the present invention, moving image data can be distributed to a portable terminal so that the moving image is superimposed on the printed material and reproduced by photographing the printed material output from the image printing apparatus with the portable terminal.

1 is a schematic configuration diagram of a moving image data reproduction system according to a first embodiment. It is a figure which shows the example of the printed matter output from an image printing apparatus. It is a block diagram of a portable terminal. It is a functional block diagram implement | achieved by running a moving image registration reproduction | regeneration application. It is a block diagram of a server apparatus. It is a block diagram of an image printing apparatus. It is a figure which shows the example of the display screen at the time of the image purchase in an image printing apparatus. It is a sequence diagram explaining the moving image data reproduction method which concerns on 1st Embodiment. It is a schematic block diagram of the moving image data reproduction system which concerns on 2nd Embodiment. It is a figure which shows the example of the printed matter output from an image printing apparatus. It is a block diagram of a server apparatus. It is a figure which shows the example of management information. It is a block diagram of an image printing apparatus. It is a functional block diagram implement | achieved by running an image sales program. It is a figure which shows the example of the display screen at the time of the image purchase in an image printing apparatus. It is a block diagram of a portable terminal. It is a functional block diagram implement | achieved by running a video reproduction application. It is a sequence diagram explaining the moving image data reproduction method which concerns on 2nd Embodiment. It is a flowchart explaining the moving image data reproduction method which concerns on 2nd Embodiment. It is a figure which shows the example of management information.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a schematic configuration diagram of a moving image data reproduction system according to the first embodiment of the present invention. As shown in FIG. 1, the moving image data distribution system includes a server device 1, an image print device 2, and a mobile terminal 4. The mobile terminal 4 is a smartphone or a tablet device having a camera or a communication function. The portable terminal 4 registers image data and moving image data in the server device 1. The moving image data is, for example, a moving image shot with the camera of the mobile terminal 4. The image data is, for example, still image cut out from moving image data or image data taken by a camera.

  When the server device 1 receives the image data and the moving image data from the mobile terminal 4, the server device 1 transmits storage location information (for example, URL or content ID) for accessing the moving image data to the mobile terminal 4.

  When the user transfers image data and storage location information from the portable terminal 4 to the image printing apparatus 2 and pays a fee, the image printing apparatus 2 encodes the storage location information to generate a two-dimensional code, and generates a two-dimensional code image. And the image of the image data received from the portable terminal 4, print the combined image on photographic paper, and output the printed matter 3. Thereby, as shown in FIG. 2, the printed matter 3 on which the image 31 and the two-dimensional code 32 are printed is created.

  When the user photographs the two-dimensional code 32 on the printed matter 3 with the camera of the portable terminal 4, the portable terminal 4 reads the two-dimensional code 32 to acquire storage location information, and accesses the server device 1.

  The portable terminal 4 acquires image data from the server device 1 and instructs the user to take an image 31 of the printed matter 3 with the camera of the portable terminal 4. When the user captures the image 31 with the camera of the mobile terminal 4, the mobile terminal 4 compares and compares the feature points of the captured image 31 and the image acquired from the server device 1. When the two images match each other by the collation, the mobile terminal 4 acquires the moving image data corresponding to the image 31 from the server device 1 and reproduces the moving image on the display. A known AR technique is used for moving image reproduction, and the moving image is superimposed and reproduced on the printed matter 3 photographed by the camera of the mobile terminal 4.

  When the portable terminal 4 reads the two-dimensional code 32 on the printed matter 3 and acquires the storage location information, if the portable terminal 4 is out of the communication range or the server device 1 is down, the portable terminal 4 The device 1 cannot be accessed. In this case, the portable terminal 4 compares the feature points of the image stored in the memory of its own device registered in the server device 1 and the image 31 of the printed matter 3 captured by the camera of the portable terminal 4. Collate. When the two images match each other as a result of the collation, the mobile terminal 4 reproduces the moving image on the display using the moving image data stored in the memory of the own device registered in the server device 1. Even when the mobile terminal 4 cannot access the server apparatus 1, the mobile terminal 4 reproduces the moving image superimposed on the printed matter 3 photographed by the camera of the mobile terminal 4 using the data stored in the memory of the mobile terminal 4.

  As described above, in the moving image data reproduction system, image data and moving image data are registered from the mobile terminal 4 to the server device 1, and the server device 1 notifies the mobile terminal 4 of storage location information of the moving image data. The image printing apparatus 2 sells the printed matter 3 on which the image data and the storage location information are transferred from the portable terminal 4 and the image 31 and the two-dimensional code 32 are printed.

  A user who has purchased the printed matter 3 can read the two-dimensional code 32 with the mobile terminal 4, access the server device 1 to receive / reproduce the moving image data, and view the moving image corresponding to the image 31. When the mobile terminal 4 cannot access the server device 1, the mobile terminal 4 can view a moving image corresponding to the image 31 using data stored in the memory of the mobile device 4. Thereby, the printed matter 3 can be enjoyed at any time as a photograph with a moving image.

  FIG. 3 is a block configuration diagram of the mobile terminal 4. The mobile terminal 4 includes a CPU 40, a memory 41, a camera 42, a communication unit 43, and a display (touch panel) 44.

  The memory 41 stores moving image data and a moving image registration / reproduction application (application program). The memory 41 includes a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, an SD or XD memory), a RAM (Random Access Memory), an SRAM (Static Random Access Memory), a ROM (Read- It includes at least one type of storage medium such as only memory (EEPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), and magnetic memory.

  When the CPU 40 executes the moving image registration / playback application, as shown in FIG. 4, a registration unit 50, a transfer unit 60, and a playback unit 70 are realized, and data registration to the server device 1 and data to the image printing device 2 are realized. Transfer, acquisition / reproduction of moving images from the server device 1 and the like can be performed.

  The registration unit 50 includes a cutout unit 51, an image selection unit 52, a data transmission unit 53, and a storage location information reception unit 54.

  The cutout unit 51 cuts out a still image from the moving image data stored in the memory 41. The cutout unit 51 detects a portion (referred to as a cut or a cut break) where the scene of the moving image data changes greatly, and divides the moving image data for each detected cut. An image between adjacent frames is divided into a grid (rectangle), an average of pixel values for each rectangle is obtained, and when the absolute value of the difference between the average values is large, it is determined that the place is a cut. A representative image is determined from among a plurality of still images included in a moving image block between cuts. The representative image is the first image, the last image, the temporally intermediate image, or the like of the moving image block. By determining a representative image for each moving image block, the cutout unit 51 cuts out a plurality of still images from the moving image data.

  A plurality of still images cut out from the moving image data are displayed on the display 44. The image selection unit 52 receives a still image selection from the user. The image data of the selected still image is stored in the memory 41.

  The data transmission unit 53 transmits the moving image data and the image data of the selected still image to the server device 1 using the communication unit 43. The storage location information receiving unit 54 receives storage location information for accessing moving image data from the server device 1 via the communication unit 43.

  The transfer unit 60 uses the communication unit 43 to print the image data of the still image transmitted from the data transmission unit 53 to the server device 1 and the storage location information received from the server device 1 by the storage location information reception unit 54. Transfer to device 2.

  The reproduction unit 70 includes a code reading unit 71, a communication state detection unit 72, an image acquisition unit 73, a feature point comparison unit 74, and a moving image reproduction unit 75.

  The code reading unit 71 reads storage location information from the two-dimensional code 32 printed on the printed material 3 using the camera 42.

  When the storage location information is read by the code reading unit 71, the communication state detection unit 72 detects whether or not communication with the server device 1 is possible. For example, when the mobile terminal 4 is set to a radio wave off mode in which radio waves are not transmitted / received or the server device 1 is down, it is detected that communication with the server device 1 is impossible.

  If communication with the server device 1 is possible, the image acquisition unit 73 uses the storage location information to access the server device 1 via the communication unit 43 and acquires image data corresponding to the image 31. The image acquisition unit 73 acquires the image 31 on the printed material 3 using the camera 42.

  The feature point comparison unit 74 compares feature points between the image acquired from the server device 1 and the image 31 of the printed matter 3 photographed by the camera 42.

  When it is impossible to communicate with the server apparatus 1 and image data cannot be acquired from the server apparatus 1, the feature point comparison unit 74 compares the image stored in the memory 41 and the image 31 of the printed matter 3 photographed by the camera 42. Compare feature points. The image stored in the memory 41 is an image transmitted by the data transmission unit 53 to the server device 1 and is substantially the same as an image acquired from the server device 1 when communication with the server device 1 is possible.

  When the feature point of the image acquired from the server apparatus 1 and the image 31 of the printed matter 3 photographed by the camera 42 matches with the server apparatus 1, the moving image reproduction unit 75 receives the moving image data from the server apparatus 1. Receive. The moving image reproducing unit 75 reproduces the received moving image data by superimposing the received moving image data on the image 31 of the printed matter 3 photographed by the camera 42 using a known AR technique. For example, the feature point of the detected image 31 is tracked to estimate the position and orientation of the camera 42, and the camera image and the moving image are aligned.

  When the feature points of the image stored in the memory 41 and the image 31 of the printed matter 3 photographed by the camera 42 coincide with each other, the moving image reproducing unit 75 uses a known AR technique. The moving image data stored in the memory 41 is superimposed on the image 31 of the printed matter 3 photographed by the camera 42 and reproduced. The moving image stored in the memory 41 is a moving image transmitted to the server device 1 by the data transmission unit 53 and is substantially the same as the moving image acquired from the server device 1 when communication with the server device 1 is possible.

  When the feature points do not match, it is displayed on the display 44 that the moving image cannot be reproduced.

  As described above, when communication with the server device 1 is impossible, the portable terminal 4 performs feature point comparison with the image 31 of the printed matter 3 using the image data stored in the memory 41 of the own device, The moving image data stored in the memory 41 is reproduced. Thereby, even if it is a case where communication with the portable terminal 4 and the server apparatus 1 is impossible, a moving image can be superimposed and reproduced on the printed matter 3.

  FIG. 5 is a configuration diagram of the server device 1. The server device 1 includes an image data receiving unit 11, a moving image data receiving unit 12, a storage location information transmitting unit 13, an image data transmitting unit 14, a moving image data transmitting unit 15, a moving image distribution management unit 16, and a database 17, for example, a computer. Consists of.

  The image data receiving unit 11 and the moving image data receiving unit 12 receive image data and moving image data from the portable terminal 4 and store them in the database 17 in association with each other. When receiving the moving image data from the mobile terminal 4, the storage location information transmission unit 13 transmits storage location information for accessing the moving image data to the mobile terminal 4.

  When the mobile terminal 4 accesses the server device 1 based on the storage location information, the image data transmission unit 14 transmits image data corresponding to the moving image data indicated by the storage location information to the mobile terminal 4.

  The video data transmission unit 15 transmits video data to the mobile terminal 4 in response to a request from the mobile terminal 4. The moving image data transmitted to the mobile terminal 4 corresponds to the storage location information used when the mobile terminal 4 has accessed.

  The database (storage unit) 17 stores image data, moving image data, and management information. The image data and the moving image data are those received from the mobile terminal 4. The management information includes information such as the correspondence between image data and moving image data, the date on which the data was registered, and the like. The server device 1 further includes a terminal information acquisition unit that acquires terminal information of the mobile terminal 4 that has registered the image data and moving image data (such as individual identification information of the terminal and an application ID of the installed moving image registration reproduction application), The acquired terminal information may be included in the management information. Thereby, the registration date of the image data and moving image data in the database 17 and the information of the portable terminal 4 used for registration can be managed.

  The database 17 stores at least one type of flash memory type, hard disk type, multimedia card micro type, card type memory (eg, SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory. Includes media.

  When the image data and moving image data received from the mobile terminal 4 are stored in the database 17, the moving image distribution management unit 16 generates storage location information and creates management information including the storage location information. The moving image distribution management unit 16 refers to the management information and identifies the corresponding image data and moving image data from the storage location information when the mobile terminal 4 has accessed.

  FIG. 6 is a block configuration diagram of the image printing apparatus 2. As illustrated in FIG. 6, the image printing apparatus 2 includes a CPU (central processing unit) 20, a touch panel (display unit, operation unit) 21, a charging unit 22, a printer 23, a communication unit 24, and a memory 25. The image printing apparatus 2 reads image data from a storage medium in which image data photographed by a digital camera or the like is stored, displays the read image on the touch panel 21, and selects an image and designates a print size and the number of prints from the user. A function of receiving and printing out an image by the printer 23 is provided.

  The charging unit 22 identifies and counts the inserted money and withdraws change. The printer 23 is, for example, a sublimation thermal transfer type high resolution color printer.

  The memory 25 stores a control program. The memory 25 is a memory of at least one of flash memory type, hard disk type, multimedia card micro type, card type memory (for example, SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory. Includes media.

  The CPU 20 functions as a control unit that controls each unit of the image printing apparatus 2 by executing the control program.

  When receiving a print order start instruction for a photograph with moving images from the user via the touch panel 21, the communication unit 24 receives image data and storage location information from the portable terminal 4 using a wireless communication or wired communication function. As shown in FIG. 7, the control unit displays the sample image 80 of the printed material 3 on the touch panel 21 using the received image data.

  When the user selects a purchase image and pays a fee, the control unit generates a composite image by combining the two-dimensional code image obtained by encoding the storage location information and the image data, and transfers the composite image to the printer 23. The printer 23 prints this composite image on photographic paper and outputs a printed product 3.

  FIG. 8 is a diagram showing a flow of processing of image / video registration and video playback according to the present embodiment.

  When the user operates the mobile terminal 4 to execute the moving image registration / playback application and starts the registration process, the moving image data in the memory 41 can be selected. When the user selects any moving image data, the cutout unit 51 cuts out a plurality of still images from the selected moving image data and displays them on the display 44. The user selects one of a plurality of still images (step S101).

  The data transmission unit 53 of the portable terminal 4 transmits the moving image data selected by the user in step S101 and the image data of the still image to the server device 1 (step S102).

  The server device 1 stores the moving image data and image data received from the mobile terminal 4 in the database 17 and notifies the mobile terminal 4 of storage location information for accessing the moving image data (step S103). In the mobile terminal 4, the storage location information is associated with the image data and moving image data of the still image transmitted to the server device 1 and stored in the memory 41.

  The user operates the image printing apparatus 2 and starts a print order for a photograph with a moving image. Using the communication function of the image printing apparatus 2, the same image data transmitted to the server apparatus 1 and the storage location information notified from the server apparatus 1 are transferred from the portable terminal 4 to the image printing apparatus 2 (steps). S104).

  When receiving the payment of the fee from the user, the image printing apparatus 2 encodes the storage location information to generate a two-dimensional code image, and synthesizes the two-dimensional code image and the image data transferred from the portable terminal 4. Then, a composite image is generated (step S105).

  The printer 23 prints this composite image on photographic paper and outputs it (step S106). Thereby, the user can obtain the printed matter 3 on which the image 31 and the two-dimensional code 32 are printed (step S107).

  When the user executes the moving image registration / reproduction application of the mobile terminal 4 and starts the reproduction process, the camera 42 is activated and a two-dimensional code reading instruction is displayed on the display 44. The portable terminal 4 reads the storage location information by photographing the two-dimensional code 32 of the printed matter 3 with the camera 42 (step S108).

  When the mobile terminal 4 can communicate with the server device 1, the image acquisition unit 73 of the mobile terminal 4 accesses the server device 1 via the communication unit 43 based on the storage location information and requests image data (step S109). ). The server device 1 refers to the management information and transmits image data corresponding to the moving image data indicated by the storage location information when accessed from the mobile terminal 4 to the mobile terminal 4 (step S110).

  When the mobile terminal 4 receives image data from the server device 1, an instruction to read the image 31 is displayed on the display 45. When the image 31 of the printed matter 3 is taken by the camera 42, the feature point comparison unit 74 compares the feature points of the image received from the server device 1 and the image 31 taken by the camera 42 (step S111). If the feature points match, the portable terminal 4 requests the moving image data from the server device 1 (step S112).

  The server apparatus 1 transmits the moving image data corresponding to the image data transmitted to the mobile terminal 4 in step S110 to the mobile terminal 4 (step S113). When the mobile terminal 4 receives the moving image data, the moving image reproducing unit 75 reproduces the moving image data by superimposing it on the image 31 captured by the camera 42 using a known AR technique (step S114). Thereby, on the display 45, a moving image is reproduced as if the image 31 in the printed matter 3 started to move.

  When the portable terminal 4 cannot communicate with the server device 1, the image data associated with the storage location information read in step S <b> 108 is extracted from the memory 41. In addition, an instruction to read the image 31 is displayed on the display 45. When the image 31 of the printed matter 3 is photographed by the camera 42, the feature point comparison unit 74 compares the feature points of the image extracted from the memory 41 and the image 31 photographed by the camera 42 (step S115).

  If the feature points match, the moving image playback unit 75 takes out the moving image data associated with the storage location information read in step S108 from the memory 41, and uses the known AR technique to capture the image 31 captured by the camera 42. The moving image data is reproduced by superimposing on (Step S116). Thereby, on the display 45, a moving image is reproduced as if the image 31 in the printed matter 3 started to move.

  As described above, according to the present embodiment, if the mobile terminal 4 can communicate with the server device 1 by photographing the printed product 3 output from the image printing apparatus 2 with the mobile terminal 4, the printed product 3 is displayed. The moving image data can be distributed from the server device 1 to the portable terminal 4 so that the moving image is superimposed and reproduced. On the other hand, when the mobile terminal 4 and the server device 1 cannot communicate with each other, the moving image can be superimposed on the printed matter 3 and reproduced using the moving image data in the memory 41 of the portable terminal 4.

  The moving image data can be registered in the server device 1 by the user himself / herself. Therefore, not only the image printed on the printed matter 3 but also a related moving image can be enjoyed regardless of the communication status of the mobile terminal 4.

  In the first embodiment, after registering image data and moving image data from the portable terminal 4 to the server device 1, the image data and / or moving image data is deleted from the memory 41 of the portable terminal 4 by a user operation or the like. There is a case. When image data or moving image data is erased from the memory 41, when the mobile terminal 4 and the server device 1 cannot communicate with each other, image feature points cannot be compared and moving images cannot be reproduced.

  For this reason, when the playback unit 70 of the mobile terminal 4 plays back a moving image in a state where it can communicate with the server device 1, the playback unit 70 checks the presence or absence of image data and moving image data in the memory 41, and there is no image data or moving image data (erased In this case, the server device 1 may be requested for image data or moving image data. The server device 1 transmits image data and moving image data in response to a request from the mobile terminal 4. The reproducing unit 70 stores the data received from the server device 1 in the memory 41. As described above, when the communication with the server device 1 is possible, the data necessary for the reproduction of the moving image when the communication with the server device 1 is impossible is obtained, and the moving image is superimposed on the printed matter 3 and reproduced at any time. Will be able to.

  This data acquisition processing from the server device 1 can also be applied to the case where moving image reproduction is performed on a mobile terminal different from the mobile terminal that has registered data in the server device 1. For example, consider a case where the user A registers image data and moving image data in the server apparatus 1 using the portable terminal 4A, creates a printed matter 3 with the image printing apparatus 2, and transfers the printed matter 3 to the user B.

  Image data and moving image data registered in the server device 1 are not stored in the memory of the mobile terminal 4B of the user B. Therefore, the portable terminal 4B reads the storage location information from the two-dimensional code 32 of the printed matter 3 and requests the server device 1 for image data or moving image data when accessing the server device 1 and reproducing the moving image. . In response to this request, the server device 1 transmits image data and moving image data to the portable terminal 4B so that they can be stored. The portable terminal 4B stores the received data in its own memory. For example, the mobile terminal 4B downloads and reproduces moving image data.

  As a result, image data and moving image data are also stored in the memory of the portable terminal 4B, so that even when the portable terminal 4B and the server device 1 cannot communicate with each other, image feature points can be compared and moving images can be reproduced. become.

  The storage location information notified from the server device 1 to the mobile terminal 4A may include not only URL information and content ID information but also URL information of the mobile terminal 4A. When the portable terminal 4B reads the storage location information from the two-dimensional code 32 of the printed matter 3, the portable terminal 4B accesses the portable terminal 4A based on the URL information of the portable terminal 4A included in the storage location information, and the content ID in the portable terminal 4A Can be searched for content (images and moving images), and feature point comparison and moving image reproduction can be performed.

  When the CPU 40 of the mobile terminal 4 executes the video registration / playback application, a time limit management unit is implemented to prevent the printed matter 3 from being purchased and not being registered in the server device 1 without being purchased. May be. For example, when moving image data and image data are registered in the server device 1 and the storage location information is received, the image data and the storage location information are not transferred to the image printing device 2 within a certain period (for example, one month). When the purchase of the product 3 has not been performed, the time limit management unit uses the display 44 of the portable terminal 4 to notify that there is an unpurchased print product 3 or erases the data registered in the server device 1. Or an alarm is displayed.

  Further, when the number of image data and storage location information that have not been transferred to the image printing apparatus 2 reaches a certain number, that is, when the number of unpurchased printed materials 3 reaches a certain number, the time limit management unit The display 44 of the mobile terminal 4 may be used to notify that the unpurchased printed matter 3 has accumulated, or to display an alarm that data registered in the server device 1 will be deleted. As a result, it is possible to prevent the printed matter 3 from being purchased and to prevent only the data from being registered in the server device 1 and to efficiently use the database 17 of the server device 1.

  After transferring the image data and the storage location information to the image printing apparatus 2, that is, when it is considered that the printed matter 3 has been purchased, the video registration / playback application displays the two-dimensional code 32 on the display 44 of the portable terminal 4. An instruction to read and try to play a moving image may be displayed. When the portable terminal 4 reads the two-dimensional code 32 and accesses the server device 1, the server device 1 can grasp that the moving image-added photo has been purchased for the registered moving image data and image data.

  The server device 1 determines a data storage period (for example, two years) when there is an access to playback of a moving image, and when the end of this storage period approaches, the end of the storage period is near to the mobile terminal 4 that has been accessed. You may notify that. The server device 1 may accept an extension of the storage period and purchase of moving image data from the mobile terminal 4 for a fee. The payment of the fee is performed by online payment using a credit card, for example. When the user selects the extension of the storage period, the server device 1 extends the storage period of the registration data. When the user selects to purchase moving image data, the server device 1 transmits the moving image data to the mobile terminal 4 and deletes the moving image data from the database 17. Accordingly, it is possible to prevent the moving image data that is no longer reproduced from being registered in the server device 1 and to efficiently use the database 17 of the server device 1.

  The server device 1 confirms the terminal information of the mobile terminal 4, the identification information (application ID) that identifies the video registration / playback application installed in the mobile terminal 4, the access source IP address, etc. The size of data registered (uploaded) from the mobile terminal 4 may be limited.

  In the first embodiment, the example in which the image data of the still image cut out from the moving image data is registered in the server device 1 and the two-dimensional code is combined with the still image and printed on the printed matter 3 has been described. However, if there is no favorite image among the still images cut out from the image data, other image data in the memory 41 is selected, registered in the server device 1, transferred to the image printing device 2, and transferred to the printed matter 3. You may make it print.

  When transmitting the storage location information to the mobile terminal 4, the server device 1 may transmit the two-dimensional code print position information together. The image printing apparatus 2 receives the print position information together with the storage location information from the portable terminal 4 and synthesizes a two-dimensional code image based on the print position information. This is to prevent the two-dimensional code 32 from interfering with the feature point comparison when performing the feature point comparison (steps S111 and S115 in FIG. 8).

  The portable terminal 4 preferably copies the image data transmitted to the server device 1, holds the copied image data so that it cannot be edited, and transfers the image data to the image printing device 2. In addition, when image data that cannot be edited is transferred from the portable terminal 4, the image printing apparatus 2 performs control so that image editing using the editing function (adding a stamp, a frame, etc.) of the own apparatus cannot be performed. When image data is edited after being transmitted to the server device 1 and before being printed by the image printing device 2, the feature points between the edited image on the printed matter 3 and the image stored in the server device 1 are different. This is because they can be inconsistent.

  In the first embodiment, the example in which the image printing apparatus 2 performs the coding process of the storage location information has been described. However, the image printing apparatus 2 may perform the moving image registration / playback application. In this case, the two-dimensional code image is transferred from the portable terminal 4 to the image printing apparatus 2.

  As shown in FIG. 7, when the sample image 80 of the printed matter 3 is displayed on the touch panel 21, it is preferable that the two-dimensional code in the sample image 80 is a two-dimensional code based on dummy information. Thereby, the act of photographing the sample image 80 displayed on the touch panel 21 with the portable terminal and reproducing the moving image can be prevented.

  In the first embodiment, when the mobile terminal 4 and the server device 1 cannot communicate, the video data in the memory 41 of the mobile terminal 4 is used to superimpose and play the video on the printed matter 3. When the mobile terminal 4 and the server device 1 can communicate with each other, even if the moving image data is stored in the memory 41, the received moving image data is reproduced by accessing the server device 1. According to this method, censoring of moving image data reproduced on the mobile terminal 4 can be performed at any time on the server device 1 side.

  When image data and moving image data are stored in the memory 41, even if the mobile terminal 4 and the server device 1 can communicate with each other, feature point comparison and moving image reproduction may be performed using the data. That is, regardless of whether the mobile terminal 4 and the server device 1 can communicate with each other, when the moving image data and the image data are stored in the memory 41, the image acquisition unit 73 is based on the read storage location information. The image data is taken out from the memory 41. The feature point comparison unit 74 compares feature points between the extracted image and the image 31 of the printed matter 3 taken by the camera 42. When the feature points match, the moving image reproducing unit 75 reproduces the moving image data stored in the memory 41.

  As described above, even if the mobile terminal 4 and the server device 1 can communicate with each other, the communication amount of the mobile terminal 4 can be reduced by using the image data and the moving image data stored in the memory 41. Moreover, even if the radio wave condition of the portable terminal 4 is not good, the moving image data can be stably reproduced.

[Second Embodiment]
FIG. 9 is a schematic configuration diagram of a moving image data reproduction system according to the second embodiment. As shown in FIG. 9, the moving image data reproduction system includes a server device 100 and an image print device 200. The server device 100 stores image data and moving image data corresponding to the image data. The server apparatus 100 transmits image data and storage location information (for example, URL) for accessing the corresponding moving image data to the image printing apparatus 200.

  The image data is, for example, image data of a person such as a sports player or a celebrity, an animal, or a car. The moving image data is data of a moving image in which an object in the image is moving or producing a sound. The case where the image data is human image data will be described below.

  When the fee is paid by the user, the image printing apparatus 200 encodes the storage location information received from the server apparatus 100 and generates a two-dimensional code. Then, the image printing apparatus 200 combines the two-dimensional code image and the person image based on the image data received from the server apparatus 100, prints the combined image on photographic paper, and outputs a printed matter 300. As a result, as shown in FIG. 10, a printed material 300 on which a person image 310 and a two-dimensional code 320 are printed is created.

  The user has a portable terminal 400 such as a smartphone or a tablet device, and a program (moving image reproduction application) for reproducing moving image data is stored in the portable terminal 400. When the portable terminal 400 executes this program, the camera function of the portable terminal 400 is activated to read the two-dimensional code. When the user captures the two-dimensional code 320 on the printed material 300 with the camera of the portable terminal 400, the portable terminal 400 reads the two-dimensional code 320 to acquire storage location information and accesses the server apparatus 100.

  The portable terminal 400 acquires image data from the server device 100 and instructs the user to take a human image 310 of the printed material 300 with the camera of the portable terminal 400. When the user captures a human image 310 with the camera of the mobile terminal 400, the mobile terminal 400 compares and compares the feature points of the captured human image 310 and the image acquired from the server device 100. When the two images match each other as a result of the collation, the mobile terminal 400 acquires moving image data corresponding to the person image 310 from the server device 100 and reproduces the moving image on the display. A known AR technique is used for moving image reproduction, and the moving image is superimposed and reproduced on the printed material 300 photographed by the camera of the portable terminal 400.

  The mobile terminal 400 requests the server device 100 for image data and moving image data in order to save the image data and moving image data in its own memory when the moving image is reproduced for the first time. In response to this request, the server device 100 transmits image data and moving image data so that the portable terminal 400 can store them. The portable terminal 400 stores the received image data and moving image data in its own memory.

  The portable terminal 400 also accesses the server device 100, compares the feature points of the images, and reproduces the moving image in the same manner as the first time even during the second and subsequent reproduction of the moving image. However, when the mobile terminal 400 is out of the communication range or the server device 100 is down and the mobile terminal 400 cannot access the server device 100, the image data and the video stored in the memory of the mobile terminal 4 Play video using data.

  That is, when the mobile terminal 400 cannot access the server apparatus 100, the feature points of the image stored in the memory of the own device and the image 310 of the printed matter 300 that is displayed by the camera of the mobile terminal 400 are compared. Collate. When the two images match each other as a result of the collation, the mobile terminal 400 reproduces the moving image on the display using the moving image data stored in its own memory. Even when the mobile device 400 cannot access the server device 100, the mobile device 400 superimposes and reproduces a moving image on the printed material 300 taken by the camera of the mobile device 400 using data stored in its own memory.

  As described above, in the moving image data reproduction system, the image printing apparatus 200 sells the printed matter 300 on which the person image 310 and the two-dimensional code 320 are printed. A user who has purchased the printed material 300 can read a two-dimensional code with the mobile terminal 400, access the server apparatus 100 to receive / reproduce moving image data, and view a moving image corresponding to the person image 310. In addition, when the mobile terminal 400 cannot access the server apparatus 100 after the initial playback of the moving image, the mobile terminal 400 can view the moving image corresponding to the image 310 using the data stored in the memory of the mobile device 400.

  FIG. 11 is a configuration diagram of the server apparatus 100. The server apparatus 100 includes an image data transmission unit 101, a storage location information transmission unit 110, a terminal information acquisition unit 120, a moving image data transmission unit 130, a moving image distribution management unit 140, and a database 150, and is configured by a computer, for example.

  The database (storage unit) 150 stores image data, moving image data, and management information. The image data and moving image data are provided from a licensor, for example.

  FIG. 12 shows an example of management information. The management information is divided into a plurality of boxes, and each box includes storage location information, the number of sellable devices, the number of reproducible devices, image information, and moving image information. The management information is divided into boxes by management units for image sales and video distribution.

  The number of sellable items is the number of printable products 300 that can be sold (upper limit number). The storage location information is information necessary for accessing the server apparatus 100 when reproducing the moving image, and a two-dimensional code 320 encoded with the information is printed on the printed material 300. For example, in image sales corresponding to the box 001, the same number of storage location information as the number X1 that can be sold is prepared. These X1 storage location information are different URLs, but the same box (box 001) is referred to and the same moving image data is reproduced by the conversion process inside the server apparatus 100. .

  In response to a request from the image printing apparatus 200, the storage location information transmission unit 110 transmits any of the storage location information that has not been transmitted to the image printing apparatus 200. As a result, the printed matter 300 sold by the image printing apparatus 200 has a different two-dimensional code 320 even if the person image 310 is the same.

  When the number of prints 300 sold reaches the number that can be sold, there is no storage location information that has not been sent, so the server device 100 notifies the image printing device 200 that the print device 300 has been sold out.

  The number of reproducible devices indicates the number of portable terminals 400 that can reproduce a moving image with one printed matter 300. Therefore, the terminal information acquisition unit 120 acquires the identification information of the mobile terminal 400 that requests the video data, for example, the individual identification information of the terminal, or the application ID of the video playback application installed in the mobile terminal 400. The moving image distribution management unit 140 stores the storage location information when the mobile terminal 400 accesses the server device 100 and the identification information of the mobile terminal 400 in association with each other, and uses the same storage location information. The number of accessed mobile terminals 400 is counted. After the count value reaches the number of reproducible devices, when there is an access from another portable terminal 400 using the same storage location information, the video distribution management unit 140 rejects the video playback.

  The number of sellable devices and the number of reproducible devices may be set by a licensor that provides image data and moving image data. The storage location information may be one type without setting the number of available sales. Further, it is not necessary to set the number of reproducible devices.

  The image information indicates information about image data to be sold. The moving image information indicates information about moving image data corresponding to image data.

  The image data transmission unit 101 transmits image data to the image printing apparatus 200 and the portable terminal 400. For example, when image data and moving image data are registered in the server apparatus 100, the image data transmission unit 101 transmits the image data to the image printing apparatus 200. Further, when the mobile terminal 400 accesses the server device 100 based on the storage location information, the image data transmission unit 101 refers to the image information of the box corresponding to the storage location information and transmits the image data to the mobile terminal 400. To do. For example, when the storage location information is included in the box 001, the image data transmission unit 101 transmits the image data P1 to the portable terminal 400, and when the storage location information is included in the box 002, the image data transmission is performed. The unit 101 transmits the image data P2 to the portable terminal 400.

  The moving image data transmission unit 130 transmits moving image data to the mobile terminal 400 in response to a request from the mobile terminal 400. The moving image data transmitted to the portable terminal 400 corresponds to the image data transmitted to the portable terminal 400.

  The image data transmitting unit 101 and the moving image data transmitting unit 130 can store the image data and moving image data in the portable terminal 400 when requested by the portable terminal 400 during the first moving image reproduction (first access) of the portable terminal 400. To be sent.

  FIG. 13 is a block configuration diagram of the image printing apparatus 200. As illustrated in FIG. 13, the image printing apparatus 200 includes a CPU (central processing unit) 201, a touch panel (display unit, operation unit) 210, a charging unit 220, a printer 230, a communication unit 240, and a memory 250. The image printing apparatus 200 is connected to the server apparatus 100 via the communication unit 240 so as to be communicable.

  The memory 250 stores the control program, the image sales program, and the image data received from the server device 100. The memory 250 is a memory of at least one of flash memory type, hard disk type, multimedia card micro type, card type memory (eg, SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory. Includes media.

  When the CPU 201 executes the control program, it functions as a control unit that controls each unit of the image printing apparatus 200.

  When the CPU 201 executes the image sales program, a sample image display unit 260, a storage location information acquisition unit 270, a coding processing unit 280, and an image composition unit 290 are realized as shown in FIG. It becomes possible to sell.

  Using the image data received from the server device 100, the sample image display unit 260 displays a sample image 700 of the printed material 300 on the touch panel 210 as shown in FIG. When the user selects a purchase image and pays a fee, the storage location information acquisition unit 270 acquires storage location information corresponding to the purchased image from the server device 100.

  The encoding processing unit 280 encodes the storage location information acquired by the storage location information acquisition unit 270 to generate a two-dimensional code image. The image synthesis unit 290 creates a synthesized image by synthesizing the two-dimensional code image with the purchased image. When the combined image is printed on the photographic paper by the printer 230, the printed matter 300 is output.

  The image printing apparatus 200 reads image data from a storage medium in which image data captured by a digital camera or the like is stored, displays the read image on the touch panel 210, and selects an image and designates a print size and the number of prints from the user. A function of receiving and printing out an image by the printer 230 may be provided.

  FIG. 16 is a block diagram of the mobile terminal 400. The portable terminal 400 includes a CPU 401, a memory 410, a camera 420, a communication unit 430, a GPS 440, and a display (touch panel) 450.

  The memory 410 stores a moving image playback application (application program). The memory 410 is a memory of at least one of flash memory type, hard disk type, multimedia card micro type, card type memory (for example, SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory. Includes media.

  When the CPU 401 executes the video playback application, a code reading unit 460, a communication state detection unit 462, an acquisition unit 470, a feature point comparison unit 480, and a video playback unit 490 are realized as shown in FIG. Playback can be performed.

  The code reading unit 460 reads storage location information from the two-dimensional code 320 printed on the printed material 300 using the camera 420. Using the read storage location information, the acquisition unit 470 accesses the server device 100 via the communication unit 430 and acquires image data corresponding to the person image 310. In addition, the acquisition unit 470 acquires the person image 310 using the camera 420.

  The feature point comparison unit 480 compares the feature points between the image acquired from the server device 100 and the person image 310 of the printed matter 300 captured by the camera 420. If the feature points match, the video playback unit 490 receives video data from the server device 100. When the feature points do not match, it is displayed on the display 450 that playback is impossible.

  The moving image reproduction unit 490 reproduces the received moving image data by superimposing the received moving image data on the person image 310 photographed by the camera 420 using a known AR technique. For example, the feature point of the detected person image 310 is tracked to estimate the position and orientation of the camera 420, and the camera image and the moving image are aligned.

  When the moving image reproduction unit 490 reproduces a moving image for the first time, the acquisition unit 470 requests the server device 100 to transmit image data and moving image data in order to save it in the memory 410. The image data and moving image data transmitted from the server apparatus 100 in response to this request are stored in the memory 410.

  When the storage location information is read by the code reading unit 460, the communication state detecting unit 462 detects whether communication with the server apparatus 100 is possible. For example, when the mobile terminal 400 is set to a radio wave off mode in which radio waves are not transmitted or received or the server apparatus 100 is down, it is detected that communication with the server apparatus 100 is impossible.

  When the mobile terminal 400 cannot communicate with the server device 100 during the second and subsequent video playback (when reading the two-dimensional code 320 of the printed material 300), the mobile terminal 400 displays the image data and video data stored in the memory 410. The feature points are compared to reproduce the moving image superimposed on the printed material 300. That is, the feature points of the image stored in the memory 410 and the image 310 of the printed matter 300 photographed by the camera 420 are compared. In addition, the moving image data stored in the memory 410 is reproduced by being superimposed on the image 310 of the printed matter 300 photographed by the camera 420. Thereby, even when communication between the portable terminal 400 and the server apparatus 100 is impossible, a moving image can be reproduced by being superimposed on the printed material 300.

  The GPS 440 acquires the position information of the mobile terminal 400 and notifies the server apparatus 100 of the position information in response to a request from the server apparatus 100.

  FIG. 18 is a diagram showing the flow of processing of image sales and video playback (first playback) according to this embodiment.

  When sales images and moving images are provided from the licensor and the number of sellable devices and the number of reproducible devices are set, management information including these pieces of information is registered in the database 150 (step S201). The server apparatus 100 generates the number of storage location information corresponding to the number that can be sold and stores it in the management information. As the management information is registered, the server apparatus 100 transmits image data to the image printing apparatus 200 (step S202). The transmission timing of the image data from the server apparatus 100 to the image printing apparatus 200 is arbitrary.

  The transmitted image data is stored in the memory 250. As a result, the image printing apparatus 200 starts selling images. The sample image display unit 260 displays the sample image 700 of the printed material 300 on the touch panel 210 as shown in FIG. 15 using the image data.

  When the user operates the image printing apparatus 200 to purchase an image and pays a fee (step S203), the storage location information acquisition unit 270 of the image printing device 200 stores storage location information of moving image data corresponding to the purchased image. Is requested to the server apparatus 100 (step S204).

  Based on the request from the image printing apparatus 200, the server apparatus 100 refers to the box corresponding to the management information in the database 150, and transmits unsent storage location information to the image printing apparatus 200 (step S205). For example, when the management information shown in FIG. 12 is stored in the database 150 and the image purchased in step S203 is the image P1, the server apparatus 100 transmits unsent storage location information included in the box 001. A transmitted flag is set in the transmitted storage location information in the management information.

  When the image printing apparatus 200 acquires the storage location information from the server device 100, the encoding processing unit 280 encodes the storage location information and generates a two-dimensional code image. Then, the image composition unit 290 composes the two-dimensional code image and the image purchased in step S203 to generate a composite image (step S206).

  The printer 230 prints this composite image on photographic paper and outputs it (step S207). Thereby, the user can obtain the printed material 300 on which the person image 310 and the two-dimensional code 320 are printed (step S208).

  When the user executes the video playback application of the mobile terminal 400, the camera 420 is activated, and a two-dimensional code reading instruction is displayed on the display 450 (step S209). The portable terminal 400 acquires storage location information by photographing the two-dimensional code 320 of the printed matter 300 with the camera 420 (step S210).

  Based on the storage location information, the image acquisition unit 460 of the portable terminal 400 accesses the server device 100 via the communication unit 430 and requests image data (step S211). The server device 100 refers to the management information box corresponding to the storage location information when accessed from the mobile terminal 400, and transmits the image data to the mobile terminal 400 (step S212). For example, when the storage location information is included in the box 001, the server device 100 transmits the image data P1 to the portable terminal 400.

  When the portable terminal 400 receives image data from the server device 100, an instruction to read the person image 310 is displayed on the display 450. When the person image 310 of the printed material 300 is photographed by the camera 420, the feature point comparison unit 480 compares the feature points of the image received from the server device 100 and the person image 310 photographed by the camera 420 (step S213). When the feature points match, the mobile terminal 400 requests the moving image data from the server device 100 (step S214).

  The server apparatus 100 transmits the moving image data corresponding to the image data transmitted to the mobile terminal 400 in step S212 to the mobile terminal 400 (step S215). For example, the server device 100 refers to the management information box 001 and transmits the moving image data M1 corresponding to the image data P1 to the portable terminal 400.

  When the mobile terminal 400 receives the moving image data, the moving image reproduction unit 490 reproduces the moving image data by superimposing it on the person image 310 photographed by the camera 420 using a known AR technique (step S216). As a result, the moving image is reproduced on the display 450 as if the person image 310 in the printed material 300 started to move.

  Furthermore, the portable terminal 400 stores the image data and moving image data received from the server device 100 in the memory 410 (step S217).

  FIG. 19 is a flowchart for explaining the second and subsequent moving image reproduction processes using the printed material 300.

  When the user executes the video playback application of the mobile terminal 400, the camera 420 is activated, and a two-dimensional code reading instruction is displayed on the display 450 (step S301). The portable terminal 400 acquires storage location information by photographing the two-dimensional code 320 of the printed matter 300 with the camera 420 (step S302).

  The communication state detection unit 462 detects whether or not communication with the server device 100 is possible (step S303). If communication with the server apparatus 100 is possible (step S303_Yes), the acquisition unit 470 accesses the server apparatus 100 via the communication unit 430 based on the storage location information and requests image data (step S304).

  When the image data is received from the server apparatus 100, an instruction to read the person image 310 is displayed on the display 450. When the person image 310 of the printed material 300 is photographed by the camera 420, the feature point comparison unit 480 compares the feature point between the image received from the server device 100 and the person image 310 photographed by the camera 420 (step S305).

  If the feature points match (step S306_Yes), the mobile terminal 400 receives the moving image data from the server device 100, and the moving image playback unit 490 superimposes it on the person image 310 captured by the camera 420 using a known AR technique. Then, the moving image data is reproduced (step S307). If the feature points do not match (step S306_No), it is displayed on the display 450 that the images do not match or that the moving image cannot be played back (step S308).

  If communication with the server apparatus 100 is not possible (step S303_No), the image data stored in step S217 in FIG. In addition, an instruction to read the person image 310 is displayed on the display 450. When the person image 310 of the printed material 300 is photographed by the camera 420, the feature point comparison unit 480 compares the feature points of the image extracted from the memory 410 and the person image 310 photographed by the camera 420 (step S309).

  If the feature points match (step S310_Yes), the portable terminal 400 retrieves the corresponding moving image data from the memory 410 and superimposes the moving image data on the image 310 captured by the camera 420 using a known AR technique. (Step S311). If the feature points do not match (step S310_No), it is displayed on the display 450 that the images do not match or that the moving image cannot be played back (step S308).

  As described above, according to the present embodiment, when the printed matter 300 output from the image printing apparatus 200 is photographed by the portable terminal 400, the moving image is superimposed on the printed matter 300 and reproduced to the portable terminal 400. Video data can be distributed. When the portable terminal 400 and the server device 100 cannot communicate, the moving image can be superimposed on the printed material 300 and reproduced using the moving image data in the memory 410 of the portable terminal 400.

  In addition, by setting the number of sellable devices and the number of reproducible devices using management information stored in the database 150 of the server apparatus 100, it is possible to limit the number of prints 300 issued and the number of moving image playback terminals. For example, the value of the printed material 300 can be increased by limiting the number of issued sheets. In addition, by limiting the number of moving image playback terminals, it is possible to suppress duplication / distribution of the printed material 300.

  Different videos may be played back depending on the position of the user (mobile terminal 400) and the video playback time. In this case, as shown in FIG. 20, a plurality of moving images corresponding to conditions such as position and time are defined in the management information. The position of the user (mobile terminal 400) is acquired by the GPS 440. For example, if condition 1 is set to time 5:00 to 10:00, condition 2 is set to time 10:00:00 to 18:00, and condition 3 is set to time 18:00:00 to 5:00, then morning, noon, and night You can change the video to be played.

  When “position” is set as the condition, a specific moving image is played at a specific location. For example, by playing a video at an event venue, a special video related to the event can be viewed.

  In the management information condition, not only the position and time but also various device information (manufacturer, setting language, model name, IP address zone, network segment) that can be acquired from the mobile terminal 400 can be set. Identification information (application ID) that identifies the video playback application may be set in the management information condition.

  In the management information, a distribution period of moving image data may be set. Further, the distribution period of the moving image data may be extended by additional charging. In this case, the distribution period of the moving image data is different for each storage location information.

  The moving image data distribution system may include a plurality of image printing apparatuses 200. The server apparatus 100 may set an image sales period for each image printing apparatus 200.

  When the sample image display unit 260 displays the sample image 700 on the touch panel 210, the two-dimensional code in the sample image 700 is preferably a two-dimensional code based on dummy information. As a result, it is possible to prevent the act of shooting the sample image 700 displayed on the touch panel 210 with the mobile terminal and playing back the moving image.

  When the server apparatus 100 transmits the storage location information in response to a request from the image printing apparatus 200, the print position information of the two-dimensional code may be transmitted together. The image composition unit 290 composes a two-dimensional code image with the image purchased by the user based on the print position information, and creates a composite image. Thus, by specifying the print position of the two-dimensional code, the two-dimensional code can be printed without deteriorating the quality of the person image. The code print position information may be determined by a licensor that provides image data. The code print position information may be transmitted when the server apparatus 100 transmits image data to the image printing apparatus 200 (step S202 in FIG. 18).

  In the above embodiment, an example in which image data is transmitted from the server apparatus 100 to the image printing apparatus 200 has been described. However, the memory 250 of the image printing apparatus 200 is connected by connecting a storage medium storing the image data to the image printing apparatus 200. Image data may be written in the.

  In this case, the image printing apparatus 200 may generate (issue) storage location information and code the storage location information to generate a two-dimensional code image. The image printing apparatus 200 includes a storage location information generation unit that generates storage location information by adding an additional digit number to a number that can be referred to by a corresponding box in the server device 100 (hereinafter referred to as an original number). By changing the additional digit number every time an image is purchased, the printed matter 300 sold by the image printing apparatus 200 has a different two-dimensional code 320 even if the person image 310 is the same. Since the storage location information includes a source number that allows the storage location of the moving image data to be specified even if the additional digit number is different, the same box is referred to by the internal processing of the server device 100. ing.

  The original number used by the image printing apparatus 200 for generating the storage location information can be written in the memory 250 by an operator operating the image printing apparatus 200, for example.

  When the moving image data distribution system includes a plurality of image printing apparatuses 200, the number ID of the image printing apparatus 200 may be included in the additional digit number. Thereby, each image printing apparatus 200 can generate different storage location information.

  As described above, the image printing apparatus 200 generates the storage location information, whereby the management load of the storage location information in the server device 100 can be reduced. In addition, by giving an upper limit to the additional digit number, the number of printed products 300 sold can be managed by the image printing apparatus 200. Further, the communication function with the server apparatus 100 can be omitted from the image printing apparatus 200.

  The server device 100 registers the storage location information when accessed from the mobile terminal 400 in the management information, and based on the number of reproducible devices, uses the registered storage location information to determine the number of mobile terminals that can play back moving images. Can be limited.

  In the second embodiment, when the mobile terminal 400 and the server device 100 can communicate with each other, even if the moving image data is stored in the memory 410, the received moving image data is reproduced by accessing the server device 100. However, when image data and moving image data are stored in the memory 410, feature point comparison and moving image reproduction (processing in steps S309 to S311 in FIG. 19) may be performed using the data. That is, when image data and moving image data are stored in the memory 410 regardless of whether communication between the mobile terminal 400 and the server device 100 is possible or not, the feature point comparison and moving image are not accessed without accessing the server device 100. Perform playback. Thereby, the communication amount of the portable terminal 400 can be reduced. In addition, the moving image data can be stably reproduced regardless of the communication state (radio wave condition) of the mobile terminal 400.

  When the number of playback times of moving image data is set by a licensor or the like, the moving image playback unit 490 of the mobile terminal 400 counts the number of playback times, and after the count value reaches a predetermined value, playback of the moving image data in the memory 410 is performed. You may make it restrict | limit.

  The moving image reproduced on the portable terminals 4 and 400 is not limited to the one superimposed on the printed matter 3 and 300, and may be reproduced on the displays 44 and 450 regardless of the position of the printed matter 3 and 300.

  At least a part of the moving image data reproduction system described in the above-described embodiment may be configured by hardware or software. When configured by software, a program that realizes at least a part of the functions of the moving image data distribution system may be stored in a recording medium such as a flexible disk or a CD-ROM, and read and executed by a computer. The recording medium is not limited to a removable medium such as a magnetic disk or an optical disk, but may be a fixed recording medium such as a hard disk device or a memory.

  Further, a program for realizing at least a part of the functions of the moving image data reproduction system may be distributed via a communication line (including wireless communication) such as the Internet. Further, the program may be distributed in a state where the program is encrypted, modulated or compressed, and stored in a recording medium via a wired line such as the Internet or a wireless line.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 Server apparatus 2 Image printing apparatus 3 Printed matter 4 Portable terminal

Claims (9)

A transmission unit for transmitting the moving image data and the image data stored in the storage unit to the server device;
A receiving unit for receiving data storage location information from the server device;
A transfer unit for transferring the image data and the storage location information to an image printing apparatus;
A reading unit that reads the storage location information from a code image printed on a printed matter created by the image printing apparatus;
After reading the storage location information, if communication with the server device is possible, an acquisition unit that accesses the server device based on the storage location information and acquires the image data;
If communication with the server device is possible, the feature points of the image acquired from the server device and the image on the printed matter acquired using a camera are compared. A comparison unit that compares feature points of an image stored in the unit and an image on the printed matter acquired using the camera;
When the feature points of the image acquired from the server device match the image on the printed matter acquired using the camera, the moving image data is received from the server device, and the received moving image data is reproduced. A reproduction unit that reproduces the moving image data stored in the storage unit when the feature points of the image stored in the storage unit and the image on the printed matter acquired using the camera coincide with each other When,
As a program that allows the mobile terminal computer to function.   When the reproducing unit reproduces the moving image received from the server device, if the moving image data or the image data is not stored in the storage unit, the moving image data or the image data is acquired from the server device. The program according to claim 1, wherein the computer is caused to function so as to be stored in the storage unit. A moving image data reproduction method using a portable terminal, an image printing device, and a server device,
The mobile terminal transmits moving image data and image data stored in a storage unit to the server device,
The server device stores the moving image data and image data received from the mobile terminal, transmits data storage location information to the mobile terminal,
The portable terminal transfers the image data and the storage location information to the image printing apparatus;
The image printing apparatus prints a composite image obtained by synthesizing the code image obtained by encoding the storage location information and the image data on a photographic paper, and outputs a printed matter.
The mobile terminal uses a camera to read the code image printed on the printed matter to obtain the storage location information,
When communication with the server device is possible, the server device is accessed based on the storage location information to acquire the image data, the image acquired from the server device, and the printed matter acquired using the camera Compare the feature points with the image above, and if the feature points match, receive the movie data from the server device, play the received movie data,
When communication with the server device is not possible, the feature points of the image data stored in the storage unit and the image on the printed matter acquired using the camera are compared. A moving image data reproducing method, wherein moving image data stored in the storage unit is reproduced. A server device that stores image data and moving image data corresponding to the image data, and transmits the moving image data to a mobile terminal,
To the portable terminal that has read and accessed the storage location information from the code image of a printed matter on which the image data and a code image obtained by encoding the storage location information of the moving image data in the server device are printed, Send video data,
The server device, wherein the image data and the moving image data are transmitted so as to be storable by the portable terminal when the moving image data is reproduced for the first time on the portable terminal. A reading unit that reads storage location information of moving image data in the server device from a code image printed on a printed matter;
An acquisition unit for accessing the server device based on the storage location information to acquire image data;
A comparison unit that compares feature points between an image acquired from the server device and an image on the printed matter acquired using a camera;
A reproduction unit that receives the moving image data from the server device and reproduces the received moving image data when the feature points match;
As a program for causing a mobile terminal computer to function,
When the moving image data is reproduced for the first time, the image data and the moving image data received from the server device are stored in the storage unit. When the comparison unit performs feature point comparison, receives moving image data from the server device, the playback unit plays back, and cannot communicate with the server device, the image data stored in the storage unit A program that causes the computer to function so as to perform feature point comparison by the comparison unit using and to reproduce the moving image data stored in the storage unit. A moving image data reproduction method using a portable terminal, an image printing device that stores image data, and a server device that stores the image data and moving image data,
When the image printing apparatus receives a purchase order for the image data, the image data storage location information of the moving image data in the server apparatus is encoded to generate a code image, and the image data and the code image are combined. Is printed on photographic paper and the printed matter is output.
The portable terminal reads the storage location information from the code image printed on the printed matter using a camera, accesses the server device based on the storage location information, acquires the image data, and When the feature points of the image acquired from the apparatus and the image on the printed matter acquired using the camera are compared, and the feature points match, the moving image data is received from the server device, and the received moving image Play the data,
The moving image data reproducing method, wherein the server device transmits the image data and the moving image data so as to be stored in the storage unit of the portable terminal when the moving image data is reproduced for the first time on the portable terminal.   When the portable terminal cannot communicate with the server device, the feature points of the image data stored in the storage unit and the image on the printed matter acquired using the camera are compared, and the feature points match. The moving image data reproduction method according to claim 6, wherein the moving image data stored in the storage unit is reproduced when the image data is stored. A transmission unit for transmitting the moving image data and the image data stored in the storage unit to the server device;
A receiving unit for receiving data storage location information from the server device;
A transfer unit for transferring the image data and the storage location information to an image printing apparatus;
A reading unit that reads the storage location information from a code image printed on a printed matter created by the image printing apparatus;
After the storage location information is read, if the moving image data and the image data are stored in the storage unit, the image data is extracted from the storage unit, and the moving image data or the image data is stored in the storage unit. If not, an acquisition unit that accesses the server device based on the storage location information and acquires the image data;
A comparison unit that compares feature points of an image acquired by the acquisition unit from the storage unit or an image acquired from the server device and an image on the printed matter acquired using a camera;
When the feature points of the image extracted from the storage unit and the image on the printed matter acquired using the camera match, the moving image data stored in the storage unit is reproduced, and the server device A reproduction unit that receives moving image data from the server device and reproduces the received moving image data when the acquired image and the feature point of the image on the printed matter acquired using the camera match;
As a program that allows the mobile terminal computer to function. A reading unit that reads storage location information of moving image data in the server device from a code image printed on a printed matter;
An acquisition unit that accesses the server device based on the storage location information to acquire image data when the storage location information is read for the first time;
A comparison unit that compares feature points between an image acquired from the server device and an image on the printed matter acquired using a camera;
A reproduction unit that receives the moving image data from the server device and reproduces the received moving image data when the feature points match;
As a program for causing a mobile terminal computer to function,
When the moving image data is reproduced for the first time, the image data received from the server device and the moving image data are stored in the storage unit,
When the storage location information is read from the second time onward, the comparison unit compares feature points between the image stored in the storage unit and the image on the printed matter acquired using a camera, and the reproduction is performed. The program causes the computer to function so as to reproduce the moving image data stored in the storage unit.

Images