JP5619214B2 - Image processing apparatus, image processing program, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing program, and an image processing method.

  There is a technology in which electronic devices such as home computers, digital televisions, and hard disk recorders are connected to each other via a local area network to share contents such as videos and images. As a specification for interconnecting and operating these electronic devices, for example, UPnP (Universal Plug and Play) is used. In UPnP UPnP AV Architecture of the UPnP, a control point is transferred from the media server to the media server. A specification is defined in which information of stored content is acquired, content to be reproduced is selected from the content, and the media renderer is instructed to acquire / reproduce the selected content.

  Here, as an example of a technique using UPnP, there is a technique for playing back a content obtained by combining contents stored in a plurality of storage media (for example, Patent Document 1). In Patent Literature 1, an access device creates virtual content obtained by editing a plurality of contents stored in a storage / playback device, transmits edit information of each content constituting the virtual content to a server device, and the server device Content transmission for requesting the storage / playback device to play back the content based on the editing information and sending it to the presentation device, and for the storage / playback device to play back the content based on the request from the server device A control system is disclosed.

JP2008-262643

  Here, when an image stored in the media server is displayed on the media renderer, it is preferable that a video suitable for the user can be generated based on the image and the generated video can be displayed. However, the technique of Patent Literature 1 reproduces content that is a combination of a plurality of contents stored in a server device, and does not consider generating a video from an image stored in the server. Therefore, there is a possibility that the video according to the user's preference cannot be displayed.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image processing program, and an image processing method capable of displaying a video suitable for a user based on an image stored in a server.

In order to solve the above problems, an image processing apparatus according to an embodiment includes an acquisition unit that acquires one or more still images from one or more external devices via a network, the acquired still image, and the still image Generating video data including a moving image including at least a part of the image element in the screen of the still image based on index information including information for specifying contents of one or more image elements which are constituent elements of Generating means, first transmitting means for transmitting the location of the video data generated by the generating means to the display device via the network, and specifying the location of the video data from the display device And second transmission means for transmitting the video data to the display device via the network in response to the data acquisition request.

The image processing program according to the embodiment is an acquisition unit that acquires one or more still images from one or more external devices via a network, the acquired still image, and a component 1 of the still image. Generating means for generating video data having a moving image including at least a part of the image element in the screen of the still image based on the index information including information for specifying the content of the image element , and the generation In response to a data acquisition request specifying the location of the video data from the display device, and a first transmission means for transmitting the location of the video data generated by the means to the display device via the network The computer is caused to function as second transmission means for transmitting the video data to the display device via the network.

Furthermore, the image processing method according to the embodiment includes a step in which an image processing apparatus acquires one or more still images from one or more external devices via a network, the acquired still image, and components of the still image Generating video data having a moving image including at least a part of the image element in the still image screen based on index information including information for specifying the content of one or more image elements In response to the step of transmitting the generated location of the video data to the display device via the network, and the data acquisition request specifying the location of the video data from the display device, The image processing device transmitting the video data to the display device via the network.

  An image suitable for the user can be displayed.

The figure which shows the network system which concerns on one Embodiment of this invention. The figure which shows the system configuration | structure of the computer which concerns on one Embodiment of this invention. The figure which shows the functional block of the computer which concerns on one Embodiment of this invention. The figure which shows the structural example of the index information database which concerns on one Embodiment of this invention. The figure which shows the example of an image selection screen concerning one Embodiment of this invention. The figure which shows the example of an image selection screen concerning one Embodiment of this invention. The figure which shows the example of a user preference designation | designated screen which concerns on one Embodiment of this invention. The figure which shows the example of an image | video which the video generation engine which concerns on one Embodiment of this invention produces | generates. The figure which shows the example of an image | video which the video generation engine which concerns on one Embodiment of this invention produces | generates. The figure which shows the example of an image | video which the video generation engine which concerns on one Embodiment of this invention produces | generates. The figure which shows the example of an image | video which the video generation engine which concerns on one Embodiment of this invention produces | generates. The figure which shows the example of an image | video which the video generation engine which concerns on one Embodiment of this invention produces | generates. The figure which shows the structural example of the image | video which the video generation engine which concerns on one Embodiment of this invention produces | generates. The figure which shows the example of a processing flow of the indexing process which concerns on one Embodiment of this invention. The figure which shows the example of a processing flow of the video production | generation process of the computer which concerns on one Embodiment of this invention. The figure which shows the process sequence example of the image | video production | generation process which concerns on one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating a configuration example of a network system including an image processing apparatus according to the present embodiment. Here, the image processing apparatus according to the present embodiment is realized as, for example, a notebook type computer 100. The computer 100 is connected to a network 200 configured by a wired or wireless LAN, and a storage device 300, a display device 400, and the like are connected to the network 200.

  The computer 100 includes an LCD 105, an HDD 110, a media control program 150 (not shown in FIG. 1), a communication unit 170, a video generation engine 180 (not shown in FIG. 1), and the like. Here, the LCD 105 is a display for displaying an image. The HDD 110 is a storage medium that stores data such as images. The media control program 150 is a program that functions as a DMC, which will be described later, and can control the storage device 300 and the display device 400. The communication unit 170 communicates with the storage device 300, the display device 400, the communication device 500, and the like via the network 200. The video generation engine 180 has a function of generating a video based on an image input to the video generation engine.

  The storage device 300 includes a storage unit 301, a communication unit 302, and the like. Image data stored in the storage unit 301 is transferred to the computer 100 and the display device 400 via the communication unit 302 under the control of the computer 100. It has a function to transmit. Each image data stored in the storage unit 301 is assigned a URI, and the storage device 300 responds to the data request when there is a data request specifying the URI from the computer 100 or the display device 400. The image data corresponding to the URI is transmitted.

  The display device 400 includes a communication unit 401, a decoding unit 402, an LCD 403, and the like, and receives image data and video data from the computer 100 and the storage device 300 via the communication unit 401 under the control of the computer 100. Then, the decoding unit 402 can decode (reproduce) the received image data and video data, and the LCD 403 can display the decoded image data and video data.

  Here, the computer 100, the storage device 300, and the display device 400 have a UPnP (Universal Plug And Play) function for recognizing the existence of each device on the network 200 and controlling each other's functions, and further a UPnP function Can be used as a home network device defined by, for example, the guidelines of DLNA (Digital Living Network Alliance). Home network devices are classified into categories such as a digital media server (DMS), a digital media controller (DMC), and a digital media renderer (DMR).

  The DMC is a device that controls devices such as DMS and DMR, and the DMS is a device that provides content data stored in the storage unit of the DMS to the DMR in response to a request from the DMC. The DMS uses the UPnP CDS (Content Directory Service) for the URI of the image data stored in the DMS, so that the image information can be obtained by the DMC and released to the network. To do.

  The DMC can acquire the image information released from the DMS by the CDS, for example, by a Browse action, and can acquire the URI of the image data. Also, the DMC sets the URI of the data to be played back in the media renderer by the SetAVTransportURI action of UPnP AVT (Audio / Visual Transport service).

  When the DMR sets the URI of the data to be played back by the DMC, the DMR acquires the data from the URI. Subsequently, when the DMR receives a Play action from the AVT, the DMR plays the acquired data.

  In the present embodiment, the computer 100 can function as any device of DMC, DMS, and DMR. The storage device 300 functions as a DMS, and the display device 400 functions as a DMR.

  By the way, in the conventional network system, when reproducing the image data stored in the DMS to the DMR, the DMC instructs the DMR to acquire and reproduce the image from the DMS, and the DMR acquires / reproduces the image data according to the instruction. To do.

  On the other hand, in the network system according to the present embodiment, when DMS image data is displayed on the DMR according to an instruction of the DMC, that is, when image data stored in the storage device 300 is reproduced on the display device 400 according to an instruction of the computer 100. The video engine 180 of the computer 100 generates video data based on the image data in the storage device 300, and the video data can be displayed on the display device 400, and a video suitable for the user can be provided. This will be described with reference to FIGS.

  In FIG. 1, the computer 100 and the storage device 300 are shown as the DMS, but other DMSs may be connected to the network 200, and devices other than the computer 100 and the display device 400 may be used for the DMR. It may be connected to the network 200.

Next, a system configuration of the computer 100 according to the present embodiment will be described with reference to FIG.
The computer 100 includes a CPU 101, a north bridge 102, a main memory 103, a GPU 104, a VRAM 104A, an LCD 105, a south bridge 106, a sound controller 107, a speaker 108, a BIOS-ROM 109, an HDD 110, a LAN controller 111, an ODD 112, a video processor 113, a memory 113A, A wireless LAN controller 114, an EC / KBC 115, a keyboard 116, a pointing device 117, and the like are provided.

  The CPU 101 is a processor that controls the operation of the computer 100, and executes various application programs such as an operating system (OS) 160 and a media control program 150 that are loaded from the HDD 110 to the main memory 103. The CPU 101 also executes a BIOS (Basic Input / Output System) stored in the BIOS-ROM 109. The BIOS is a program for controlling the hardware of the computer 100.

  The north bridge 102 is a bridge device that connects the local bus of the CPU 101 and the south bridge 106. The north bridge 102 also includes a memory controller that controls access to the main memory 103. The north bridge 102 also has a function of executing communication with the GPU 104.

  The graphics controller 104 is a display controller that controls the LCD 105 used as a display of the computer 100. The graphics controller 104 executes display processing for drawing display data in the VRAM 104A based on a drawing request input from the CPU 101 via the north bridge 102.

  The south bridge 106 controls each device connected to a peripheral component interconnect (PCI) bus and a low pin count (LPC) bus. The south bridge 106 includes a built-in IDE (Integrated Drive Electronics) controller for controlling the HDD 110 and the ODD 112 and a memory controller for controlling access to the BIOS-ROM 109. Further, the south bridge 106 has a function of executing communication with the sound controller 107 and the LAN controller 111.

  The sound controller 107 is a sound source device and outputs audio data to be reproduced to the speaker 108. The HDD 110 is a storage medium for storing programs such as the OS and the media control program 150, image data, and the like. The LAN controller 111 is a wired communication device that executes IEEE 802.3 standard wired communication, for example, and the wireless LAN controller 112 is a wireless communication device that executes IEEE 802.11 standard wireless communication, for example. The ODD 113 is a device that controls writing / reading of data with respect to a portable optical disk.

  The video processor 114 executes image processing such as indexing processing for image data acquired from the storage device 300 or the HDD 110 and video generation processing for generating video data based on the acquired image data. The memory 114A includes a video processor 114 working memory. Here, the video processor 114 detects the face image, the position and size of the face image from the image data, and the face image and the face image included in the image data obtained by the face detection process. Based on information such as position and size, it is possible to execute processing for generating an image having an effect focused on the face of a person.

  Although a large amount of calculation is required to perform image processing on image data, the video processor 114 performs image processing as a back-end processor of the CPU 101, thereby reducing the load on the CPU 101. Can be executed.

  The EC / KBC 115 is a controller that includes an embedded controller for managing power of the computer 100 and a keyboard controller for controlling the keyboard 116 and the pointing device 117. The EC / KBC 115 has a function of powering on / off the computer 100 in accordance with a user operation.

Next, a function example of the computer 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of functional blocks of the computer 100.
The computer 100 includes functional blocks such as a media control program 150, an OS 160, a communication unit 170, and a video generation engine 180.
The media control program 150 includes an instruction unit 151, an image selection unit 152, and the like, and has a function of transmitting an instruction to the storage device 300 and the display device 400 via the OS 160 and the communication unit 170. The OS 160 has a function of controlling the hardware of the computer 100 and includes a UPnP API 161, an HTTP server API 162, and the like.

  The communication unit 170 includes a LAN controller 111, a wireless LAN controller 114, and the like, and communicates with the storage device 300 and the display device 400 via the network 200. The video generation engine 180 has a function of indexing the image data input from the storage device 300 or the HDD 110, generating video data based on the input image data, and outputting the generated video data to the display device 400. Have.

  The instruction unit 151 instructs various operations to the storage device 300 and the display device 400 connected to the network 200 via the UPnP API 161. First, the instruction unit 151 recognizes the existence and type of a device connected to the network 200 using a protocol called UPnP SSDP (Simple service discovery protocol).

  Further, the instruction unit 151 obtains the details of the service operating in the device by acquiring an XML (Extensible Markup Language) file called device description and service description defined by UPnP from the device connected to the network 200. recognize. Here, the details of the service operating on the device include, for example, information on the format of images and videos that can be reproduced by the display device 400.

  In addition, the instruction unit 151 acquires information about image data that the storage device 300 discloses on the network 200 by UPnP CDS or the like by sending a UPnP CDS Browse action request via the UPnP API 161. Can do. That is, the instruction unit 151 transmits a Browse action request and receives a Browse action response as a response to the request via the UPnP API 161, whereby the image data included in the Browse action response and stored in the storage device 300 is received. Information about the image data including the title name and URI can be acquired.

  Then, after receiving the Browse action response, the instruction unit 151 designates the URI of each image data stored in the storage device 300 included in the Browse action response, and receives a data request for acquiring image data and the like. Send an HTTP GET request. Subsequently, the instruction unit 151 acquires image data stored in the storage device 300 by receiving an HTTP response that is a response to the HTTP GET request and includes image data of the designated URI. Note that transmission and reception of HTTP GET requests and HTTP responses are performed via the HTTP server API 162. Then, the instruction unit 151 instructs the selection screen generation unit 153 to generate an image selection screen based on the acquired image data.

  In addition, when the image generation instruction and the URI information of the selected image are input from the image selection unit 152, the instruction unit 151 transmits the data request specifying the URI of the selected image to the image generation engine 180 to the storage device 300. And instructing the video generation engine 180 to generate a video.

  Alternatively, when the instruction unit 151 inputs an instruction to reproduce the selected image and the URI information of the selected image from the image selection unit 152, the instruction unit 151 sends the data request to the storage device 300 that specifies the URI of the selected image to the display device 400. Instructs transmission and reproduction of the selected image, and does not instruct the video generation engine 180 to transmit a data request and generate a video.

  The instruction unit 151 acquires URI information of the video data generated by the video generation engine 180. The instruction unit 151 instructs the display device 400 to transmit a data request specifying the URI of the video data, and also instructs the display device 400 to reproduce the video data.

  The image selection unit 152 has a function of generating an image selection screen for selecting an image and outputting it from the GPU 104, and a function of selecting an image in response to an operation input on the image selection screen.

  Here, the image selection unit 152 includes a selection screen generation unit 153. When the selection screen generation unit 153 receives an instruction from the instruction unit 151 to generate an image selection screen based on the acquired image data, the image selection unit 152 includes an image including an image. Generate a selection screen. The image selection screen will be described later with reference to FIG.

  The image selection unit 152 receives an operation input from the pointing device 117 and the keyboard 116 for the image selection screen from the EC / KBC 115, and selects an image based on the operation input. Further, when receiving an operation input to the video playback button on the image selection screen from the EC / KBC 115, the image selection unit 152 outputs URI information and a video generation instruction for each selected image to the instruction unit 151.

  The image selection unit 152 can also select image data satisfying a predetermined selection condition from a plurality of image data stored in the storage device 300 or the HDD 110 by referring to the index information database 110A.

  In addition, when receiving an operation input for reproducing a still image of the selected image from the EC / KBC 115, the image selection unit 152 outputs a reproduction instruction for the selected image and URI information for each selected image to the instruction unit 151.

  The video generation engine 180 includes an indexing processing unit 181 and a video generation unit 182, and generates a video based on the image data and publishes the generated video to each device on the network 200 by UPnP CDS or the like. be able to.

  The indexing processing unit 181 executes an indexing process for creating the index information database 110A. In the indexing process, for example, a face detection process for detecting a face image from image data is executed. Here, when a plurality of face images are included in the image, each of the plurality of face images is detected.

  The detection of the face image can be performed, for example, by analyzing the feature of the image data and searching for a region having a feature similar to a face image feature sample prepared in advance. The position (coordinates) and size of each face image included in the image are detected by the face detection process. As described above, the indexing processing unit 181 functions as a face image detection unit that detects the positions and sizes of a plurality of face images included in an image.

  Further, the indexing processing unit 181 detects a character by searching a region having a feature similar to a character feature sample prepared in advance from image data. Here, the characters shown in the image are, for example, characters on a signboard.

  Further, the indexing processing unit 181 groups a plurality of image data stored in the storage device 300 or the HDD 110 based on, for example, the imaging date and time, and generates information for identifying each group. Here, for example, when the imaging date / time of the image data exceeds a predetermined period between two pieces of image data whose imaging date / time is continuous in time, the indexing processing unit 181 sets the interval between the two as image data. To do.

  Then, the indexing processing unit 181 outputs index information including information about the face image obtained by the indexing process, character information, group information, and the like to the HDD 110, and stores the index information in the index information database 110A.

  The video generation unit 182 generates video data based on the input image data. Here, the video generation unit 182 generates, from the input image data, a video in which an effect focused on a human face detected by, for example, an indexing process is added. In other words, the video generation unit 182 can generate a video composed of a plurality of frames including an image different from the image based on the input image data. The generated video data will be described later with reference to FIGS.

  In addition, when a plurality of pieces of image data are input, the video generation unit 182 can generate one video that displays the video based on each image data in the order based on the index information of each image data. That is, for example, when generating a video based on a group of images having the same shooting date and time, the video based on the landscape image data in which no person is captured is displayed before the video based on the other image data. Create a video that allows the user to recognize the image group, or display a video based on the image data with the largest number of people ahead of the video based on other image data, and who and what It is possible to generate a video that allows the user to recognize the image group at the time. Also, for example, by generating a video that displays an image based on an image including characters such as a signboard of a tourist spot in an order ahead of a video based on another image, the user can determine where the image group was captured. Video that can be recognized can be generated.

  Next, the index information database 110A generated by the indexing processing unit 181 will be described with reference to FIG. FIG. 4 is a diagram showing a configuration example of the index information database 110A. The index information database includes, for example, image IDA1, shooting date / time A2, face image information A3, face image A4, frontality A5, size A6, classification information A7, object information A8, character information A9, group information A10, URI information A11. Etc. are stored.

  Here, the image IDA1 is identification information assigned to each image data, and the imaging date / time A2 is time information indicating the imaging date / time of each still image. The face image information A3 is information that is generated when a person is captured in the still image data, and is generated for a plurality of persons when a plurality of persons are captured in the image data.

  In the index information database 110A, face image A4, front degree A5, size A6, classification information A7, and the like are stored as face image information A3. Here, the face image A4 is a face image extracted from the image data by the indexing processing unit 181. The front degree A5 is the front degree of the face image, and the size A6 is the size of the face image. The classification information A7 is information for classifying images estimated to be the same person, and is generated by classifying a plurality of face images extracted from the image data by the indexing processing unit 181.

  The object information A8 is information about the target object that is generated when the specific target object is reflected in the image data. Here, the specific object is, for example, a building, a famous monument, a plant, an animal, a mountain, a car, or the like, but is not limited thereto, and may be any object.

The character information A9 is information about the character that is generated when some character appears in the image data.
The group information A10 is information for grouping a plurality of image data stored in the storage device 300 or the HDD 110 based on, for example, the imaging date and time, and identifying the grouped group.

The URI information A11 is information indicating the URI of the image data when stored in the storage device 300 or the like.
That is, the computer 100 can determine from the index information database 110A whether each image data includes a face image, who's face image, whether a character is captured, and which group it belongs to. . Therefore, the computer 100 can detect image data or the like in which a desired person or character is captured from one or more image data stored in the storage device 300 or the HDD 110 by using the index information database 110A.

Next, the image selection screen generated by the selection screen generation unit 153 will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of an image selection screen.
Here, on the image selection screen, an image list area B1, an image area B2, an image selection box B3, a video reproduction button B4, a still image reproduction button B5, a server device selection column B6, a display device selection column B7, and the like are displayed.

  Here, the image list area B1 is an area for displaying the acquired image, and the image data acquired from the server device selected in the server selection field B6 is displayed as a list. Here, the server device is a server device connected to the network 200 such as the storage device 300, for example.

  The image area B2 is an area for displaying an image acquired from the storage device 300, and not only the image but also the title of the image may be displayed in the image area B2. The image selection box B3 is for receiving a user's selection operation for one or a plurality of images, and is provided for each image.

  The video playback button B4 is a button for playing back a video based on the image selected by the user. When the computer 100 receives an operation input for the button, the computer 100 transitions to a user preference designation screen described later with reference to FIG. After that, the video generation process described above is performed, and the display device 400 is instructed to transmit a data request for the video generated by the video generation process and to reproduce the video. The video playback button B4 is displayed when a plurality of image data is selected on the image selection screen, and may not be displayed when a single image data is selected.

  On the other hand, the still image reproduction button B5 is a button for reproducing the image selected by the user without performing the video generation process. When the computer 100 receives an operation input to the still image reproduction button B5, the display device The display device 400 displays the selected image in the order of the title name or the creation date and time of the selected image, for example, instructing 400 to transmit a data request specifying the URI of the selected image and to reproduce the selected image.

  The server device selection field B6 is a selection field for selecting which server device image data is to be displayed in the image list area B1, and for identifying the server device obtained by UPnP device description or service description. And a selection button for selecting the server device. In the server device selection field B6, the HDD 110 that is a storage device of the computer 100 may be displayed as an option.

  The display device selection field B7 is a selection field for reproducing / displaying a video or a still image. The display device selection field B7 is a selection field for identifying the display device obtained by UPnP device description or service description, and for selecting the display device. A selection button is arranged to do this. In the display device selection field B6, the LCD 105 that is the display device of the computer 100 may be displayed as an option.

  In the image list area B1, not only the image data of the server device selected in the server selection field B6 but also image data selected by the user may be displayed. That is, for example, even when the user selects the server A, selects an image of the server A, and then cancels the selection of the server A, the image data selected by the user is displayed in the image list area B1. May be. Further, the display order of the images in the image list area B1 may be, for example, a display order in which image data selected by the user is preferentially arranged.

  Next, an example of the image selection screen will be described with reference to FIG. Note that the image selection screen illustrated in FIG. 6 is an image that satisfies a predetermined selection condition from among a plurality of image data stored in the storage device 300 or the HDD 110, which is generated by the selection screen generation unit 153 based on the index information database 110A. It is a screen for selecting data.

  In the screen selection screen of FIG. 6, a face image list area C1, a face image area C2, a selection button C3, a video reproduction button C4, a still image reproduction button C5, a server device selection column C6, a display device selection column C7, and the like are displayed. The

  The face image list area C1 is an area where human face images are arranged as options one by one. The face image area C2 is an area for displaying a face image, and a selection button C3 for designating the face image is arranged in the vicinity of the area.

  Here, when the image selection unit 152 receives an operation input to the selection button C3 and a face image is designated, the image selection unit 152 selects all the image data in which the person of the designated face image is captured based on the index information database 110A. You can choose. Note that the image data to be selected here is all image data in which the person of the designated face image is captured among the image data stored in the server device selected in the server device selection field C6. Good.

  The video playback button C4, still image playback button C5, server device selection column C6 and display device selection column C7 are the video playback button B4, still image playback button B5, server device selection column B6 and display device selection column of FIG. Each has a function corresponding to B7.

Next, an example of the user preference designation screen will be described with reference to FIG.
The user preference designation screen displays options for designating / determining which target object to generate a video when the video generation engine 180 generates a video based on the image data. . Here, on the screen shown in FIG. 7, for example, a person, a landscape, an animal, a signboard, a monument, a car, and the like are displayed as options. For the person, an option for designating which person is to be focused is displayed.

  Note that, on the user preference designation screen, options for an object that is not shown in the image data selected on the image selection screen described above may be selected and added, or the options may be hidden. In addition, on the user preference designation screen, an effect used in the video generated by the video generation engine 180 may be specified.

  When the operation input reception on the user preference designation screen is completed and an operation to the OK button is received, the computer 100 performs the above-described video generation processing and the video generated by the video generation processing on the display device 400. Instructing the transmission of the data request and reproduction of the video. In the video generation process, a video based on a user operation input on the user preference designation screen may be generated, or a video may be generated without receiving a user operation input on the user preference designation screen. In this case, for example, a video or the like in which a person or a landscape is uniformly focused is generated.

  Next, a video generated by the video generation engine 180 according to the present embodiment will be described with reference to FIGS. FIG. 8 is an example of a video generated by the video engine 180.

  In the video shown in FIG. 8, first, as shown in FIG. 8A, a plurality of face images are displayed in a state of being distributed and arranged in a plurality of display areas. The plurality of face images are based on face images extracted from the original image data. Then, as time elapses, the image displayed in each display area gradually changes so as to approach the size of the original image data. Finally, as shown in FIG. 8C, and the state shown in FIG. 8C continues for a certain period of time. That is, it seems to the user to stand still for a certain time.

  In the video shown in FIG. 9, the entire still image is displayed blurred. Then, as shown in FIG. 9A, a frame covering a part of the still image is displayed on the still image, and a region surrounded by the frame is highlighted. At that time, the unhighlighted area (the area outside the frame) is displayed with the brightness lowered or blurred. As this frame moves in order on the face image included in the still image as shown in FIG. 9B, the face images of the still image are highlighted and displayed one by one. For example, when all of the still image face images are displayed, the entire image as shown in FIG. 9C is displayed stationary for a certain period of time.

  In the video shown in FIG. 10, the face image is enlarged and displayed for each person. Here, in the video, as in the image of FIG. 10A, the entire image is displayed stationary on the screen, and a plurality of face images included in the image are enlarged and displayed one by one. In the video, for example, face images as shown in FIG. 10B are enlarged and displayed in order from the person on the right side of the photograph. In FIG. 10B, the face of the second person from the right is enlarged.

  In the video shown in FIG. 11, the displayed image is switched to the next image. First, as shown in FIG. 11A, an image D1 is displayed stationary for a certain period of time, and then, as shown in FIG. 10B, a graphic D2 that is a graphics image covering a part of the image D1 is, for example, It moves toward the center of the image D1 while rotating. When the figure D2 reaches the center, the next image D3 different from the image D1 is displayed in an area that coincides with the area of the figure D2, as shown in FIG. Subsequently, as shown in FIG. 10D, the figure D2 is enlarged while rotating, for example, and the area of the image D3 displayed in the figure D2 is also enlarged. Then, when the figure D2 is enlarged, the entire image D3 is displayed as shown in FIG.

  In the video shown in FIG. 12, the image shown in FIG. 12A transitions to the next video shown in FIG. That is, the mountain image shown in FIG. 12 (A) changes to the building image shown in FIG. 12 (C) through the image shown in FIG. 12 (B) like a so-called morphing image. . Here, the video generation engine 180 complements and generates an image during the transition / deformation of the image based on the image data of FIGS.

  The video generation unit 182 detects the face image, the position and size of the face image, and the like as shown in FIGS. 8 to 10 focused on the detected face by the indexing processing unit 182. However, the video generation unit 182 is not limited to the face image, for example, depending on the indexing result of various objects by the indexing processing unit 182, the characters on the signboard, the object of the tourist destination, the mountain, the car, the animal, etc. An image focused on the object may be generated, or an image applied with an effect suitable for the object may be generated depending on the object to be focused.

  Next, a configuration example of video data generated by the video generation engine 180 will be described with reference to FIG. The video data V1 is composed of scenes based on the images 1 to N, and the scenes of the respective images are a still image scene that is a scene in which the images are displayed stationary, and an image based on the images moves. It consists of an effect scene that is a scene.

  In FIG. 13, in the scenes of images 1 and 2, an effect scene is displayed in the first half of the scene and a still image scene is displayed in the second half of the scene. That is, in these scenes, for example, images as shown in FIGS. 8 and 9 are displayed. In the scene of the image scene 3, a still image scene is displayed in the first half of the scene and an effect scene is displayed in the second half. That is, in the scene of the image 3, for example, an image based on the image 3 as shown in FIG. 10 is displayed.

  In the scene of the image N-1, the still image scene S4 is displayed in the first half of the scene, and the effect scene E4 is displayed in the second half. Here, the effect scene E4 is an effect for switching two different images as shown in FIGS. 11 and 12, for example, and a part of the next image N is displayed from the middle of the effect scene E4. In the scene of the image N, when the effect scene E4 ends, a still image scene S5 is displayed.

  In the video data V1, any effect may be applied to each image scene. Further, the order of the scenes of the respective images constituting the video data V1 may be based on the index information of the respective images. For example, a scene image of a landscape in which no person is shown or an image scene in which the largest number of people is shown may be displayed at the top of the video data V1.

  Next, with reference to FIG. 14, the order of processing relating to generation of the index information database will be described. FIG. 14 is a diagram illustrating a processing flow example of indexing processing by the indexing processing unit 181.

  First, the media control program 150 acquires image information from the storage device 300 by a Browse action of CDS (S701). Note that the instruction unit 151 acquires image information for a device connected via the network 200 periodically, for example, every day.

  When the instruction unit 151 obtains the URI of the image data included in the image information, the instruction unit 151 transmits a data request designating the URI (S702). When the instruction unit 151 requests image data from the URI, the instruction unit 151 may not request image data from the URI of the image data indexed by the indexing processing unit 181 before the request. That is, the instruction unit 151 transmits a data request specifying a URI that is not included in the index information database 110A among the acquired URIs by referring to / compare the acquired URI and the URI information of the index information database 110A. be able to.

  Upon receiving the image data corresponding to the data request, the video generation engine 180 performs an indexing process on the image data (S703), and writes information obtained by the indexing process in the index information database 110A (S704).

Next, with reference to FIG. 15, an example of a processing flow related to video generation by the video generation engine 180 will be described.
First, the instruction unit 151 transmits a service information request by UPnP device description or service description, and receives a response from the display device 400 to the request, thereby acquiring information about the display device 400 (S801). Note that the information about the display device 400 includes information on the image format that the display device 400 supports.

  Subsequently, the instruction unit 151 transmits a Browse action request of UPnP CDS, and receives a Browse action response from the storage device 300 in response to the Browse action request, whereby the file name of the image data stored in the storage device 300 is changed. URI information and the like are acquired (S802).

  Then, the instruction unit 151 transmits a data request designating the URI of the image data based on the URI information of the image data acquired in S802, and receives the image data from the storage device 300 (S803). Note that each image data stored in the storage device 300 is assigned a URI, and when acquiring a plurality of image data, the instruction unit 151 transmits a request specifying the URI assigned to each image data. To do.

  Next, the selection screen generation unit 153 generates an image selection screen based on the received image data, and outputs the generated image selection screen from the GPU 104 (S804). Here, the screen output from the GPU 104 may be either one of the screens shown in FIGS. 5 and 6.

  When the selection screen generation unit 153 generates the image selection screen shown in FIG. 6, if image data is stored in the index information database 110 </ b> A, the selection screen generation unit 153 does not acquire the image data from the storage device 300. An image selection screen may be generated using image data.

  Subsequently, the image selection unit 152 selects an image based on a selection operation input on the image selection screen (S805). Further, the image selection unit 152 selects a display device for reproducing / displaying an image or video based on an operation input on the image selection screen (S806).

  Next, when receiving an operation input for reproducing a still image on the image selection screen (No in S807), the image selection unit 152 outputs a reproduction instruction for the selected image and URI information of the selected image to the instruction unit 151. Then, the instruction unit 151 transmits the data request specifying the URI of the selected image selected by the image selection unit 152 to the display device 400, acquires the selected image, and acquires the acquired image. An instruction is given to reproduce the selected image (S808). Note that the computer 100 does not have to generate a video when receiving an operation input for reproducing a still image.

  On the other hand, when an operation input for video playback is received in S807 (Yes in S807), the selection screen generation unit 153 generates a user preference designation screen, causes the generated screen to be output from the GPU 104, and the computer 100 operates the screen. The input is accepted (S809). Next, the image selection unit 152 outputs the URI information of the image data that has received the video generation instruction and the selection operation to the instruction unit 151 to the video generation engine 180, and also displays the video based on the operation input on the user preference designation screen. The generation is instructed to the video generation engine 180. The instruction unit 151 can also instruct the video generation engine 180 to generate a video having a predetermined image format.

  Here, the predetermined image format is, for example, MPEG-2 which is a format generally displayable by DLNA DMR, or a video format which can be displayed by the display apparatus 400 acquired in S801. Then, the video generation engine 180 receives the image data from the storage device 300 by transmitting a data request designating the input URI (S810). Here, the video generation engine 180 may acquire image data stored in the HDD 110.

  Then, the video generation engine 180 starts generating video data based on the acquired image data (S811), and discloses to the network 200 that the video data is available. Note that the video data generated here may be video in an image format instructed from the instruction unit 151. Upon receiving a data request designating the URI of the video data from the display device 400, the video generation engine 180 outputs the video data to the display device 400 (S812). If the computer 100 receives a selection operation using the LCD 105 of the computer 100 as a display device in S806, the video generation engine 180 outputs the generated video to the GPU 104 in S812.

  Next, the processing sequence of the storage device 300, the media control program 150, the video generation engine 180, and the display device 400 of this embodiment will be described with reference to FIG. FIG. 16 is a diagram illustrating an example of a processing flow by the media control program 150, the video generation engine 180, the storage device 300, and the display device 400.

  First, the media control program 150 acquires information about the display device 400 in response to a service information request using UPnP device description or service description (S901). Next, the media control program 150 receives information about the image data from the storage device 300 by the Browse action (S902). Then, the media control program 150 transmits an image data request specifying the URI of the image data included in the information about the image data, and receives the image data from the storage device 300 (S903).

  Subsequently, the media control program 150 generates an image selection screen for selecting an image, and outputs the image selection screen from the GPU 104 to the LCD 105 (S904). Here, the media control program 150 selects a display device for displaying an image based on an operation input to a button in the display device selection field displayed on the image selection screen (S905). Then, the media control program 150 selects an image by receiving an operation input on the image selection screen (S906), and receives an operation input on the user preference designation screen.

  Then, the media control program 150 instructs the video generation engine 180 to transmit a data request including the URI of the image data selected in S905 and generate the video (S907). At this time, the media control program 150 instructs the video generation engine 180 to generate a video in a predetermined image format and to generate a video based on a user operation input on the user preference designation screen (S908).

  When receiving the data request instruction from the media control program 150, the video generation engine 180 acquires the URI image data included in the instruction from the storage device 300 (S909). The URI is allocated for each image data, and when acquiring a plurality of image data, the video generation engine 180 acquires the image data of a plurality of URIs.

  Based on the acquired image data, the video generation engine 180 starts generating a video having the configuration shown in FIG. 13 including the scenes shown in FIGS. 8 to 12 (S910). At this time, a URI is assigned to the video data generated by the video generation engine 180, and information about the video data including the URI is disclosed to each device on the network 200 by, for example, a UPnP CDS.

  Subsequently, the media control program 150 acquires the URI of the video data generated by the video generation engine 180, instructs the display device 400 to request data including the URI, and instructs the reproduction of the acquired video data. (S911).

  Upon receiving the data request transmission instruction including the URI of the video data from the media control program 150, the display device 400 acquires the video data from the URI included in the instruction (S912). Then, the media control program 150 instructs the display device 400 to display a video (S913), and the display device 400 decodes and displays the acquired video data (S914).

  In the processing shown in FIG. 16, the media control program 150 and the video generation engine 180 may acquire image data stored in the HDD 110, and the video generation engine 180 outputs the generated video to the LCD 105 of the computer 100. It doesn't matter.

  According to this embodiment, the computer 100 can generate video data based on the image data stored in the storage device 300 connected to the network 200 or the HDD 110. Then, the computer 100 outputs the generated video data to the display device 400 connected via the network 200 and can instruct the display device 400 to reproduce the video data. Thereby, the computer 100 can display a video suitable for the user based on the image data acquired from the device connected via the network on the device connected via the network.

  Further, the computer 100 performs indexing processing such as face detection on the acquired image data, generates video data based on information obtained by the indexing processing, stores index information for each image data, and stores the index data. For example, by selecting image data for each person based on the information, video data suitable for the user can be generated / output.

  Further, the computer 100 can generate / output video data in an image format that can be reproduced by the display device 400 by acquiring information about an image format that can be reproduced by the display device 400 from the display device 400. Alternatively, the computer 100 can output video data in an image format that can be reproduced by the display device 400 by generating video data in an image format generally used in DLNA.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  The invention described in the claims at the time of patent appraisal in the parent application of the original application of the present application will be added below.

  [1] An acquisition means for acquiring one or more images from one or more server devices via a network, and a second scene including a first scene including a still image and a moving image based on the still image from the acquired image. A first generation unit configured to generate video data including a scene, a reproduction instruction for the video data generated by the generation unit and a location of the video data to be transmitted to the display device via the network. An image processing apparatus comprising: transmission means; and second transmission means for transmitting the video data from the image processing apparatus main body to the display apparatus via the network in response to a request from the display apparatus.

  [2] Means for outputting a plurality of options respectively corresponding to a plurality of types of image elements which are constituent elements of an image, and means for designating an arbitrary image element by a selection operation from the plurality of output options The image processing apparatus according to [1], wherein the generation unit generates video data focused on the designated image element from the image acquired by the acquisition unit.

  [3] The image processing device according to [1], wherein the video data includes a first scene including a still image and a second scene including a moving image including at least a part of motion in the still image.

  [4] The image according to [1], wherein the video data includes a first scene including a still image and a second scene including a moving image including at least one moving part superimposed on the still image. Processing equipment.

  [5] The image processing apparatus according to [1], further including an indexing unit that generates index information about the acquired image in association with a scene order of the video data.

  [6] The image processing device according to [1], further including a selection unit that selects at least one image from the server device, wherein the acquisition unit acquires the at least one selected image.

  [7] The display device further includes display means for displaying information about the image to be selected from the server device and enabling the displayed information to be selected, and the selection means includes at least one based on the selected information. The image processing device according to [6], wherein an image is selected.

  [8] The first transmission unit transmits an instruction to reproduce at least one image selected by the selection unit to the display device via the network, and the generation unit transmits the at least one selected item. The image processing device according to [6], wherein the video data is not generated when the first transmission unit transmits an image reproduction instruction to the display device.

  [9] For selecting one of a process of generating the video data and a process of the first transmission unit transmitting an instruction to reproduce the selected at least one image to the display device The image processing apparatus according to [8], further including output means for outputting a screen including buttons.

  [10] The image processing device according to [1], wherein the generation unit generates the video data in an image format that can be displayed by the display device.

  [11] The acquisition unit according to [10], wherein the acquisition unit acquires information about the image format via the network, and the generation unit generates video data of an image format based on the acquired information. Image processing device.

  [12] The acquisition unit acquires one or more images published by the server device on the network, and the generation unit generates the video data from the acquired images, and the generated video data The image processing apparatus according to [1], further comprising: a disclosure unit that releases the data via the network; and an output unit that outputs the video data in response to a data request for the released video data.

  [13] An image processing method in an image processing apparatus provided with a first display device, wherein the image processing apparatus acquires one or more images from one or more server devices, and a static image is acquired from the acquired images. Generating image data having a first scene including an image and a second scene including a moving image based on the still image, and processing for outputting the image data to the first display device And an instruction to reproduce the video data generated by the image processing device and the location where the video data exists are transmitted from the image processing device to the second display device via the network, and then from the second display device. A process of transmitting the video data from the image processing apparatus to the second display apparatus via the network in response to the request of the image. An image processing method comprising the steps of management apparatus executes.

  Further, the invention described in the scope of claims at the time of the patent assessment in the original application of the present application will be added below.

  [1] An acquisition unit that acquires one or more still images from one or more server devices via a network, a generation unit that generates video data including a moving image based on the acquired still images, and the generation unit First transmission means for transmitting the location of the generated video data to a display device via the network, and in response to a request from the display device, the video data is displayed from the main body of the image processing device. An image processing apparatus comprising: a second transmission unit configured to transmit to the apparatus via the network.

  [2] The image processing device according to [1], wherein the video data includes a first scene including a still image and a second scene including a moving image based on the still image.

  [3] The image processing device according to [1], wherein the video data includes a first scene including a still image and a second scene including a moving image including at least a part of motion in the still image.

  [4] The image according to [1], wherein the video data includes a first scene including a still image and a second scene including a moving image including at least one moving part superimposed on the still image. Processing equipment.

  [5] The image processing apparatus according to [1], further including an indexing unit that generates index information about the acquired still image associated with a scene order of the video data.

  [6] Means for outputting a plurality of options respectively corresponding to a plurality of types of image elements which are constituent elements of an image, and means for designating an arbitrary image element by a selection operation from the plurality of output options The image processing apparatus according to [1], wherein the generation unit generates video data focused on the designated image element from the still image acquired by the acquisition unit.

  [7] The image processing device according to [1], further including selection means for selecting at least one still image from the server device, wherein the acquisition means acquires the selected at least one still image.

  [8] The information processing apparatus further includes display means for displaying information about the still image to be selected from the server device and enabling the displayed information to be selected. The selection means is based on at least one of the selected information. The image processing device according to [7], wherein one still image is selected.

  [9] The first transmission unit transmits an instruction to reproduce at least one still image selected by the selection unit to the display device via the network, and the generation unit transmits the at least one selected image. The image processing device according to [7], wherein the video data is not generated when the first transmission unit transmits an instruction to reproduce one still image to the display device.

  [10] To select one of a process for generating the video data and a process for the first transmission unit to transmit an instruction to reproduce the selected at least one still image to the display device. The image processing apparatus according to [9], further including output means for outputting a screen including the button.

  [11] The image processing device according to [1], wherein the generation unit generates the video data in an image format that can be displayed by the display device.

  [12] The acquisition unit according to [11], wherein the acquisition unit acquires information about the image format via the network, and the generation unit generates video data of an image format based on the acquired information. Image processing device.

  [13] The acquisition unit acquires one or more still images published by the server device on the network, and the generation unit generates the video data from the acquired still images, and the generated video The image processing apparatus according to [1], further comprising: a disclosure unit that releases data via the network; and an output unit that outputs the video data in response to a data request for the released video data.

  [14] An acquisition unit that acquires one or more still images from one or more server devices via a network, a generation unit that generates video data including a moving image based on the acquired still images, and the generation unit First transmission means for transmitting the location of the generated video data to a display device via the network, and in response to a request from the display device, the video data is displayed from the main body of the image processing device. An image processing program for causing a computer to function as second transmission means for transmitting to an apparatus via the network.

  [15] The image processing device acquires one or more still images from one or more server devices via the network, and the image processing device generates video data including a moving image based on the acquired still images. A step in which the image processing device transmits the location of the generated video data to the display device via the network; and the video data is sent to the image processing device in response to a request from the display device. Transmitting to the display device via the network.

  DESCRIPTION OF SYMBOLS 100 ... Computer, 101 ... CPU, 102 ... North bridge, 103 ... Main memory, 104 ... GPU, 105 ... LCD, 106 ... South bridge, 107 ... Sound controller, 108 ... Speaker, 109 ... BIOS-ROM, 110 ... HDD, 111 ... LAN controller, 112 ... ODD, 113 ... video processor, 114 ... wireless LAN controller, 115 ... EC / KBC, 116 ... keyboard, 117 ... pointing device, 150 ... media control program, 151 ... instruction unit, 152 ... image selection Reference numeral 153... Selection screen generation unit 161 161 UPnP API 162 HTTP server API 170 Communication control unit 180 Video generation engine 200 Network 300 Storage device 400 Display device

Claims (21)

Obtaining means for obtaining one or more still images from one or more external devices via a network;
Based on the acquired still image and index information including information for specifying the contents of one or more image elements that are constituent elements of the still image , at least one of the image elements in the screen of the still image. Generating means for generating video data having a moving image including a section;
First transmitting means for transmitting the location of the video data generated by the generating means to a display device via the network;
An image processing apparatus comprising: a second transmission unit configured to transmit the video data to the display device via the network in response to a data acquisition request specifying the location of the video data from the display device. And further comprising indexing means for generating the index information including information relating to the one or more image elements by detecting an image region having characteristics predetermined for each of the image elements from the acquired still image. Item 6. The image processing apparatus according to Item 1. The image processing apparatus according to claim 1, wherein the generation unit generates video data having a timing at which a plurality of partial images respectively extracted from the acquired still images are simultaneously displayed as the video data to be generated. Means for outputting one or more options corresponding to the one or more image elements to the display device ;
Means for designating an arbitrary image element by a selection operation from one or more options output to the display device ;
Further comprising
The image processing apparatus according to claim 1, wherein the generation unit generates video data focused on the designated image element based on the still image acquired by the acquisition unit and the index information. The image processing apparatus according to claim 1, wherein the video data includes an image area to be highlighted. The image processing apparatus according to claim 1, wherein the video data includes a moving image that is partially enlarged. The image processing apparatus according to claim 1, wherein the video data includes a morphed video. The image processing apparatus according to claim 1, wherein the video data includes a first scene including the still image and a second scene including a moving image based on the still image. The image processing apparatus according to claim 1, wherein the video data includes a first scene including the still image and a second scene including a moving image including at least a part of movement in the still image. The image processing apparatus according to claim 1, wherein the video data includes a first scene including the still image and a second scene including a moving image including at least one moving part superimposed on the still image. The image processing apparatus according to claim 1, wherein the index information is associated with a scene order of the video data. The acquisition means acquires a plurality of still images from a plurality of server devices,
An image selection screen generating unit for generating an image selection screen to be displayed on the display device for displaying a plurality of still images acquired from the plurality of server devices and receiving an image selection operation;
The generation means selects the first scene including the selected still image and the selected scene based on the still image selected from the plurality of acquired still images by the image selection operation and the index information. The image processing apparatus according to claim 1, further comprising: a second scene including a moving image generated from the still image. A selection means for selecting at least one still image from among the still images stored in the server device;
The image processing apparatus according to claim 1, wherein the acquisition unit acquires the selected at least one still image. Further comprising display means for displaying information about a still image to be selected from the server device and enabling the displayed information to be selected;
The image processing apparatus according to claim 13, wherein the selection unit selects at least one still image based on the selected information. The first transmission unit transmits an instruction to reproduce at least one still image selected by the selection unit to the display device via the network;
The image processing apparatus according to claim 13, wherein the generation unit does not generate the video data when the first transmission unit transmits an instruction to reproduce the selected at least one still image to the display device. A button for selecting one of a process for generating the video data and a process for the first transmission means to transmit an instruction to reproduce the selected at least one still image to the display device; The image processing apparatus according to claim 15, further comprising output means for outputting a screen including the image.   The image processing device according to claim 1, wherein the generation unit generates the video data in an image format that can be displayed by the display device. The acquisition means acquires information about the image format via the network,
The image processing apparatus according to claim 17, wherein the generation unit generates video data in an image format based on the acquired information. The acquisition means acquires at least one still image whose location on the network is disclosed by an external device on the network,
The generation means generates the video data from the acquired still image,
Publishing means for publishing at least the location of the generated video data on the network via the network;
The image processing apparatus according to claim 1, further comprising an output unit that outputs the video data in response to a data output request designating a location of the video data on the network . Obtaining means for obtaining one or more still images from one or more external devices via a network;
Based on the acquired still image and index information including information for specifying the contents of one or more image elements that are constituent elements of the still image, at least a part of the image element in the screen of the still image Generating means for generating video data having a moving image including:
First transmitting means for transmitting the location of the video data generated by the generating means to a display device via the network;
Second transmission means for transmitting the video data to the display device via the network in response to a data acquisition request specifying the location of the video data from the display device;
An image processing program that causes a computer to function. An image processing device acquiring one or more still images from one or more external devices via a network;
Based on the acquired still image and index information including information for specifying the contents of one or more image elements that are constituent elements of the still image , at least one of the image elements in the screen of the still image. Generating video data having a moving image including a portion;
The image processing device transmitting the generated location of the video data to the display device via the network; and
An image processing method comprising: the image processing device transmitting the video data to the display device via the network in response to a data acquisition request specifying the location of the video data from the display device.

Images