JP2012088984A - Portable terminal, image display device and image display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily capture image data from a portable terminal to an image display device while ensuring security.SOLUTION: A portable terminal includes imaging means 12, analysis means 10 which recognizes a face from an image captured by the imaging means to produce face analysis information, index generation means 10 which generates an index on the basis of the face analysis information, user information storage means 24 which stores, as user information, at least one of a user's face, the face analysis information and the index of the user which are produced by imaging the user's face through the imaging means, display means 36 which displays the index, determination means 10 which collates at least one of a face of a subject, the face analysis information and an index of the subject which are included in the captured image produced by imaging the subject through the imaging means with at least one of the user's face, the face analysis information and the index of the user to determine whether the user information is included in the captured image, and display control means 26 which allows the display means to display the index when it is determined that the user information is included in the captured image by the determination means.

Description

  The present invention relates to a portable terminal, an image display device, and an image display system including the portable terminal and the image display device.

  2. Description of the Related Art Conventionally, there are digital photo frames (hereinafter referred to as DPFs) that can display and appreciate images taken with a digital camera. When displaying an image captured by a digital camera on the DPF, the DPF needs to capture image data from the digital camera by a memory card, wired connection, or wireless communication (see, for example, Patent Document 1).

JP 2010-224715 A

  However, taking image data from a conventional digital camera to the DPF is time-consuming, and it is difficult for an unspecified person to import image data from the digital camera to the DPF, thus ensuring security. .

  An object of the present invention is to provide a portable terminal, an image display apparatus, and an image display system including the portable terminal and the image display apparatus that can easily capture image data from the portable terminal to the image display apparatus while ensuring security. Is to provide.

  The portable terminal of the present invention includes an imaging unit, an analysis unit that recognizes a face from an image captured by the imaging unit, and generates face analysis information, and an index generation unit that generates an index based on the face analysis information And user information storage means for storing at least one of the user's face, face analysis information, and the user's index generated by imaging the user's face by the imaging means as user information, and the index Display means for displaying, at least one of the subject's face, face analysis information, and the subject index included in the captured image generated by imaging the subject with the imaging means, and the user's face, face analysis information Determining means for determining whether or not the user information is included in the captured image by comparing with at least one of the index of the user, and the user information is included in the captured image by the determination means When it is determined that, characterized in that it comprises a display control means for permitting the display of the indicator on the display unit.

  The image display apparatus of the present invention also recognizes a face from an image captured by the imaging unit, a display unit that displays an image, an imaging unit, a receiving unit that receives image data from a mobile terminal, Analysis means for generating analysis information, an index analysis means for recognizing an index including face analysis information from an image captured by the imaging means, and reading face analysis information from the index; and imaged by the imaging means; The face analysis information generated by the analysis unit and the face analysis information read by the index analysis unit based on a captured image of a subject that includes a face and an index including the face analysis information within the same angle of view. It is determined that the determination means for determining whether or not they are the same, and the face analysis information generated by the analysis means by the determination means is the same as the face analysis information read by the index analysis means sometimes Characterized in that it comprises a reception control means for permitting the reception of the image data to the receiving means.

  The image display system of the present invention is an image display system including a mobile terminal and an image display device, and the mobile terminal recognizes a face from a first image capturing unit and an image captured by the image capturing unit. A first analysis unit that generates face analysis information; an index generation unit that generates an index based on the face analysis information; and the user's face and face generated by imaging the user's face with the imaging unit First user information storage means for storing at least one of the analysis information and the user index as user information, a first display means for displaying the index, and an image of a subject generated by the imaging means Matching at least one of the face of the subject, face analysis information, and the index of the subject included in the captured image with at least one of the user's face, face analysis information, and the index of the user, to the captured image First display means for determining whether or not user information is included, and permitting the display means to display the indicator when the determination means determines that the user information is included in the captured image. Display control means, wherein the image display device is a second display means for displaying an image, a second imaging means, a receiving means for receiving image data from the portable terminal, and an image captured by the imaging means. Second analysis means for recognizing a face from the image and generating face analysis information; and index analysis means for recognizing an index including face analysis information from an image captured by the imaging means and reading the face analysis information from the index And the face analysis information generated by the analysis unit and the index analysis unit based on a captured image of a subject captured by the imaging unit and containing a face and an index including the face analysis information within the same angle of view. Read in Second determination means for determining whether or not the face analysis information is the same, the face analysis information generated by the analysis means by the determination means and the face analysis information read by the index analysis means And receiving control means for permitting reception of image data from the portable terminal to the receiving means when it is determined that they are the same.

  According to the present invention, it is possible to easily capture image data from a portable terminal to an image display device while ensuring security.

It is a block diagram which shows the structure of the image display system which concerns on embodiment. It is a flowchart which shows the transmission process of the image data with respect to an image display apparatus from the portable terminal in the image display system which concerns on embodiment. It is a figure which shows the state of transmission of the image data with respect to an image display apparatus from the portable terminal in the image display system which concerns on embodiment. It is a figure which shows the state of transmission of the image data with respect to an image display apparatus from the portable terminal in the image display system which concerns on embodiment.

  Hereinafter, an image display system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a system configuration of an image display system according to an embodiment. As shown in FIG. 1, the image display system 1 includes a digital camera 2 that functions as a portable terminal and a DPF 4 that functions as an image display device.

  The digital camera 2 includes a CPU 10 that is configured by a microprocessor or the like and controls each part of the digital camera 2 in an integrated manner. The CPU 10 is connected with an imaging unit 12, a power supply controller 14, a card controller 16, a gyro sensor 18, a WiFi driver 20, an I / F driver 22, a memory 24, and an LCD driver 26.

  The imaging unit 12 has an imaging element composed of a CCD, a CMOS, etc., images light from a subject via a photographing lens (not shown), and outputs an imaging signal to the CPU 10. The power controller 14 controls power supply from the battery 28 to the CPU 10. The card controller 16 controls data writing and reading with respect to an SD card, CF card, etc. 31 mounted in the card slot 30. The gyro sensor 18 detects the attitude of the digital camera 2 and outputs a detection result to the CPU 10.

  The WiFi driver 20 controls the WiFi transmission / reception unit 32 and performs data communication with the DPF 4. The I / F driver 22 detects the operation state of the operation switch 34 including a power switch, a release button, etc., and outputs it to the CPU 10. The memory 24 stores image data and the like based on the imaging signal output from the imaging unit 12. The LCD driver 26 performs display control of a display 36 having an LCD panel arranged on the back surface of the digital camera 2.

  The DPF 4 includes a CPU 40 that is configured by a microprocessor or the like and controls each part of the DPF 4 in an integrated manner. The CPU 40 is connected to an imaging unit 42, a USB controller 44, a card controller 46, a power controller 48, an I / F driver 50, a memory 52, an LCD driver 54, and a WiFi driver 56.

  The imaging unit 42 includes an imaging element configured by a CCD, a CMOS, and the like, images light from a subject via a photographing lens (not shown), and outputs an imaging signal to the CPU 40. The USB controller 44 controls data communication with a PC (personal computer) 60 or the like connected via the USB connector 58. The card controller 46 controls writing and reading of data with respect to the SD card, CF card, etc. 64 mounted in the card slot 62. The power controller 48 controls power supply from the battery 66 and the AC power source 68 to the CPU 40.

  The I / F driver 50 detects the operation state of the operation switch 70 including a power switch and outputs it to the CPU 40. The memory 52 stores image data and the like based on the imaging signal output from the imaging unit 42. The LCD driver 54 performs display control of the display 72. The WiFi driver 56 controls the WiFi transmission / reception unit 74 to perform data communication with the digital camera 2.

  Next, image data transmission processing from the digital camera 2 to the DPF 4 in the image display system 1 according to the embodiment of the present invention will be described with reference to the flowchart shown in FIG. First, the user of the digital camera 2 captures an image with the digital camera 2 facing his / her face in order to register the digital camera 2 in the digital camera 2 that the digital camera 2 is his property. The CPU 10 analyzes the face of the user present in the image data based on the image data output from the imaging unit 12 by the face recognition analysis, and the face analysis information which is information of the analyzed user face and analysis result Is stored in the memory 24. In addition, based on the image data output from the imaging unit 12, an index (marker) including a predetermined image pattern including the user's face recognition information is created, and the created index data is stored in the memory 24. . Here, the user's face, face analysis information, and index data are stored in the memory 24 as user information.

  A user who uses the digital camera 2 operates the operation switch 34 to shift the digital camera 2 to the photo frame interlocking mode. When the operation switch 34 is operated, the CPU 10 shifts the digital camera 2 to the photo frame interlocking mode (step S10). Then, the user takes an image with the digital camera 2 facing the face of the user. The CPU 10 analyzes a face existing in the image data by face recognition analysis based on the image data output from the imaging unit 12, acquires user information (user face, face analysis information, and index data), and acquires The obtained user information is collated with the user information stored in the memory 24. As a result of the collation, when at least one piece of user information stored in the memory 24 is included in the image data from the imaging unit 12, a person existing in the image data is registered as a user. It is determined that the person is a person. When it is determined that the user is a registered user, the CPU 10 transmits a signal for establishing wireless communication to the DPF 4 located in the vicinity of the digital camera 2 by the WiFi transmitting / receiving unit 32, and the digital The WiFi transmitter / receiver 32 receives a signal returned from the DPF 4 existing in the vicinity of the camera 2 to authenticate the DPF 4 for mutual communication and establish wireless communication (step S11). When the mutual communication can be established with the DPF 4 existing in the vicinity of the digital camera 2 (Yes in step S12), the CPU 10 transmits to the DPF 4 interlocking information for interlocking the DPF 4 with the digital camera 2 ( Step S13). Further, a marker based on the index data stored in the memory 24 is displayed on the display 36 (step S14).

  When the CPU 40 of the DPF 4 receives the interlocking information from the digital camera 2 via the WiFi transmitting / receiving unit 74 (step S30), the CPU 40 shifts the DPF 4 to the interlocking mode (step S31). Then, the CPU 40 starts imaging the subject at a predetermined frame rate by the imaging unit 42, and displays a through image based on the imaging signal output from the imaging unit 42 on the display 72 (step S32). Here, the image data of the through image captured by the imaging unit 42 is temporarily stored in the memory 52.

  For example, as shown in FIG. 3, the face of the person holding the digital camera 2 and the display 36 arranged on the back of the digital camera 2 are within the imaging field angle of the imaging unit 42 of the DPF 4. The display 72 displays a through image including the face of a person holding the digital camera 2 and the display 36 of the digital camera 2. In this case, the CPU 40 detects the marker 80 from the image data temporarily stored in the memory 52 (step S33, Yes), and reads the human face information (face recognition information) from the detected marker 80 ( Step S34). Further, the face of the person existing within the angle of view of the through image is analyzed by face recognition analysis (step S35).

  Then, the face recognition information (face information registered as a user) based on the detected marker is collated with the analysis result by the face recognition analysis (step S36), and the analyzed human face is registered as a user. If it is determined that the person is a person (step S37, Yes), a signal requesting transmission of a set of image data stored in the memory 24 is transmitted to the digital camera 2 via the WiFi transmitting / receiving unit 74 ( Step S38). When the CPU 10 of the digital camera 2 receives a signal requesting transmission of a set of image data from the DPF 4 via the WiFi transmission / reception unit 32 (step S15), the CPU 10 transmits the set of image data stored in the memory 24 to the DPF 4 (step S15). S16).

  The CPU 40 of the DPF 4 receives a set of image data from the digital camera 2 via the WiFi transmission / reception unit 74 (step S39) and stores it in the memory 52. Further, based on the position and shape of the marker detected in step S33, the image data stored first in the image data stored in the memory 52 is affine transformed, and the shape of the entire image is set to a predetermined value. The shape is deformed (step S40). That is, the image data is affine transformed in order to synthesize and superimpose the image on the display 36 of the digital camera 2 in the image displayed on the display 72 shown in FIG. Then, an image 82 based on the image data subjected to affine transformation on the marker 80 displayed on the display 36 of the digital camera 2 in the image displayed on the display 72 is synthesized and superimposed (step S41). In step S39, the CPU 40 of the DPF 4 acquires the face of the person holding the digital camera 2, face analysis information, and index data based on the image data from the imaging unit 42, and the acquired face, The face analysis information and index data may be collated with user information of the user of the digital camera 2 stored in advance in the memory 52 (user face, face analysis information and index data). The user information of the user of the digital camera 2 is acquired from the digital camera 2 by wireless communication or the like and stored in the memory 52 in advance. In this case, the CPU 40 of the DPF 4 receives a set of image data from the digital camera 2 when the image data from the imaging unit 42 includes at least one piece of user information stored in advance in the memory 52. Allow that.

  When the operation switch 34 is operated by the operator or the gyro sensor 18 detects that the attitude of the digital camera 2 is inclined by a predetermined angle (step S17, Yes), the CPU 10 of the digital camera 2 operates Based on the operation of the switch 34 or the attitude of the digital camera 2, the marker 80 displayed on the display 36 is switched to a predetermined display for controlling the display of the image 82 synthesized and superimposed on the display 72 (step S18). That is, when the operation switch 34 for instructing the enlargement of the image or the change in the attitude of the digital camera 2 is detected by the gyro sensor 18, the CPU 10 displays the combined and superimposed display on the display 72. A control index for instructing enlargement of the image 82 is created and displayed on the display 36.

  When the CPU 40 of the DPF 4 detects a control index for instructing enlargement of the image 82 from the image data temporarily stored in the memory 52 (step S42, Yes), the image 82 that is synthesized and superimposed on the display 72 is displayed. Is performed (step S43). Here, the CPU 10 of the digital camera 2 reduces the image 82 based on the operation of the operation switch 34 or the attitude of the digital camera 2 detected by the gyro sensor 18 in addition to the control index for instructing the enlargement of the image 82. It is possible to create a control index for instructing rotation, sending to the next image, returning to the previous image, and the like.

  For example, when the operation switch 34 is operated or the attitude of the digital camera 2 is changed by the gyro sensor 18, a control index for instructing to send the image 82 synthesized and superimposed on the display 72 of the DPF 4 to the next image is created. When displayed on the display 36, the CPU 40 of the DPF 4 detects a control index for instructing to send the image 82 to the next image from the image data temporarily stored in the memory 52. A process of sending the combined / superimposed image 82 to the next image is performed. That is, the image data stored second in the image data stored in the memory 52 is affine transformed, and an image based on the affine transformed image data is displayed on the display 72 instead of the image 82.

  According to the image display system according to the embodiment of the present invention, the DPF needs to operate the operation switch when it is determined that the person who is within the imaging angle of view by the imaging unit of the DPF is the user of the digital camera. It is possible to receive the image data from the digital camera without doing so. Therefore, it is possible to easily capture image data from the digital camera to the DPF while ensuring security. In addition, when the through image is displayed on the DPF display, an image based on the image data received from the digital camera is synthesized and superimposed on the marker displayed on the DPF display. It is possible to obtain a virtual reality feeling as if an image is being manipulated, and to enjoy image data transfer work that tends to be cumbersome and difficult to avoid.

  In the above-described embodiment, the DPF 4 determines whether or not the person is registered as a user of the digital camera 2 based on the through image, but the user of the digital camera 2 based on the still image. It may be determined whether or not the person is registered as. In this case, an image based on the image data received from the digital camera 2 is synthesized and superimposed on the display 36 of the digital camera 2 in the still image displayed on the display 72. Thereby, it is not necessary for the user to always hold the digital camera 2 facing the DPF 4, and it can be said that the user is highly convenient in terms of the operation posture. Furthermore, the processing load on the CPU 40 of the DPF 4 is reduced, and it can also be realized by a DPF that employs a cheaper CPU having a low processing capacity.

  In the above-described embodiment, the position where the image based on the image data received from the digital camera 2 is synthesized and superimposed is set as the position of the display 72 of the digital camera 2 displayed on the display 72 of the DPF 4. The position of the image to be synthesized / superimposed may be set to an arbitrary position on the through image displayed on the display 72, the position of the object, or the like. For example, when the position of a human face existing within the angle of view of the through image is specified by face recognition analysis, the DPF 4 combines and superimposes the image in a balloon automatically arranged around the specified face. Also good. In this case, the DPF 4 performs analysis by face recognition analysis without detecting the marker 80 displayed on the display 72 of the digital camera 2, and receives image data from the digital camera 2 through wireless communication with the digital camera 2. An image based on the received image data can be synthesized and superimposed.

  In the above-described embodiment, the DPF 4 receives a set of image data from the digital camera 2 and combines and superimposes the image 82 based on the received image data on the marker 80 displayed on the display 72 of the DPF 4. Although displayed, an image based on image data stored in advance in the memory 52 of the DPF 4 may be synthesized and superimposed.

  In the above-described embodiment, the digital camera 2 and the DPF 4 directly transmit and receive image data by wireless communication. However, when the digital camera 2 stores image data in a server on the network, the DPF 4 The image data on the server may be acquired. In this case, the CPU 10 of the digital camera 2 creates an index including the user's face recognition information and the URL information of the server on the network where the image data is stored, and displays a marker based on the created index 36. To display. When detecting the marker displayed on the display 36, the CPU 40 of the DPF 4 reads URL information from the detected marker and connects to a server on the network based on the read URL information to acquire image data on the server.

  In the above-described embodiment, it is desirable to use a geometric pattern of shape, color, and contrast that is not confused with the texture of other devices or natural objects for the marker 80 displayed by the digital camera 2.

  In the above-described embodiment, the digital camera 2 that controls the transmission / reception of the image data to and from the DPF 4 and the processing for changing the display of the image combined and superimposed on the display 72 of the DPF 4 is used as a dedicated terminal. The digital camera 2 is a general-purpose digital camera or portable terminal equipped with functions, display for display, and a sufficient computing capability, and an application that implements communication with the DPF 4, transmission / reception of image data, selection operation of image data, and the like. It may be used instead.

  DESCRIPTION OF SYMBOLS 1 ... Image display system, 2 ... Digital camera, 4 ... DPF 10, 40 ... CPU, 12, 42 ... Imaging part, 18 ... Gyro sensor, 26, 54 ... LCD driver, 32, 74 ... WiFi transmission / reception part, 34, 70: Operation switch, 36, 72: Display.

Claims (12)

Imaging means;
Analyzing means for recognizing a face from an image captured by the imaging means and generating face analysis information;
Index generating means for generating an index based on the face analysis information;
User information storage means for storing at least one of the user's face, face analysis information, and the user's index generated as a result of imaging the user's face with the imaging means,
Display means for displaying the indicator;
At least one of the subject's face, face analysis information, and the subject index included in the captured image generated by imaging the subject with the imaging means, and the user's face, face analysis information, and the user indicator Determining means for comparing at least one and determining whether or not the user information is included in the captured image;
A portable terminal comprising: a display control unit that permits the display unit to display the index when the determination unit determines that the user information is included in the captured image.   The transmission unit for transmitting the image data stored in the image data storage unit to the image display device when user authentication is completed in the image display device. Mobile device. A detection unit for detecting the attitude of the mobile terminal;
The index creation unit creates a control index including image control information for controlling an image displayed on the image display device based on a detection result in the detection unit,
The mobile terminal according to claim 1, wherein the display control unit displays the control index on the display unit.   The mobile terminal according to claim 1, wherein the index includes URL information of a server on a network in which the image data is stored.   The mobile terminal according to claim 1, wherein the transmission unit is a wireless communication unit. Display means for displaying an image;
Imaging means;
Receiving means for receiving image data from a mobile terminal;
Analyzing means for recognizing a face from an image captured by the imaging means and generating face analysis information;
An index analysis unit that recognizes an index including face analysis information from an image captured by the imaging unit and reads the face analysis information from the index;
The face analysis information generated by the analysis unit and the index analysis unit are read based on the captured image of the subject captured by the imaging unit and containing the face and the index including the face analysis information within the same angle of view. Determining means for determining whether the received face analysis information is the same;
When the determination unit determines that the face analysis information generated by the analysis unit and the face analysis information read by the index analysis unit are the same, the reception unit is allowed to receive image data. An image display apparatus comprising: a reception control means for performing User information storage means for storing at least one of the user's face, face analysis information, and the user's index generated as a result of imaging the user's face with the imaging means,
The determination means includes at least one of the subject's face, face analysis information, and the subject's index generated by imaging the subject with the imaging means, and the user's face, face analysis information, and the user's index. Collating with at least one to determine whether or not the user information is included in the captured image;
The image display apparatus according to claim 6, wherein the reception control unit permits the reception unit to receive image data when the determination unit determines that the user information is included in the captured image. . An image deforming unit configured to deform an image based on the image data received via the receiving unit according to the position and shape of the display unit of the mobile terminal displayed on the display unit;
The image display device according to claim 6, wherein the display control unit superimposes and displays the image deformed by the image deforming unit on the display unit of the mobile terminal displayed on the display unit.   When a control index including image control information displayed on the display unit of the mobile terminal is captured by the imaging unit, the image is displayed on the display unit based on the image control information included in the control index. The image display apparatus according to claim 6, further comprising an image control unit that controls an image on the screen.   When the index includes URL information of a server on the network in which the image data is stored, the receiving unit receives the image data from the server indicated by the URL information. The image display device according to any one of claims 6 to 9.   The image display device according to claim 6, wherein the receiving unit is a wireless communication unit. An image display system comprising a mobile terminal and an image display device,
The portable terminal includes first imaging means,
First analysis means for recognizing a face from an image captured by the imaging means and generating face analysis information;
Index generating means for generating an index based on the face analysis information;
First user information storage means for storing, as user information, at least one of the user's face, face analysis information, and the user's index generated by imaging the user's face with the imaging means;
First display means for displaying the indicator;
At least one of the subject's face, face analysis information, and the subject index included in the captured image generated by imaging the subject with the imaging means, and the user's face, face analysis information, and the user indicator First determination means for comparing at least one and determining whether or not the user information is included in the captured image;
A display control unit that permits the display unit to display the index when the determination unit determines that the user information is included in the captured image;
The image display device includes:
Second display means for displaying an image;
Second imaging means;
Receiving means for receiving image data from the portable terminal;
Second analysis means for recognizing a face from the image captured by the imaging means and generating face analysis information;
An index analysis unit that recognizes an index including face analysis information from an image captured by the imaging unit and reads the face analysis information from the index;
The face analysis information generated by the analysis unit and the index analysis unit are read based on the captured image of the subject captured by the imaging unit and containing the face and the index including the face analysis information within the same angle of view. Second determination means for determining whether or not the received face analysis information is the same;
When the determination unit determines that the face analysis information generated by the analysis unit and the face analysis information read by the index analysis unit are the same, image data is transferred from the portable terminal to the reception unit. An image display system comprising: a reception control unit that permits reception of the image.

Images