WO2015029393A1 - 情報処理装置および情報処理方法 - Google Patents
情報処理装置および情報処理方法 Download PDFInfo
- Publication number
- WO2015029393A1 WO2015029393A1 PCT/JP2014/004298 JP2014004298W WO2015029393A1 WO 2015029393 A1 WO2015029393 A1 WO 2015029393A1 JP 2014004298 W JP2014004298 W JP 2014004298W WO 2015029393 A1 WO2015029393 A1 WO 2015029393A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- information processing
- information
- processing apparatus
- scene
- Prior art date
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 70
- 238000003672 processing method Methods 0.000 title claims description 4
- 238000001514 detection method Methods 0.000 claims abstract description 5
- 238000003860 storage Methods 0.000 claims description 58
- 238000012545 processing Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 4
- 238000000034 method Methods 0.000 description 45
- 238000010586 diagram Methods 0.000 description 15
- 238000005516 engineering process Methods 0.000 description 11
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 6
- 238000004891 communication Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 230000001133 acceleration Effects 0.000 description 5
- 238000003709 image segmentation Methods 0.000 description 5
- 125000002066 L-histidyl group Chemical group [H]N1C([H])=NC(C([H])([H])[C@](C(=O)[*])([H])N([H])[H])=C1[H] 0.000 description 3
- 230000000284 resting effect Effects 0.000 description 3
- 235000009508 confectionery Nutrition 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000001747 pupil Anatomy 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5862—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9032—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
Definitions
- the present technology relates to an information processing apparatus and an information processing method used for a user to carry and view content.
- Patent Document 1 the technique disclosed in Patent Document 1 is applied to an image display system that reads a predetermined number of images from a memory that stores a plurality of images converted into image data and outputs them on a screen.
- a user's attention direction is determined by detecting the user's line-of-sight direction, and an image having a feature similar to the feature of the attention portion is selected and displayed.
- Patent Document 2 The technique disclosed in Patent Document 2 is applied to an information presentation system that includes a gaze tracking device that determines a gaze position on a display device and is configured to present information to a user.
- this information presenting system new information is selected and presented based on the correlation with the level of interest of the user by monitoring the gaze position for a plurality of displayed information.
- Patent Document 3 discloses a technique for detecting the state of a user in a multimedia information space where a user can view video or music and outputting the next media information based on the result. Yes.
- An object of the present technology is to provide a functionally superior information processing apparatus that provides information related to content to a user who is viewing the content.
- An information processing apparatus includes a line-of-sight position detection unit that detects a user's line-of-sight position with respect to a display screen, a position information acquisition unit that acquires the user's position information, and the detected line-of-sight position Based on the above, the scene that the user pays attention to in the content displayed on the display screen and the object in the scene are determined, and search information is generated from the determined object and the acquired position information. And a control unit.
- the control unit may search the recommended content in units of scenes using the search information for the storage device storing the search target content.
- the storage device is a device that can be synchronized with the storage device of the recorder through the network.
- the control unit may cause a recorder connected through a network to set a recording reservation using the search information.
- the control unit determines a scene that the user pays attention to in the content displayed on the display screen based on the detected line-of-sight position, and determines an image of the determined scene based on a feature amount.
- the region may be divided into the above regions, a region having the highest gaze frequency is determined based on the detected gaze position, and the object is determined based on the feature amount of the region.
- the control unit may determine the object in consideration of priorities between genres based on a user's viewing location and viewing time.
- the control unit determines, based on the detected line-of-sight position, a scene that a user pays attention to in the content displayed on the display screen, and a plurality of other users are determined with respect to the determined scene.
- the object may be determined based on statistical information indicating what is the most focused object for the scene.
- the control unit determines a scene that a user pays attention to in the content displayed on the display screen based on the detected line-of-sight position, and the determined scene is determined by a plurality of other users.
- the object may be determined based on a feature vector of an area where the line of sight is most collected with respect to the scene.
- the control unit may determine the object in consideration of metadata regarding the scene given to the determined scene.
- An information processing apparatus includes a result of detecting a user's line-of-sight position with respect to a display screen of the user information processing apparatus from the user information processing apparatus, position information of the user information processing apparatus, and the The identification information of the content being viewed on the display screen is acquired, and the scene that the user pays attention to in the content specified based on the acquired identification information of the content based on the acquired line-of-sight position, and this scene And a control unit that generates search information from the determined object and the acquired position information, and transmits the search information to the information processing apparatus of the user.
- FIG. 1 is a diagram illustrating an overview of an overall configuration of a system 100 that is a first embodiment of the present technology.
- 2 is a diagram showing an outline of the configuration of the system 100.
- FIG. 2 is a block diagram showing an overall configuration of a recorder 200.
- FIG. 2 is a block diagram showing an overall configuration of a mobile terminal 300.
- FIG. 2 is a block diagram showing a configuration of a portable storage 400.
- FIG. 4 is a flowchart showing an operation of the mobile terminal 300 related to the above technique. It is a figure for demonstrating the gaze object determination process 1.
- FIG. It is a figure which shows the example of the priority between genres based on a user's viewing place and viewing time. It is a figure for demonstrating the gaze object determination process 2.
- FIG. 1 is a diagram illustrating an overview of an overall configuration of a system 100 that is a first embodiment of the present technology.
- FIG. 2 is a diagram showing an outline of the configuration of the system 100.
- FIG. 10 is a flowchart of an operation of gaze object determination processing 3.
- 10 is a flowchart showing a modified example of the operation related to the technique in the portable terminal 300 and the recorder 200.
- 10 is a flowchart showing a modified example of the operation related to the technique in the portable terminal 300 and the recorder 200.
- FIGS. 1 and 2 are diagrams showing an outline of the overall configuration of a system 100 using the present technology.
- the system 100 includes a plurality of recorders 200, a plurality of portable terminals 300, and a plurality of portable storages 400.
- Each user has a recorder 200, a portable terminal 300, and a portable storage 400, and these devices are connected to each other via a network 500.
- FIG. 1 shows a state where three users each view content in the portable storage 400.
- FIG. 2 shows a state in which the user U3 connects his / her portable storage 400 to his / her recorder 200 and copies contents from the recorder 200 to the portable storage 400.
- the recorder 200 receives and records television broadcast radio waves, receives and records cable television broadcast signals, and acquires and stores content such as video from the network 500. Specifically, it is a Blu-ray (registered trademark) recorder or the like.
- the portable storage 400 is connected to the recorder 200 or the portable terminal 300 for use.
- the connection may be made by wire or by wireless communication based on a wireless standard such as Wi-Fi (Wireless (Fidelity).
- the mobile terminal 300 is a smartphone, a tablet, a portable PC, or the like. As shown in FIG. 1, when the mobile terminal 300 is connected to the portable storage 400, the user can play and view the content in the portable storage 400 using application software (application) of the mobile terminal 300. .
- application application
- the application of the mobile terminal 300 searches the content data stored in the portable storage 400 for content data that is expected to be interesting to the user, and recommends viewing to the user.
- FIG. 3 is a block diagram showing the overall configuration of the recorder 200.
- the recorder 200 includes a network I / F (Interface) 1, an antenna 2, a tuner 3, a recording processing unit 4, an information processing unit 5, an I / F unit 6, a memory 7, a disk drive 8, and a first storage (database) 9. , A reproduction processing unit 10, a display processing unit 11, and a feature extraction unit 13.
- the recorder 200 includes a system controller 14, a data bus 15, and a control bus 16.
- the network I / F 1 exchanges information with the mobile terminal 300, the portable storage 400, and the like via the network 500.
- the tuner 3 receives broadcast radio waves via the antenna 2.
- the recording processing unit 4 performs processing for recording content data acquired by the tuner 3 or the network I / F 1 in the first storage 9.
- the information processing unit 5 searches content on the web in the network 500 such as the Internet based on information received from the mobile terminal 300 and accumulated in the first storage 9.
- the I / F unit 6 receives a signal from the operation unit 17 which is a remote controller.
- the memory 7 is a work area of the recorder 200.
- the disk drive 8 reads content recorded on the disk-shaped recording medium 8a, and writes content stored in the first storage 9 to the disk-shaped recording medium 8a.
- the first storage 9 records content data acquired by the recorder 200 and information related to the recording reservation acquired from the mobile terminal 300.
- the reproduction processing unit 10 performs reproduction processing of content data recorded in the first storage 9.
- the display processing unit 11 outputs an image to a display 12 such as a television receiver or a monitor connected to the recorder 200.
- the feature extraction unit 13 extracts features from content image, sound, and text information. The extracted features are used for recording reservation.
- the system controller 14 controls each part of the recorder 200.
- the data bus 15 is a bus for exchanging data such as contents between the respective units in the recorder 200.
- the control bus 16 is a bus for exchanging data for performing control between each unit in the recorder 200 and the system controller.
- FIG. 4 is a block diagram showing the overall configuration of the mobile terminal 300.
- the mobile terminal 300 includes an antenna 30, a GPS (Global Positioning System) receiver 21, a camera 22, a distance sensor 23, an acceleration sensor 24, a processor 25 such as a CPU (Central Processing Unit), a communication unit 26, a storage device 27, and an operation.
- the unit 28 and the display unit 29 are included.
- the GPS receiver 21 receives a signal from a GPS satellite via the antenna 30 and detects the current position of the mobile terminal 300.
- the camera 22 captures scenery and the like, but is used here to detect the direction of the line of sight of a user who views content using the mobile terminal 300.
- the distance measuring sensor 23 detects a distance to a user (for example, a face) using the mobile terminal 300.
- the acceleration sensor 24 detects the movement of the mobile terminal 300 and detects in what state the mobile terminal 300 is held. For example, when a user views content on the mobile terminal 300, the mobile terminal 300 is considered to be placed in a substantially stationary state.
- the processor 25 controls the entire mobile terminal 300 and executes application software (application) stored in the storage device 27.
- the communication unit 26 communicates with the portable storage 400 via the network 500 based on a mobile phone communication standard such as 3G (Third Generation) or LTE (Long Term Evolution) or a wireless communication standard such as Wi-Fi. Communicate.
- the operation unit 28 receives an input for a user to operate the mobile terminal 300.
- the display unit 29 displays an operation screen of the mobile terminal 300, a playback screen when playing back content in the portable storage 400, and the like.
- FIG. 5 is a block diagram showing the configuration of the portable storage 400.
- the portable storage 400 includes an antenna 40, a wireless unit 41, a control unit 42, a second storage 43, and an operation unit 44.
- the wireless unit 41 communicates with the recorder 200 and the mobile terminal 300 via the antenna 40.
- the control unit 42 controls the entire portable storage 400, synchronizes content data with the first storage 9 of the recorder 200, and sends out content data to the mobile terminal 300.
- the control unit 42 may also perform content data encryption processing and decryption processing.
- the second storage 43 stores the content data copied from the first storage 9 in synchronization with the recorder 200.
- the second storage 43 stores the recorded content and a unique identifier for uniquely identifying the content.
- the operation unit 44 receives a user instruction regarding whether the portable storage 400 is connected to the recorder 200 or the portable terminal 300. According to the received instruction, the control unit 42 exchanges data with the recorder 200 or the portable terminal 300 via the wireless unit 41.
- the present embodiment relates to a technology for searching for information related to contents viewed by the user of the mobile terminal 300 and providing the information to the user in the system 100 having such a configuration.
- FIG. 6 is a flowchart showing the operation of the portable terminal 300 related to the above technique.
- the user of the mobile terminal 300 operates the operation unit 28 to instruct the reproduction of arbitrary content (for example, content downloaded from the network 500, content acquired from the portable storage 400 and stored in the storage device 27) (Ste 101).
- the processor 25 reproduces the content in accordance with this instruction and displays it on the screen of the display unit 29 (step S102).
- the processor 25 detects the viewing state of the content by the user (step S103).
- the processor 25 When detecting the viewing state, the processor 25 detects a state in which the user gazes at an object of a certain scene (reproduction time) (Y in step S104), and determines the scene and the object being watched in the scene. (Step S105). This object determination will be described in detail later.
- the processor 25 acquires the position information of the user (mobile terminal 300) using the GPS receiving unit 21 (step S106).
- the processor 25 If the user position information is successfully acquired, the processor 25 generates content search information from the position information and the object information determined in step S105.
- the content search information is information for searching for content stored in the portable storage 400, for example.
- the processor 25 executes a search for the content group stored in the portable storage 400 using the generated content search information (step S108). This search is performed on metadata related to content stored in the portable storage 400.
- the content metadata includes start / end times and scene details (for example, data on stores, products, commercials, etc.) as metadata for each scene.
- a search is performed on the metadata for each scene using a search query.
- step S108 the processor 25 uses the metadata of the content scene as a list of information on the scene of the corresponding content as a list of recommended content scenes. It is presented on the screen of the display unit 29 (step S109). If the search for the corresponding content fails (N in step S108), the processor 25 does nothing.
- the processor 25 reads the data of the corresponding scene from the portable storage 400 and reproduces it.
- step S104 When the gaze state by the user is not detected in step S104 (N in step S104), the processor 25 does nothing.
- the user's line-of-sight state is captured by the camera 22 to measure the position of the pupil in the contour of the user's eyes, the resting time (continuous resting time, average resting time, etc.), etc. And the distance to the user's face.
- the position (line-of-sight position) that the user is looking at can be detected from the position of the pupil and the distance from the mobile terminal 300 to the user's face, and the stationary time can determine whether the action that the user sees corresponds to gaze. it can.
- time-series data of the line-of-sight position indicating which position in the image has been viewed for how long for each scene is obtained.
- the processor 25 first detects a scene watched by the user (hereinafter referred to as a “watching scene”).
- the gaze scene is, for example, a scene with little movement of the user's line of sight.
- the processor 25 detects, from the time series data of the line-of-sight position, a scene in which the movement of the line of sight is equal to or less than a predetermined threshold as the gaze scene.
- the processor 25 determines one or more objects to be watched by the user from one or more objects existing in the image from the image of the gaze scene. This process is referred to herein as “gazing object determination process”. Next, the gaze object determination process will be described.
- FIG. 7 is a diagram for explaining the gaze object determination process 1.
- the image is segmented into one or more regions where the characteristics are uniform (image segmentation). Process).
- the processor 25 determines an area having the highest line-of-sight frequency based on time-series data of the line-of-sight position for one or more areas divided by the image segmentation process.
- the processor 25 determines information indicating what kind of object the image in the region is based on the feature amount of the image in the region with the highest line-of-sight frequency.
- FIG. 8 is a diagram illustrating an example of priorities between genres based on a user's viewing location and viewing time. For example, when the viewing place is “Ginza” and the viewing time is “16:00”, the priority among genres is “sweets” ⁇ “fashion” ⁇ “entertainment” from the top.
- FIG. 9 is a diagram for explaining the gaze object determination process 2.
- the processor 25 determines which of the objects in the image of those regions is the final determination result. It is determined based on the priority among genres. For example, in FIG. 9, it is assumed that two objects “cake” and “painting” are obtained by the gaze object determination process 1. At this time, in the priority between genres based on the viewing location and viewing time of the user shown in FIG. 8, if the viewing location is “Ginza” and the viewing time is “16:00”, the genre to which “cake” belongs “Sweets” has the highest priority. From this, “cake” is obtained as the final object determination result.
- the priority between genres based on the viewing location and viewing time of the user may be prepared according to the user's sex, age, occupation, address, and other types. Further, the priority information between the genres may be updated sequentially.
- FIG. 10 is a diagram for explaining the gaze object determination process 3.
- the processor 25 acquires and accumulates statistical information indicating what is the most focused object for the gaze scene in the mobile terminals 300 of other users. 2.
- the processor 25 determines the information of the object that has collected the most line of sight from the accumulated statistical information with respect to the image of the gaze scene by the user.
- the object information obtained in the mobile terminals 300 of other users is obtained by, for example, the gaze object determination process 1 described above.
- one gaze is finally determined based on the priority between genres based on the viewing location and the viewing time as in the gaze object determination process 2. You may make it narrow down to the attribute of an object.
- FIG. 11 is a diagram for explaining the gaze object determination process 4.
- the processor 25 acquires and accumulates feature vectors of an area where the line of sight is most collected with respect to the gaze scene by a plurality of other users.
- the processor 25 divides the image into one or more regions each having a uniform feature by analyzing the features of the detected gaze scene image, for example, features such as brightness, color, texture, etc. Image segmentation processing).
- the processor 25 determines an area having a feature amount closest to the accumulated feature amount among the divided regions, and based on the feature amount, an attribute indicating what object the image of the region is. Determine.
- Post metadata is metadata generated by, for example, artificially checking the contents of each program scene.
- the post-metadata includes, for example, information such as program name and broadcast time that is metadata related to a program, scene time and scene content that is metadata related to a scene, and the like.
- post metadata is stored in a post metadata server on the network.
- the post-metadata server manages post-metadata for each program and performs a service for distributing the post-metadata through the network.
- FIG. 12 is a diagram for explaining an operation for determining a gaze object in consideration of metadata after the gaze object determination process 5.
- FIG. 13 is a flowchart of the operation of the gaze object determination process 3.
- the image is segmented into one or more regions where the characteristics are uniform (image segmentation). Process) (step S301).
- the processor 25 determines an area having the highest line-of-sight frequency based on time-series data of the line-of-sight position for one or more areas divided by the image segmentation process (step S302).
- the processor 25 detects the top n regions with the highest line-of-sight frequency and feature quantities of neighboring images (step S303).
- the feature amount of the neighboring image is detected, for example, when determining “cake” based on the image feature amount, the image feature amount of “dish” is assumed to have a high probability that the cake is on the plate. This is also to determine whether or not the object is “cake” comprehensively. In this way, when determining a certain object, it is possible to increase the probability of determination accuracy by determining an object related to the object. This process can also be used for other gaze object determination processes.
- the processor 25 accesses the post-metadata server, acquires the post-gaze metadata, refers to the scene content at the corresponding scene time in the post-capture metadata, and uses the morphological analysis or the like to refer to the scene contents. (Step S304).
- the processor 25 determines information indicating what object each image is based on the feature amount of the image of the top n regions, and the word obtained from the post-metadata, The information of the objects in the top n areas is compared, and the information of the object that matches the word is used as the final object determination result (step S305).
- the operation of determining the gaze object with the metadata added thereafter is executed when the processor 25 can acquire the post metadata, and is executed when it can be acquired. You may make it switch so that the process 1 may be performed.
- the post-gaze scene metadata hit in this process may be added to the metadata stored in the portable storage 400 and the recorder 200. As a result, the accuracy of the search for the content stored in the portable storage 400 can be improved.
- the present embodiment it is possible to obtain information for searching for content related to an object of a scene being watched by the user of the mobile terminal 300 and position information of the user.
- the corresponding content can be searched from the content group stored in the portable storage 400 in units of scenes. Thereby, the searchability for the content stored in the portable storage 400 is improved.
- FIG. 14 and FIG. 15 are flowcharts showing modifications of the operation related to the above technique in the portable terminal 300 and the recorder 200.
- the content search information generated by the mobile terminal 300 of the first embodiment is used for recording reservation by the recorder 200.
- the operations from step S201 to step S209 are the same as the operations from step S101 to step S109 of the first embodiment, and thus description thereof is omitted.
- step S210 the processor 25 determines whether or not the recording mode is set (step S210).
- the recording mode is a mode arbitrarily set by the user of the mobile terminal 300.
- the processor 25 transfers the content search information generated in step S207 to the recorder 200 through the network (step S212).
- the system controller 14 of the recorder 200 receives the content search information, the system controller 14 determines whether or not the recording reservation of the program specified based on this information has been completed. When the recording reservation has been completed (Y in step S213), the system controller 14 of the recorder 200 responds to the portable terminal 300 to that effect.
- the data for specifying the program for example, the program name, the broadcast start end time, and the like are included in the information for content search. This is the case where information that can specify a program such as broadcast station information is included.
- step S211 the processor 25 of the portable terminal 300 shifts to determination of the end of content reproduction (step S211), and if not completed (N in step S211), the content reproduction state of step S202 is entered. Return.
- step S215 if the system controller 14 of the recorder 200 cannot identify the program based on the content search information (N in step S215), it accumulates the received content search information (step S214). Wait until a sufficient amount of information is accumulated. Thereafter, the process proceeds to step S208.
- the system controller 14 of the recorder 200 searches for the corresponding content using the accumulated information (step S216). This search is performed on, for example, an electronic program guide or a server that provides a content publishing service on the web.
- step S217 When the corresponding content is searched (Y in step S217), the system controller 14 of the recorder 200 sets a recording reservation for the content (step S218). When the corresponding content is not found (N in step S217), the system controller 14 of the recorder 200 performs a predetermined warning process (step S219). Thereafter, the process proceeds to step S211.
- the warning process indicates, for example, a display indicating that the content search has failed in the mobile terminal 300, or a content search result and a recording state for each transfer of metadata from the mobile terminal 300 in the recorder 200, for example. Processing such as displaying a list can be considered.
- the recorder 200 is used by using the information for searching for related content generated from the object of the scene being watched by the user of the mobile terminal 300 and the position information of the user. On the other hand, it is possible to set a recording reservation for related contents.
- At least a part or all of the processing of the mobile terminal 300 can be performed by one or more servers (corresponding to the information processing apparatus according to claim 11) arranged in the network 500. is there.
- the server acquires the result of detecting the user's line-of-sight position, the user's position information, and the identification information of the content being viewed on the display screen from the user's portable terminal 300. Based on the acquired line-of-sight position, the server determines a scene that the user pays attention to in the content specified based on the acquired content identification information, and an object in the scene, and determines the determined object and Information for search is generated from the acquired position information and transmitted to the mobile terminal 300 of the user. Thereby, the processing load of the portable terminal 300 can be reduced.
- the above gaze object determination process may be executed only when it is determined that the mobile terminal 300 is in the following situation. That is, the mobile terminal 300 is provided with an acceleration sensor 24 that detects the movement of the mobile terminal 300 and a distance measurement sensor 23 that detects a distance to a user (for example, a face) using the mobile terminal 300.
- the processor 25 is in close proximity to a user who is using the mobile terminal 300, and the movement of the mobile terminal 300 is determined in advance based on the output of the acceleration sensor 24.
- the predetermined value is a level that allows the user to determine the state of less movement when the user is holding the mobile terminal 300 and is gazing at the content being displayed on the display unit 29, It can be set to such an extent that a state where the portable terminal 300 is placed in a stable place can be determined.
- These threshold values may be appropriately switched by the user.
- the accuracy of the gaze object determination can be improved. That is, when the user holds the portable terminal 300 in his / her hand, the movement of the portable terminal 300 when the user is gazing at the displayed content is naturally reduced. This is also because the situation where the user is present near the portable terminal 300 installed in a stable place is one of the situations when the user is watching the content being displayed.
- the acceleration sensor 24 is used to detect the movement of the mobile terminal 300, other types of sensors may be used as long as the movement of the mobile terminal 300 can be detected.
- the distance measuring sensor 23 is used to detect the presence of a user who is using the mobile terminal 300, other types of sensors can be used as long as the presence of the user using the mobile terminal 300 can be detected. May be.
- this technique can also take the following structures.
- a line-of-sight position detection unit that detects a user's line-of-sight position with respect to the display screen;
- a location information acquisition unit that acquires location information of the user; Based on the detected line-of-sight position, the scene that the user is interested in in the content displayed on the display screen and the object in the scene are determined, and a search is performed from the determined object and the acquired position information.
- a control unit for generating information for An information processing apparatus comprising:
- the control unit searches a recommended content in units of scenes using the search information for a storage device in which the content to be searched is stored.
- the storage device is a device capable of performing synchronization processing with a storage device of a recorder through a network.
- the control unit causes a recorder connected through a network to set a recording reservation using the search information.
- the information processing apparatus is Based on the detected line-of-sight position, determine a scene that the user pays attention to in the content displayed on the display screen, Dividing the image of the determined scene into one or more regions based on the feature amount; Determine the region with the highest gaze frequency based on the detected gaze position, An information processing apparatus that determines the object based on a feature amount of the area.
- the information processing apparatus is An information processing apparatus that determines the object in consideration of priority between genres based on a user's viewing location and viewing time.
- the information processing apparatus is Based on the detected line-of-sight position, determine a scene that the user pays attention to in the content displayed on the display screen, An information processing apparatus that determines the object of the determined scene based on statistical information indicating what is the most focused object for the scene in portable terminals of a plurality of other users.
- the information processing apparatus is Based on the detected line-of-sight position, determine a scene that the user pays attention to in the content displayed on the display screen, An information processing apparatus that determines the object for the determined scene based on a feature vector of a region in which a plurality of other users gather the most gaze for the scene.
- the information processing apparatus is An information processing apparatus that determines the object in consideration of metadata regarding the scene assigned to the determined scene.
- the information processing apparatus according to any one of (1) to (9), A first sensor for detecting the presence or absence of a user who is using the information processing apparatus; A second sensor for detecting movement of the information processing apparatus; The control unit has a user who is using the information processing apparatus based on the output of the first sensor, and the movement of the information processing apparatus is determined in advance based on the output of the second sensor. An information processing apparatus that determines the object when the value is equal to or less than a predetermined value.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Television Signal Processing For Recording (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- User Interface Of Digital Computer (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
<第1の実施形態>
[システム全体構成の概要]
本技術を用いるシステム全体の構成について、その概要を説明する。
次に、レコーダ200の全体構成を説明する。図3は、レコーダ200の全体構成を示すブロック図である。
次に、携帯端末300(特許請求の範囲の請求項1の情報処理装置)の全体構成を説明する。図4は、携帯端末300の全体構成を示すブロック図である。
携帯端末300は、アンテナ30、GPS(Global Positioning System)受信部21、カメラ22、測距センサ23、加速度センサ24、CPU(Central Processing Unit)などのプロセッサ25、通信部26、記憶装置27、操作部28、表示部29を含んでいる。
次に、ポータブルストレージ400の構成を説明する。図5はポータブルストレージ400の構成を示すブロック図である。
図6は携帯端末300の上記技術に係わる動作を示すフローチャートである。
まず、携帯端末300のユーザが、操作部28を操作して任意のコンテンツ(例えば、ネットワーク500からダウンロードしたコンテンツ、ポータブルストレージ400から取得して記憶装置27に保存したコンテンツ)の再生を指示する(ステップ101)。するとプロセッサ25がこの指示に従ってそのコンテンツを再生し、表示部29の画面に表示させる(ステップS102)。
次に、ステップS104およびステップS105のユーザが注視したシーンとオブジェクトを判定する方法について説明する。
次に、この注視オブジェクト判定処理について説明する。
図7は、注視オブジェクト判定処理1を説明するための図である。
1.まず、検出された注視シーンの画像について、画像の特徴、例えば、明るさ、色、テクスチャなどの特徴の解析により、この画像を特徴が各々一様な1以上の領域に分割する処理(画像セグメンテーション処理)を行う。
ユーザの視聴場所および視聴時刻に基づくジャンル間の優先度を、オブジェクトの判定に用いてもよい。
図8は、ユーザの視聴場所および視聴時刻に基づくジャンル間の優先度の例を示す図である。例えば、視聴場所が"銀座"で、視聴時刻が"16:00"である場合、ジャンル間の優先度は上位から"スイーツ"→"ファッション" →"エンタテイメント"である。
プロセッサ25は、視線位置の時系列データをもとに視線頻度が最も高い領域が複数判定された場合、それらの領域の画像のオブジェクトのうちどのオブジェクトを最終的な判定結果とするかを、上記のジャンル間の優先度をもとに決定する。
例えば、図9において、「ケーキ」と「絵画」の2つのオブジェクトが注視オブジェクト判定処理1によって得られたとする。このとき、図8に示したユーザの視聴場所および視聴時刻に基づくジャンル間の優先度では、視聴場所が"銀座"で、視聴時刻が"16:00"である場合、「ケーキ」が属するジャンルである「スイーツ」の優先度が最も高い。このことから、「ケーキ」が最終的なオブジェクトの判定結果として得られる。
図10は注視オブジェクト判定処理3を説明するための図である。
1.プロセッサ25は、他の複数のユーザの携帯端末300において、注視シーンに対して最も視線を集めたオブジェクトが何であるかという統計情報を取得して蓄積する。
2.プロセッサ25は、自ユーザによる注視シーンの画像に対して、上記の蓄積された統計情報から、最も視線を集めたオブジェクトの情報を判定する。
図11は注視オブジェクト判定処理4を説明するための図である。
1.プロセッサ25は、他の複数のユーザによって、注視シーンに対して最も視線を集めた領域の特徴ベクトルを取得して蓄積する。
2.プロセッサ25は、検出された注視シーンの画像について、画像の特徴、例えば、明るさ、色、テクスチャなどの特徴の解析により、この画像を特徴が各々一様な1以上の領域に分割する処理(画像セグメンテーション処理)を行う。
3.プロセッサ25は、分割された領域の中で、特徴量が最も上記蓄積された特徴量に近い領域を判定し、この特徴量をもとに、その領域の画像が何のオブジェクトであるかという属性を判定する。
次に、注視オブジェクトの判定において、番組に付与された後メタデータを参考にする方法を説明する。
図12は注視オブジェクト判定処理5の後メタデータを加味して注視オブジェクトを判定する動作を説明するための図である。
図13は、注視オブジェクト判定処理3の動作のフローチャートである。
これにより、特徴量に基づくオブジェクトの判定の精度限界を越えて、より高精度なオブジェクト判定結果が得られる。
図14および図15は、携帯端末300およびレコーダ200での上記技術に係わる動作の変形例を示すフローチャートである。
この場合、サーバは、ユーザの携帯端末300より、ユーザの視線位置を検出した結果とユーザの位置情報と表示画面で視聴中のコンテンツの識別情報を取得する。サーバは、取得した視線位置をもとに、取得したコンテンツの識別情報をもとに特定されるコンテンツ中でユーザが注目するシーンと、このシーン内のオブジェクトを判定し、この判定されたオブジェクトと取得された位置情報から検索用の情報を生成し、ユーザの携帯端末300に送信する。
これにより、携帯端末300の処理負荷を軽減することができる。
上記の注視オブジェクトの判定処理は、携帯端末300が次のような状況にあることが判定された場合のみ、実行されるようにしてもよい。すなわち、携帯端末300には、携帯端末300の動きを検出する加速度センサ24と、携帯端末300を利用するユーザ(例えば顔)までの距離などを検出する測距センサ23が設けられている。
(1)表示画面に対するユーザの視線位置を検出する視線位置検出部と、
前記ユーザの位置情報を取得する位置情報取得部と、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンとこのシーン内のオブジェクトを判定し、この判定されたオブジェクトと前記取得された位置情報から検索用の情報を生成する制御部と、
を具備する情報処理装置。
前記制御部は、検索対象のコンテンツが格納されたストレージデバイスに対して、前記検索用の情報を用いて推薦コンテンツをシーンの単位で検索する
情報処理装置。
前記ストレージデバイスは、ネットワークを通じてレコーダのストレージデバイスと同期処理が可能なデバイスである
情報処理装置。
前記制御部は、ネットワークを通じて接続されたレコーダに対し、前記検索用の情報を用いて録画予約を設定させる
情報処理装置。
前記制御部は、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンを判定し、
前記判定されたシーンの画像を特徴量をもとに1以上の領域に分割し、
前記検出された視線位置をもとに視線頻度が最も高い領域を判定し、
この領域の特徴量をもとに前記オブジェクトを判定する
情報処理装置。
前記制御部は、
ユーザの視聴場所および視聴時刻に基づくジャンル間の優先度を加味して、前記オブジェクトを判定する
情報処理装置。
前記制御部は、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンを判定し、
前記判定されたシーンに対して、他の複数のユーザの携帯端末において当該シーンに対して最も視線を集めたオブジェクトが何であるかという統計情報をもとに前記オブジェクトを判定する
情報処理装置。
前記制御部は、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンを判定し、
前記判定されたシーンに対して、他の複数のユーザによって当該シーンに対して最も視線を集めた領域の特徴ベクトルをもとに前記オブジェクトを判定する
情報処理装置。
前記制御部は、
前記判定されたシーンに付与された当該シーンに関するメタデータを加味して前記オブジェクトを判定する
情報処理装置。
前記情報処理装置を利用中のユーザの有無を検出する第1のセンサと、
前記情報処理装置の動きを検出する第2のセンサと、
前記制御部は、前記第1のセンサの出力をもとに前記情報処理装置を利用中のユーザが存在し、かつ前記第2のセンサの出力をもとに前記情報処理装置の動きが予め決められた値以下であるとき、前記オブジェクトの判定を行う
情報処理装置。
14…システムコントローラ
21…GPS受信部
22…カメラ
23…測距センサ
25…プロセッサ
26…通信部
27…記憶装置
29…表示部
42…制御部
43…第2のストレージ
100…システム
200…レコーダ
300…携帯端末
400…ポータブルストレージ
500…ネットワーク
Claims (13)
- 表示画面に対するユーザの視線位置を検出する視線位置検出部と、
前記ユーザの位置情報を取得する位置情報取得部と、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンとこのシーン内のオブジェクトを判定し、この判定されたオブジェクトと前記取得された位置情報から検索用の情報を生成する制御部と、
を具備する情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記制御部は、検索対象のコンテンツが格納されたストレージデバイスに対して、前記検索用の情報を用いて推薦コンテンツをシーンの単位で検索する
情報処理装置。 - 請求項2に記載の情報処理装置であって、
前記ストレージデバイスは、ネットワークを通じてレコーダのストレージデバイスと同期処理が可能なデバイスである
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記制御部は、ネットワークを通じて接続されたレコーダに対し、前記検索用の情報を用いて録画予約を設定させる
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記制御部は、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンを判定し、
前記判定されたシーンの画像を特徴量をもとに1以上の領域に分割し、
前記検出された視線位置をもとに視線頻度が最も高い領域を判定し、
この領域の特徴量をもとに前記オブジェクトを判定する
情報処理装置。 - 請求項5に記載の情報処理装置であって、
前記制御部は、
ユーザの視聴場所および視聴時刻に基づくジャンル間の優先度を加味して、前記オブジェクトを判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記制御部は、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンを判定し、
前記判定されたシーンに対して、他の複数のユーザの携帯端末において当該シーンに対して最も視線を集めたオブジェクトが何であるかという統計情報をもとに前記オブジェクトを判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記制御部は、
前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンを判定し、
前記判定されたシーンに対して、他の複数のユーザによって当該シーンに対して最も視線を集めた領域の特徴ベクトルをもとに前記オブジェクトを判定する
情報処理装置。 - 請求項5に記載の情報処理装置であって、
前記制御部は、
前記判定されたシーンに付与された当該シーンに関するメタデータを加味して前記オブジェクトを判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記情報処理装置を利用中のユーザの有無を検出する第1のセンサと、
前記情報処理装置の動きを検出する第2のセンサと、
前記制御部は、前記第1のセンサの出力をもとに前記情報処理装置を利用中のユーザが存在し、かつ前記第2のセンサの出力をもとに前記情報処理装置の動きが予め決められた値以下であるとき、前記オブジェクトの判定を行う
情報処理装置。 - ユーザの情報処理装置より、
前記ユーザの情報処理装置の表示画面に対するユーザの視線位置を検出した結果と前記ユーザの情報処理装置の位置情報と前記表示画面で視聴中のコンテンツの識別情報を取得し、
前記取得した視線位置をもとに、前記取得したコンテンツの識別情報をもとに特定されるコンテンツ中でユーザが注目するシーンと、このシーン内のオブジェクトを判定し、この判定されたオブジェクトと前記取得された位置情報から検索用の情報を生成し、前記ユーザの情報処理装置に送信する制御部と
を具備する情報処理装置。 - 視線位置検出部により、表示画面に対するユーザの視線位置を検出し、
位置情報取得部により、前記ユーザの位置情報を取得し、
制御部により、前記検出された視線位置をもとに、前記表示画面に表示されたコンテンツ中でユーザが注目するシーンとこのシーン内のオブジェクトを判定し、この判定されたオブジェクトと前記取得された位置情報から検索用の情報を生成する
情報処理方法。 - 制御部により、
ユーザの情報処理装置より、前記ユーザの情報処理装置の表示画面に対するユーザの視線位置を検出した結果と前記ユーザの情報処理装置の位置情報と前記表示画面で視聴中のコンテンツの識別情報を取得し、
前記取得した視線位置をもとに、前記取得したコンテンツの識別情報をもとに特定されるコンテンツ中でユーザが注目するシーンと、このシーン内のオブジェクトを判定し、この判定されたオブジェクトと前記取得された位置情報から検索用の情報を生成し、前記ユーザの情報処理装置に送信する
情報処理方法。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201480046374.5A CN105518666B (zh) | 2013-08-29 | 2014-08-21 | 信息处理装置和信息处理方法 |
US14/915,209 US10019058B2 (en) | 2013-08-29 | 2014-08-21 | Information processing device and information processing method |
EP14840335.5A EP3040878B1 (en) | 2013-08-29 | 2014-08-21 | Information processing device and information processing method |
KR1020167001800A KR20160049530A (ko) | 2013-08-29 | 2014-08-21 | 정보 처리 장치 및 정보 처리 방법 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013-177745 | 2013-08-29 | ||
JP2013177745A JP2015046089A (ja) | 2013-08-29 | 2013-08-29 | 情報処理装置および情報処理方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015029393A1 true WO2015029393A1 (ja) | 2015-03-05 |
Family
ID=52585983
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/004298 WO2015029393A1 (ja) | 2013-08-29 | 2014-08-21 | 情報処理装置および情報処理方法 |
Country Status (6)
Country | Link |
---|---|
US (1) | US10019058B2 (ja) |
EP (1) | EP3040878B1 (ja) |
JP (1) | JP2015046089A (ja) |
KR (1) | KR20160049530A (ja) |
CN (1) | CN105518666B (ja) |
WO (1) | WO2015029393A1 (ja) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9503687B2 (en) * | 2015-03-24 | 2016-11-22 | Fuji Xerox Co., Ltd. | Personalized meeting event capture using egocentric tracking in smart spaces |
US10168771B2 (en) | 2015-07-30 | 2019-01-01 | International Business Machines Corporation | User eye-gaze based derivation of activity stream processing augmentations |
CN107544660B (zh) * | 2016-06-24 | 2020-12-18 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
CN106354777B (zh) * | 2016-08-22 | 2019-09-17 | 广东小天才科技有限公司 | 一种应用于电子终端的搜题方法及装置 |
JP7040448B2 (ja) | 2016-08-30 | 2022-03-23 | ソニーグループ株式会社 | 送信装置、送信方法、受信装置および受信方法 |
WO2019098037A1 (ja) | 2017-11-15 | 2019-05-23 | ソニー株式会社 | 情報処理端末、情報処理装置、および情報処理方法 |
US10943115B2 (en) * | 2018-07-24 | 2021-03-09 | Apical Ltd. | Processing image data to perform object detection |
CN109110646B (zh) * | 2018-08-06 | 2020-10-09 | 机汇网(深圳)科技有限公司 | 一种对塔吊安全检查管理的实现方法、系统及存储介质 |
US20220026981A1 (en) * | 2018-11-27 | 2022-01-27 | Sony Group Corporation | Information processing apparatus, method for processing information, and program |
JP7490372B2 (ja) * | 2020-01-21 | 2024-05-27 | キヤノン株式会社 | 撮像制御装置及びその制御方法 |
WO2023228342A1 (ja) * | 2022-05-26 | 2023-11-30 | 株式会社ジオクリエイツ | 情報処理システム、情報処理装置、情報処理方法及びプログラム |
CN115407867B (zh) * | 2022-07-20 | 2023-10-24 | 夏前明 | 一种基于多传感器的智能交互系统 |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09114846A (ja) | 1995-10-16 | 1997-05-02 | Fujitsu Ltd | ブラウジング表示方法および画像表示システム |
JPH10133810A (ja) | 1996-06-25 | 1998-05-22 | Sun Microsyst Inc | 視標追跡駆動情報検索方法および装置 |
JP2001216527A (ja) | 2000-02-04 | 2001-08-10 | Nippon Telegr & Teleph Corp <Ntt> | マルチメディア情報空間入出力装置およびその方法、ならびにそのプログラムを記録した記録媒体 |
JP2004181233A (ja) * | 2002-11-25 | 2004-07-02 | Eastman Kodak Co | 画像中の重要領域決定方法及びプログラム |
JP2010034687A (ja) * | 2008-07-25 | 2010-02-12 | Sharp Corp | 付加データ生成システム |
JP2011239158A (ja) * | 2010-05-10 | 2011-11-24 | Nippon Hoso Kyokai <Nhk> | ユーザ反応推定装置、ユーザ反応推定方法およびユーザ反応推定プログラム |
JP2012504830A (ja) * | 2008-10-03 | 2012-02-23 | スリーエム イノベイティブ プロパティズ カンパニー | 複数の視点のシーン解析のシステム及び方法 |
JP2012186621A (ja) * | 2011-03-04 | 2012-09-27 | Sony Corp | 情報処理装置、情報処理方法およびプログラム |
JP2012243043A (ja) * | 2011-05-18 | 2012-12-10 | Panasonic Corp | コンテンツ評価装置、方法、及びそのプログラム |
JP2013114595A (ja) * | 2011-11-30 | 2013-06-10 | Canon Inc | 情報処理装置、情報処理方法及びプログラム |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TW275590B (en) * | 1994-12-09 | 1996-05-11 | Sega Enterprises Kk | Head mounted display and system for use therefor |
US5912721A (en) * | 1996-03-13 | 1999-06-15 | Kabushiki Kaisha Toshiba | Gaze detection apparatus and its method as well as information display apparatus |
JPH10240491A (ja) * | 1997-02-26 | 1998-09-11 | Olympus Optical Co Ltd | 情報処理装置 |
US6037914A (en) * | 1997-08-25 | 2000-03-14 | Hewlett-Packard Company | Method and apparatus for augmented reality using a see-through head-mounted display |
JPH11265248A (ja) * | 1998-03-18 | 1999-09-28 | Sony Corp | 眼鏡型画像表示装置 |
JP4735242B2 (ja) * | 2005-12-22 | 2011-07-27 | トヨタ自動車株式会社 | 注視対象物体特定装置 |
WO2008012717A2 (en) * | 2006-07-28 | 2008-01-31 | Koninklijke Philips Electronics N. V. | Gaze interaction for information display of gazed items |
US8419654B2 (en) * | 2008-05-20 | 2013-04-16 | Panasonic Corporation | Eye gaze tracking apparatus, imaging apparatus, eye gaze tracking method, program, and integrated circuit |
JP4775671B2 (ja) * | 2008-12-26 | 2011-09-21 | ソニー株式会社 | 情報処理装置および方法、並びにプログラム |
CN101916264B (zh) * | 2010-07-30 | 2012-09-19 | 浙江大学 | 基于用户面部表情与目光分布检测的个性化网页推荐方法 |
JP5678643B2 (ja) * | 2010-12-21 | 2015-03-04 | ソニー株式会社 | 情報処理装置、情報処理方法およびプログラム |
US8996510B2 (en) * | 2011-08-23 | 2015-03-31 | Buckyball Mobile, Inc. | Identifying digital content using bioresponse data |
US9024844B2 (en) * | 2012-01-25 | 2015-05-05 | Microsoft Technology Licensing, Llc | Recognition of image on external display |
US9996150B2 (en) * | 2012-12-19 | 2018-06-12 | Qualcomm Incorporated | Enabling augmented reality using eye gaze tracking |
JP5908002B2 (ja) | 2013-03-20 | 2016-04-26 | 卉菱 曽 | ベニクスノキタケに含まれる化合物及びその用途 |
-
2013
- 2013-08-29 JP JP2013177745A patent/JP2015046089A/ja active Pending
-
2014
- 2014-08-21 WO PCT/JP2014/004298 patent/WO2015029393A1/ja active Application Filing
- 2014-08-21 KR KR1020167001800A patent/KR20160049530A/ko not_active Application Discontinuation
- 2014-08-21 CN CN201480046374.5A patent/CN105518666B/zh active Active
- 2014-08-21 US US14/915,209 patent/US10019058B2/en active Active
- 2014-08-21 EP EP14840335.5A patent/EP3040878B1/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09114846A (ja) | 1995-10-16 | 1997-05-02 | Fujitsu Ltd | ブラウジング表示方法および画像表示システム |
JPH10133810A (ja) | 1996-06-25 | 1998-05-22 | Sun Microsyst Inc | 視標追跡駆動情報検索方法および装置 |
JP2001216527A (ja) | 2000-02-04 | 2001-08-10 | Nippon Telegr & Teleph Corp <Ntt> | マルチメディア情報空間入出力装置およびその方法、ならびにそのプログラムを記録した記録媒体 |
JP2004181233A (ja) * | 2002-11-25 | 2004-07-02 | Eastman Kodak Co | 画像中の重要領域決定方法及びプログラム |
JP2010034687A (ja) * | 2008-07-25 | 2010-02-12 | Sharp Corp | 付加データ生成システム |
JP2012504830A (ja) * | 2008-10-03 | 2012-02-23 | スリーエム イノベイティブ プロパティズ カンパニー | 複数の視点のシーン解析のシステム及び方法 |
JP2011239158A (ja) * | 2010-05-10 | 2011-11-24 | Nippon Hoso Kyokai <Nhk> | ユーザ反応推定装置、ユーザ反応推定方法およびユーザ反応推定プログラム |
JP2012186621A (ja) * | 2011-03-04 | 2012-09-27 | Sony Corp | 情報処理装置、情報処理方法およびプログラム |
JP2012243043A (ja) * | 2011-05-18 | 2012-12-10 | Panasonic Corp | コンテンツ評価装置、方法、及びそのプログラム |
JP2013114595A (ja) * | 2011-11-30 | 2013-06-10 | Canon Inc | 情報処理装置、情報処理方法及びプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3040878A4 |
Also Published As
Publication number | Publication date |
---|---|
CN105518666B (zh) | 2020-01-17 |
EP3040878B1 (en) | 2019-10-02 |
KR20160049530A (ko) | 2016-05-09 |
JP2015046089A (ja) | 2015-03-12 |
EP3040878A4 (en) | 2017-01-25 |
CN105518666A (zh) | 2016-04-20 |
US20160209919A1 (en) | 2016-07-21 |
EP3040878A1 (en) | 2016-07-06 |
US10019058B2 (en) | 2018-07-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2015029393A1 (ja) | 情報処理装置および情報処理方法 | |
US11659220B2 (en) | System and method for surveying broadcasting ratings | |
CN108702524B (zh) | 利用多匹配检测和基于位置的消歧的媒体频道识别 | |
CN108702543B (zh) | 基于音频指纹的视频多匹配检测和对媒体频道识别消歧 | |
US10129515B2 (en) | Display control device, recording control device, and display control method | |
US20140026156A1 (en) | Determining User Interest Through Detected Physical Indicia | |
US9078047B2 (en) | Content reproduction device | |
US11706481B2 (en) | Media content identification on mobile devices | |
US11140034B2 (en) | Automatic determination of display device functionality | |
US20120042041A1 (en) | Information processing apparatus, information processing system, information processing method, and program | |
US9191607B2 (en) | Information display device, information display method, and information display system | |
GB2553912A (en) | Methods, systems, and media for synchronizing media content using audio timecodes | |
JP2006237879A (ja) | Cm検索方法、cm検索結果受給方法、cm検索装置、cm検索結果受給装置及びcm録画予約システム | |
CN110582011A (zh) | 产生相关联于广告的客制化视频的方法及系统 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14840335 Country of ref document: EP Kind code of ref document: A1 |
|
REEP | Request for entry into the european phase |
Ref document number: 2014840335 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014840335 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 20167001800 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14915209 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112016003689 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112016003689 Country of ref document: BR Kind code of ref document: A2 Effective date: 20160222 |