WO2020255767A1 - 情報処理システム、情報処理方法、及び記録媒体 - Google Patents
情報処理システム、情報処理方法、及び記録媒体 Download PDFInfo
- Publication number
- WO2020255767A1 WO2020255767A1 PCT/JP2020/022483 JP2020022483W WO2020255767A1 WO 2020255767 A1 WO2020255767 A1 WO 2020255767A1 JP 2020022483 W JP2020022483 W JP 2020022483W WO 2020255767 A1 WO2020255767 A1 WO 2020255767A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- scenario
- content element
- information processing
- information
- Prior art date
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 170
- 238000003672 processing method Methods 0.000 title claims abstract description 9
- 230000004913 activation Effects 0.000 claims description 210
- 238000004891 communication Methods 0.000 claims description 51
- 238000010801 machine learning Methods 0.000 claims description 32
- 238000009826 distribution Methods 0.000 claims description 17
- 230000006399 behavior Effects 0.000 claims description 13
- 230000004044 response Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000002123 temporal effect Effects 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 35
- 238000012545 processing Methods 0.000 description 60
- 238000000034 method Methods 0.000 description 47
- 238000013523 data management Methods 0.000 description 43
- 230000008569 process Effects 0.000 description 33
- 230000006870 function Effects 0.000 description 20
- 238000003860 storage Methods 0.000 description 18
- 230000008451 emotion Effects 0.000 description 11
- 238000001514 detection method Methods 0.000 description 10
- 230000002354 daily effect Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 9
- 230000033001 locomotion Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 230000001419 dependent effect Effects 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 6
- 125000002066 L-histidyl group Chemical group [H]N1C([H])=NC(C([H])([H])[C@](C(=O)[*])([H])N([H])[H])=C1[H] 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 4
- 239000004065 semiconductor Substances 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000006855 networking Effects 0.000 description 3
- 235000019645 odor Nutrition 0.000 description 3
- 230000035807 sensation Effects 0.000 description 3
- 235000019615 sensations Nutrition 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000036387 respiratory rate Effects 0.000 description 2
- 241000556720 Manga Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 240000007594 Oryza sativa Species 0.000 description 1
- 235000007164 Oryza sativa Nutrition 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000036760 body temperature Effects 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 235000009566 rice Nutrition 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/435—Filtering based on additional data, e.g. user or group profiles
- G06F16/436—Filtering based on additional data, e.g. user or group profiles using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/909—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
- G06F16/63—Querying
- G06F16/635—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
- G06F16/68—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/687—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/1822—Parsing for meaning understanding
Definitions
- the present technology relates to an information processing system, an information processing method, and a recording medium, and more particularly to an information processing system, an information processing method, and a recording medium capable of providing a better user experience.
- Patent Document 1 In recent years, with the spread of information devices, various services that take advantage of the characteristics of the devices have been provided (see, for example, Patent Document 1).
- processing may be performed using context information.
- context information As a technique related to the context, the techniques disclosed in Patent Documents 2 to 5 are known.
- This technology was made in view of such a situation, and makes it possible to provide a better user experience.
- context information is associated with content elements in advance, and activation conditions can be set for at least the context information for each user, and the data set of the context information and the activation condition. It is possible to generate a user scenario consisting of, and when the sensor data obtained by sensing the user in real time satisfies the activation condition set in the user scenario, it is associated with the context information corresponding to the activation condition. It is an information processing system including a control unit that controls the created content elements so as to be presented to the user.
- the information processing apparatus can associate context information with the content element in advance, and can set an activation condition for at least the context information for each user. It is possible to generate a user scenario consisting of a data set of activation conditions, and when the sensor data obtained by sensing the user in real time satisfies the activation conditions set in the user scenario, the activation conditions are met.
- This is an information processing method that controls the content element associated with the context information so as to be presented to the user.
- the computer is associated with the content element in advance with context information, and the activation condition can be set for at least the context information for each user, and the context information and the activation condition can be set. It is possible to generate a user scenario consisting of a data set, and when the sensor data obtained by sensing the user in real time satisfies the activation condition set in the user scenario, the context information corresponding to the activation condition is used.
- a computer-readable recording medium that records a program for functioning as a control unit that controls the associated content elements to be presented to the user.
- context information is associated with content elements in advance, and activation conditions are set for at least the context information for each user.
- activation conditions are set for at least the context information for each user.
- a figure which shows another example of the information stored in a scenario DB It is a figure which shows the example of a content element. It is a figure which shows the example of the combination of a content element and a context. It is a figure which shows the example of a scenario. It is a figure which shows the example of the scenario selection / new creation screen. It is a figure which shows the example of the scenario edit screen. It is a figure which shows the 1st example of the geofence edit screen. It is a figure which shows the 2nd example of the geofence edit screen. It is a figure which showed the whole image of the information processing in the 2nd Embodiment. It is a figure which showed the whole image of the information processing in the 3rd Embodiment.
- First Embodiment Basic configuration 2.
- Second embodiment Generation of scenario DB 3.
- Third embodiment Generation of different media 4.
- Fourth embodiment Generation of user scenario DB 5.
- Sixth embodiment Configuration when the activation condition is set to a plurality of context information 7.
- Seventh Embodiment Configuration in which a plurality of devices are interlocked 8.
- Eighth embodiment Configuration in cooperation with another service 9.
- Ninth Embodiment Configuration in which the scenario is shared 10.
- Tenth Embodiment Other Examples of Data 11.
- Eleventh embodiment Configuration using user feedback 12. Modification 13.
- Computer configuration
- FIG. 1 is a representative diagram showing an outline of the present technology.
- This technology provides a better user experience by allowing users living in different locations to use one scenario.
- the creator creates a scenario by adding context information, which is context information, to a content element, which is an element constituting the content, by using an editing device such as a personal computer.
- the scenario created in this way is distributed via a server on the Internet.
- Each user operates a playback device such as a smartphone, selects a desired scenario from the distributed scenarios, and sets the activation condition, which is a condition for presenting the content element, to set the user scenario.
- the activation condition which is a condition for presenting the content element. Create each. That is, in FIG. 1, since two users, user A and user B, set their own activation conditions for the same scenario, the activation conditions of the user scenario are different for each user. ..
- FIG. 2 shows an example of the configuration of an information processing system to which the present technology is applied.
- the information processing system 1 is composed of a data management server 10, an editing device 20, and a playback device 30-1 to 30-N (N: an integer of 1 or more). Further, in the information processing system 1, the data management server 10, the editing device 20, and the playback devices 30-1 to 20-N are connected to each other via the Internet 40.
- the data management server 10 is composed of one or a plurality of servers for managing data such as a database, and is installed in a data center or the like.
- the editing device 20 is composed of information devices such as a personal computer and is managed by a business operator that provides the service.
- the editing device 20 connects to the data management server 10 via the Internet 40, performs editing processing on the data stored in the database, and generates a scenario.
- the playback device 30-1 is composed of information devices such as smartphones, mobile phones, tablet terminals, wearable devices, portable music players, game machines, and personal computers.
- the playback device 30-1 connects to the data management server 10 via the Internet 40, sets activation conditions for the scenario, and generates a user scenario.
- the playback device 30-1 reproduces the content element according to the activation condition based on the user scenario.
- the playback devices 30-2 to 30-N are composed of information devices such as smartphones, like the playback devices 30-1, and reproduce content elements according to the activation conditions based on the generated user scenario.
- the playback device 30 when it is not necessary to distinguish the playback devices 30-1 to 20-N, the playback device 30 is simply referred to.
- FIG. 3 shows an example of the configuration of the data management server 10 of FIG.
- the data management server 10 includes a control unit 100, an input unit 101, an output unit 102, a storage unit 103, and a communication unit 104.
- the control unit 100 is composed of a processor such as a CPU (Central Processing Unit).
- the control unit 100 is a central processing device that controls the operation of each unit and performs various arithmetic processes.
- the input unit 101 is composed of a mouse, a keyboard, physical buttons, and the like.
- the input unit 101 supplies an operation signal according to the user's operation to the control unit 100.
- the output unit 102 is composed of a display, a speaker, and the like.
- the output unit 102 outputs video, audio, and the like according to the control from the control unit 100.
- the storage unit 103 is composed of a large-capacity storage device such as a non-volatile memory, a semiconductor memory including a volatile memory, and an HDD (Hard Disk Drive).
- the storage unit 103 stores various data according to the control from the control unit 100.
- the communication unit 104 is composed of a communication module or the like that supports wireless communication or wired communication conforming to a predetermined standard.
- the communication unit 104 communicates with other devices according to the control from the control unit 100.
- control unit 100 includes a data management unit 111, a data processing unit 112, and a communication control unit 113.
- the data management unit 111 manages various databases and content data stored in the storage unit 103.
- the data processing unit 112 performs data processing related to various types of data. This data processing includes processing related to contents, processing related to machine learning, and the like.
- the communication control unit 113 controls the communication unit 104 to exchange various data with the editing device 20 or the playback device 30 via the Internet 40.
- the configuration of the data management server 10 shown in FIG. 3 is an example, and some components may be removed or other components such as a dedicated image processing unit may be added.
- FIG. 4 shows an example of the configuration of the editing device 20 of FIG.
- the editing device 20 includes a control unit 200, an input unit 201, an output unit 202, a storage unit 203, and a communication unit 204.
- the control unit 200 is composed of a processor such as a CPU.
- the control unit 200 is a central processing device that controls the operation of each unit and performs various arithmetic processes.
- the input unit 201 is composed of an input device such as a mouse 221 and a keyboard 222.
- the input unit 201 supplies an operation signal corresponding to the user's operation to the control unit 200.
- the output unit 202 is composed of an output device such as a display 231 and a speaker 232.
- the output unit 202 outputs information according to various data according to the control from the control unit 200.
- the display 231 displays an image corresponding to the image data from the control unit 200.
- the speaker 232 outputs a voice (sound) corresponding to the voice data from the control unit 200.
- the storage unit 203 is composed of a semiconductor memory such as a non-volatile memory.
- the storage unit 203 stores various data according to the control from the control unit 200.
- the communication unit 204 is composed of a communication module that supports wireless communication or wired communication that conforms to a predetermined standard.
- the communication unit 204 communicates with other devices according to the control from the control unit 200.
- control unit 200 includes an editing processing unit 211, a presentation control unit 212, and a communication control unit 213.
- the editing processing unit 211 performs editing processing related to various data. This editing process includes processing related to a scenario described later.
- the presentation control unit 212 controls the output unit 202 to control the presentation of information such as video and audio according to data such as video data and audio data.
- the communication control unit 213 controls the communication unit 204 and exchanges various data with the data management server 10 via the Internet 40.
- FIG. 4 Note that the configuration of the editing device 20 shown in FIG. 4 is an example, and some components may be removed or other components may be added.
- FIG. 5 shows an example of the configuration of the reproduction device 30 of FIG.
- the playback device 30 includes a control unit 300, an input unit 301, an output unit 302, a storage unit 303, a communication unit 304, a sensor unit 305, a camera unit 306, an output terminal 307, and a power supply unit 308. To.
- the control unit 300 is composed of a processor such as a CPU.
- the control unit 300 is a central processing device that controls the operation of each unit and performs various arithmetic processes.
- the input unit 301 is composed of a physical button 321 and an input device such as a touch panel 322 and a microphone.
- the input unit 301 supplies an operation signal according to the user's operation to the control unit 300.
- the output unit 302 is composed of an output device such as a display 331 and a speaker 332.
- the output unit 302 outputs information according to various data according to the control from the control unit 300.
- the display 331 displays an image corresponding to the image data from the control unit 300.
- the speaker 332 outputs a voice (sound) corresponding to the voice data from the control unit 300.
- the storage unit 303 is composed of a semiconductor memory such as a non-volatile memory.
- the storage unit 303 stores various data according to the control from the control unit 300.
- the communication unit 304 is configured as a communication module compatible with wireless LAN (Local Area Network), cellular communication (for example, LTE-Advanced, 5G, etc.), wireless communication such as Bluetooth (registered trademark), or wired communication. ..
- the communication unit 304 communicates with other devices according to the control from the control unit 300.
- the sensor unit 305 is composed of various sensor devices and the like.
- the sensor unit 305 senses the user and its surroundings, and supplies sensor data according to the sensing result to the control unit 300.
- the sensor unit 305 includes an inertial sensor that measures position, orientation, acceleration, and speed, a biological sensor that measures information such as the heart rate, body temperature, or posture of a living organism, and the magnitude and direction of a magnetic field (magnetic field).
- a magnetic sensor for measuring, a proximity sensor for measuring a proximity object, and the like can be included.
- an acceleration sensor for measuring acceleration or a gyro sensor for measuring angle (attitude), angular velocity, and angular acceleration may be used.
- the camera unit 306 is composed of an optical system, an image sensor, a signal processing circuit, and the like.
- the camera unit 306 supplies the image pickup data obtained by imaging the subject to the control unit 300.
- the output terminal 307 is connected to a device including an electroacoustic conversion device such as earphones and headphones via a cable.
- the output terminal 307 outputs data such as voice data from the control unit 300.
- the device such as earphones may be connected not only by wire but also by wireless communication such as Bluetooth (registered trademark).
- the power supply unit 308 is composed of a battery such as a secondary battery and a power management circuit, and supplies power to each unit including the control unit 300.
- control unit 300 includes a reproduction processing unit 311, a presentation control unit 312, and a communication control unit 313.
- the playback processing unit 311 performs playback processing related to data of various contents. This reproduction process includes (a part of) music, a process of reproducing data such as a character's utterance, and the like.
- the presentation control unit 312 controls the output unit 302 to control the presentation of information such as video and audio according to data such as video data and audio data. In addition, the presentation control unit 312 controls the presentation of the data reproduced by the reproduction processing unit 311.
- the communication control unit 313 controls the communication unit 304 to exchange various data with the data management server 10 via the Internet 40.
- the configuration of the playback device 30 shown in FIG. 5 is an example, and some components such as the camera unit 306 and the output terminal 307 may be removed, or other components such as the input terminal may be added. You may.
- the information processing system 1 is configured as described above. Hereinafter, the specific contents of the information processing executed by the information processing system 1 will be described.
- the storage unit 103 stores each database of the content element-context information DB 151, the scenario DB 152, and the user scenario DB 153.
- the storage unit 103 also stores the data of the content elements.
- Content element-context information DB 151 is a database that stores a table in which the content element and the context information are associated with each other.
- the content element is an element that constitutes the content.
- the content element includes lines, BGM, sound effects, environmental sounds, music, images, etc., which are generated from contents such as video and music.
- the context information is the context information given to the content element.
- the context information given according to the situation in which the content element is expected to be used is associated with the content element and accumulated in the content element-context information DB 151.
- the machine learning technique may be used to automatically add context information to the content element.
- Scenario DB 152 is a database that stores scenarios.
- the scenario is a package of a data set consisting of a combination of content elements and context information (hereinafter, also referred to as "content element-context information”) based on a certain theme.
- scenario DB 152 may store device function information related to the function of the playback device 30. By using this device function information, it is possible to execute processing according to the function of one or more playback devices 30.
- the user scenario DB 153 is a database that stores user scenarios.
- the user scenario is a scenario in which a data set consisting of content elements and context information is packaged, and activation conditions are set.
- a user scenario can be said to be a user-defined scenario.
- the activation condition is a condition for presenting the content element associated with the context information of the data set to the user.
- a spatial condition such as a position or a place, a temporal condition, a user's action, or the like can be set.
- the data management server 10 manages the above database, and the editing device 20 and the playback device 30 access the information stored in the database to perform the processing as shown in FIG. ..
- the playback device 30 senses the user in real time (S101), and it is determined whether or not the sensor data obtained by the sensing satisfies the activation condition set in the user scenario (S102).
- the processes in steps S121 to S127 are mainly the processes when the scenario generation tool is executed by the editing device 20 (control unit 200), and the processes in steps S128 to S133.
- the processing is mainly the processing when the user scenario generation tool is executed by the playback device 30 (control unit 300).
- the scenario generation tool is operated by the creator or the like who creates the scenario on the editing device 20, while the user scenario generation tool is operated by the user or the like who owns the playback device 30.
- the operators are different, or even the same operator has different operation timings.
- the content is acquired by the scenario generation tool (S121), and the candidate content elements are presented (S122). Then, a content element is cut out from the content according to the operation of the creator (S123).
- the content element and context information associated in this way are sent to the data management server 10 and stored in the content element-context information DB 151.
- the scenario generation tool generates a scenario according to the operation of the creator (S126), and the scenario is saved (S127).
- the scenario generated by the scenario generation tool is sent to the data management server 10 and stored in the scenario DB 152.
- the scenarios stored in the scenario DB 152 can be distributed via the Internet 40.
- the scenario distributed from the data management server 10 is acquired by the user scenario generation tool (S128).
- the activation condition is given according to the operation of the user (S129).
- a user scenario corresponding to the user's operation is generated from the scenario, and the user scenario is saved (S130).
- the user scenario generated by the user scenario generation tool is sent to the data management server 10 and stored in the user scenario DB 153.
- the user scenario can be shared with other users and the like.
- the created user scenario can be activated (S132) and evaluated by the user scenario generation tool (S133).
- the details of the scenario generation tool will be described later with reference to FIGS. 14 to 17.
- the details of the user scenario generation tool will be described later with reference to FIGS. 21 to 25 and 26 to 29.
- a data set composed of a combination of content elements and context information is accumulated according to the operation of the user scenario generation tool.
- the context information “home” is associated with the content elements “character utterance # 1” and “BGM # 1”.
- activation conditions given to the data set are accumulated according to the operation of the user scenario generation tool. ing.
- the content elements whose activation conditions are "center (35.631466, 139.743660)" and "radius 10 m” are "character utterance # 1" and "BGM # 1", and the context of "home”. It is given to the information.
- a and b in the center (a, b) mean latitude (north latitude) and longitude (east longitude), and represent the activation range of the content element.
- FIGS. 8 and 9 are examples, and other configurations may be used.
- common context information is given to different works (for example, work A which is “movie”, work B which is “animation”, and work C which is "literary reading”). be able to.
- the context information of "home” is "BGM # 2" of work A, "character utterance # 1" and “BGM # 1" of work B, and "reading # 1" of work C. It is associated with each content element that is.
- context information is associated with content elements in advance
- trigger conditions can be set for at least the context information for each user, and a user scenario including a dataset of context information and trigger conditions. Can be generated. Then, when the sensor data obtained by sensing the user at real timing satisfies the activation condition set in the user scenario, the content element associated with the context information corresponding to the activation condition is presented to the user. Will be done.
- each user can enjoy the world view of the scenario according to the activation conditions in the user scenario, and a better user experience can be provided.
- the contents currently distributed and distributed include formats such as videos such as movies, animations and games, still images such as photographs and paintings and manga, audio such as music and audio books, and texts such as books.
- videos such as movies, animations and games
- still images such as photographs and paintings and manga
- audio such as music and audio books
- texts such as books.
- content with a story is often composed of elements such as lines, effects, and backgrounds.
- the content may be re-edited.
- This content can be re-edited, for example, by cutting out a part of the content in time to fit the spatial and temporal size of the user's current context, or by adding the above elements to fit the context. It is taken out and presented.
- the content element of a certain content includes lines, a background, music, lyrics, a person, a symbol, a character, an object, and the like.
- the expected context information is given to this content element as the above-mentioned context information in the form of text, image, voice, etc. Further, the relationship information itself between the content element and the context information, or a collection of a plurality of relationship information is stored in the scenario DB 152 as a scenario.
- one or more context tags may be attached to one content element, or the same context tag may be attached to a plurality of content elements.
- a combination of dialogue and background music used in a certain scene is used as one audio content, and a text "encounter at an inn" is added as context information.
- the two "content element-context information" data sets shown in FIG. 12 are stored in the content element-context information DB 151.
- an image there is also a method in which a person, a background, an object, etc. are photographed separately and then combined, and content elements can be extracted from the data before composition.
- a dataset of "content element-context information" may be automatically generated from this information, or “content element-context information” may be manually set using this information as reference information. May be good.
- a scenario is a dataset of one or more "content elements-context information" that follows a certain theme, such as the title of the work from which it was re-edited, the characters that appear, the stage that was set, and the emotions that are aroused. It is composed by summarizing and is accumulated in the scenario DB 152.
- the data sets of the two "content elements-context information" shown in FIG. 12 can be stored in the scenario DB 152 as a scenario of the "city of departure".
- the user not only searches and obtains the dataset of "content element-context information” that he / she wants to use, but also searches the dataset of multiple "content element-context information” packaged based on the scenario. ⁇ You can also get it.
- This scenario generation tool is executed by the control unit 200 of the editing device 20 operated by the creator or the like, and various screens are displayed on the display 231.
- This scenario selection / new creation screen includes a map / scenario display area 251, a scenario list 252, and a new scenario creation button 253.
- the name is written on the pin 261A indicating the position on the map in the map / scenario display area 251, or the scenario display banner 262A is displayed as a list in a predetermined order such as the name order in the scenario list 252.
- the new scenario creation button 253 is operated when creating a new scenario.
- the creator can select the desired scenario by clicking the pin 261A on the map corresponding to the desired area or the scenario display banner 262A of the scenario list 252.
- the scenario editing screen of FIG. 15 includes a map / geofence display area 254, a geofence list 255, and an editing tool display area 256.
- the name of the geofence is written in the geofence areas 271A to 271E representing the area of the geofence on the map in the map / geofence display area 254, or the geofence display banner 272A is in the order of the name in the geofence list 255. It is displayed as a list in the predetermined order of.
- the shape of the geofence areas 271A to 271E various shapes such as a circle and a polygon can be set.
- the context information given to the activation condition (activation range) for which the default value is set is displayed in text or the like in each geofence, or when a desired geofence is selected. It is displayed in a balloon shape. Based on this display, the creator can confirm the context information associated with the activation range of each content element.
- the creator can select the desired geofence by clicking the geofence areas 271A to 271E on the map corresponding to the desired area and the geofence display banner 272A of the geofence list 255.
- the editing tool display area 256 includes a circular geofence creation button 273, a polygonal geofence creation button 274, a geofence move button 275, an overwrite save button 276, a new save button 277, a delete button 278, and a back button 279.
- the circular geofence creation button 273 is operated when creating a geofence having a circular shape.
- the polygon geofence creation button 274 is operated when creating a geofence having a polygonal shape.
- the geofence move button 275 is operated when moving the desired geofence.
- the overwrite save button 276 is operated when the scenario to be edited is overwritten and saved in the existing scenario.
- the new save button 277 is operated when saving the scenario to be edited as a new scenario.
- the delete button 278 is operated when deleting the scenario to be edited.
- the back button 279 is operated when returning to the scenario selection / new creation screen.
- the geofence area 271C which is the “geofence # 1” is selected.
- the corresponding geofence name may be displayed in a balloon, and the content element set in the geofence may be reproduced.
- the geofence editing screen of FIG. 16 includes the geofence detailed setting area 257.
- the geofence detailed setting area 257 is a geofence name, center position, radius, playback time, weather, content element, playback range, volume, repeat playback, fade-in / out, and playback priority as detailed setting items of the geofence. Including level.
- the geofence name corresponds to the context setting item.
- the center position, radius, reproduction time, and weather correspond to the setting items of the activation condition, and the default values thereof are set here.
- the content element, the reproduction range, the volume, the repeat reproduction, the fade-in / out, and the reproduction priority level correspond to the setting items of the content element and the reproduction condition, and the default values thereof are set here.
- geofence name input field 281A "geofence # 1" is entered as the geofence name.
- the content element input field 281F "http: xxx.com/sound/folder#1/01.mp3" is input as the default value of the content element.
- the content element selection screen 283 displayed by clicking the selection button 282 can be used.
- the audio file data of the content element stored in the storage unit 103 of the data management server 10 is displayed.
- a desired folder from the folders displayed in a hierarchical structure on the content element selection screen 283, a desired audio file in the folder can be selected.
- the search process may be performed using the desired keyword input in the search keyword input field 284A as the search condition, and a list of desired audio files according to the search result may be presented.
- playback range input field 281G and the volume input field 281H "00:00:08-00:01:35" and "5" are input as default values for the playback range and volume.
- the playback time and volume may be automatically input according to the content element.
- repeat playback input field 281I and the fade-in / out input field 281J “repeat playback: do” and “fade in / out: do” are input as default values for repeat playback and fade-in and fade-out of the audio file. There is.
- the reproduction priority level is a predetermined stage such as 3 stages of "1" to “3” and 5 stages of "1" to "5". The lower the numerical value, the higher the priority and the higher the numerical value. The lower the priority, the lower the priority.
- the geofence editing screen of FIG. 16 shows a case where the shape of geofence # 1 is circular, but when the shape is polygonal (rectangular), the geofence editing screen of FIG. 17 is displayed. Will be done.
- the geofence editing screen of FIG. 17 has a rectangular geofence apex position as a setting item of activation conditions instead of the center position and radius of the circular geofence. The point that it is provided is different.
- a vertex position input field 291B composed of a list box is provided.
- a combination of a plurality of latitudes and longitudes such as latitude # 1 and longitude # 1, latitude # 2 and longitude # 2, latitude # 3 and longitude # 3, ... Is displayed as a list, and the desired latitude and longitude combination selected from the list is set as the default value for the apex position of the rectangular geofence.
- the user interface of the scenario generation tool described above is an example, and other user interfaces may be used, such as using other widgets instead of text boxes and radio buttons.
- the geofence edit screen instead of the text box that constitutes the playback time input field 281D, the weather input field 281E, the volume input field 281H, or the playback priority level input field 281K, or the list box that constitutes the apex position input field 291B. You can use drop-down lists, combo boxes, etc.
- the information processing shown in FIG. 18 is realized by at least linking the data management server 10 (control unit 100) and the editing device 20 (control unit 200) in the information processing system 1. That is, this information processing is executed by at least one of the control unit 100 and the control unit 200.
- context information for example, "get courage”
- content element for example, "character dialogue”
- the data set of one or more "content elements-context information” is accumulated in the scenario DB 152 as a scenario (for example, "city of departure") (S204).
- the data set can be packaged based on a certain theme (the title of the work that was the source of the re-editing, the set stage, the emotions that are aroused, etc.) and stored in the scenario DB 152 ( S211).
- the content element for example, a part (a part of the music, etc.) of the streaming distribution content (the music, etc. distributed by the music streaming distribution service) can be included.
- the content ID and the playback range of the content are specified (S221), and the information indicating the content ID and the playback range is associated with the target context information.
- Content element- may be stored in the context information DB 151.
- the introduction content (other content element) such as a character may be generated for the content element (S231), and the introduction content may be presented before the content element is reproduced.
- an introductory text can be presented by a specific audio character (for example, a character of a disc jockey (DJ)) corresponding to context information.
- DJ disc jockey
- the context information can be automatically added to the new content element.
- N Neural Network
- various methods such as a neural network (NN: Neural Network) can be used, and for example, it is included in a certain scene from image information or audio information included in a moving image or a still image.
- NN Neural Network
- the range of content elements is determined, and one or more context information assumed from the identification result or a combination thereof is automatically generated. can do.
- the extracted text itself is used as a content element, for example, as a character image. It is possible to display it on a public display or a display device such as AR glass, but voice (sound) may be used.
- the AR glass is a spectacle-type device (device) compatible with augmented reality (AR).
- data such as audio data and image data with related impressions (images) are searched or synthesized from texts that compose words and sentences, and the data is used as content elements. You may.
- the text is used as a content element by searching or synthesizing the text that composes related words and sentences using machine learning technology. You may. That is, here, it is possible to add content that is not included in the existing content, or to add another modal expression that is not included in the original content, such as tactile sensation.
- TTS technology is an example of voice synthesis technology that artificially creates human voice, and voice may be generated using other technology. Alternatively, a recorded reading by a person may be used. Further, in the above description, the case where the machine learning technique is used is shown, but the data as the content element may be generated by separately analyzing the acquired data.
- the information processing shown in FIG. 19 is realized by at least linking the data management server 10 (control unit 100) and the editing device 20 (control unit 200) in the information processing system 1.
- one or more content elements for example, texts, etc.
- a first medium for example, texts, etc.
- contents e-book novels, etc.
- a content element for example, a sound corresponding to a sentence of a novel
- a second medium for example, TTS audio or the like
- context information for example, context information in which the voice of one sentence of the novel is assumed to be heard
- content element for example, voice corresponding to one sentence of the novel
- one or more "content element-context information" data sets are stored (stored) in the scenario DB 152 as a scenario (S304).
- the relationship between the first media (text, etc.) and the second media (TTS voice, etc.) is machine-learned in advance (S311), and the first media is based on the result of the machine learning.
- the content element of the second media can be generated from the content element of.
- the third embodiment has been described above.
- the user can acquire a desired scenario and a data set of desired "content element-context information" with the playback device 30 owned by the user.
- a plurality of "content element-context information" data sets included in the acquired scenario are displayed and arranged in the actual space around the user.
- a trigger condition consisting of a combination of sensingable conditions can be set for each "content element-context information" data set using the user interface for.
- This activation condition includes, for example, information on GPS (Global Positioning System), position information such as latitude and longitude estimated from information from a wireless LAN (Local Area Network) access point, wireless beacons, and short-range wireless communication. It is possible to include usage status and authentication information obtained from the history of.
- GPS Global Positioning System
- position information such as latitude and longitude estimated from information from a wireless LAN (Local Area Network) access point
- wireless beacons Wireless LAN (Local Area Network) access point
- short-range wireless communication It is possible to include usage status and authentication information obtained from the history of.
- activation conditions for example, the user position, posture, behavior estimated from the image captured by the camera, information on the surrounding environment, information on the time and time measured by the environmental information clock, and voice information obtained from the microphone. It includes environmental information and authentication information based on the above, information on body posture and movement obtained from the inertial sensor, information on riding condition, etc., and information on respiratory rate, pulse, emotion, etc. estimated from biological signal information.
- the setting of this activation condition can be set using the user scenario generation tool, but it can also be completed before using the service, or the tool is started and set while using the service. You may do so.
- the data set of "content element-context information" is displayed on the map, and the interface placed on the map by the user is used to display the sensingable activation condition on the map.
- the case of setting the range and the time zone will be described.
- the user can create a desired user scenario by operating a user scenario generation tool executed by, for example, a playback device 30 such as a smartphone or an information device such as a personal computer.
- a user scenario generation tool executed by, for example, a playback device 30 such as a smartphone or an information device such as a personal computer.
- the user scenario generation tool may be provided as a native application, or may be provided as a Web application using a browser.
- Example of UI of user scenario generation tool the user interface of the user scenario generation tool executed by the playback device 30 such as a smartphone will be described with reference to FIGS. 21 to 25.
- This user scenario generation tool is executed by, for example, the control unit 300 of the playback device 30 operated by the user, and various screens are displayed on the display 331.
- This scenario selection / playback screen shown in FIG. 21 is displayed.
- This scenario selection / playback screen includes a map / scenario display area 411, a scenario list 412, and a new scenario creation button 413.
- the scenarios are displayed as a list in the map / scenario display area 411 with the name written on the pin 411A indicating the position on the map, or in the scenario list 412 in the order of the name or the shortest distance from the current location. Will be done.
- the new scenario creation button 413 may be tapped. Further, on the scenario selection / playback screen, a search process may be performed using a desired keyword input in the search keyword input field 414 as a search condition, and a scenario corresponding to the search result may be presented.
- the user can select a desired scenario by tapping the pin 411A on the map corresponding to the desired area or the scenario display banner 412A of the scenario list 412.
- scenario # 1 is considered to be playing, and scenario # 2 and scenario # 3 are considered to be stopped.
- scenario # 2 and scenario # 3 are considered to be stopped.
- only the three scenario display banners 412A are displayed, but other scenarios may be displayed by flicking the screen to scroll.
- the pin 411B is focused on among the plurality of pins 411A in the map / scenario display area 411, the pin 411B is in the selected state, so the scenario name corresponding to the pin 411B which is the “scenario # 1”. Is displayed in a balloon shape. Then, when the edit button 412B is tapped while the scenario # 1 corresponding to the pin 411B is selected, the activation condition setting screen of FIG. 22 is displayed as the scenario edit screen.
- the activation condition setting screen of FIG. 22 includes a map / geofence display area 421, an overwrite save button 422, a new save button 423, a delete button 424, and a back button 425.
- geofence areas 421A to 421E are displayed on a map of a desired area.
- shape of the geofence regions 421A to 421E various shapes such as a circle and a polygon can be set.
- the context information given to the activation condition is displayed in text or the like in each geofence, or is displayed in a balloon shape when the desired geofence is tapped. Will be done. Based on this display, the user can confirm the context information associated with the activation range of each content element.
- Geofence can be moved on the screen.
- the geofence area 421C if attention is paid to the geofence area 421C with the pattern among the geofence areas 421A to 421E, since it is in the selected state, the geofence area 421C corresponding to the “geofence # 1”
- the offense name is displayed in a balloon.
- the user moves the geofence area 421C in the diagonally lower right direction (the direction of the arrow in the figure) to move the position.
- the area of the geofence area 421C can be enlarged or reduced by performing a pinch-out operation or a pinch-in operation with the geofence area 421C selected, or the area of the geofence area 421C can be enlarged or reduced according to a predetermined operation.
- the shape of the offense region 421C may be modified.
- the activation condition detailed setting screen of FIG. 23 is displayed.
- the activation condition detailed setting screen of FIG. 23 includes a geofence detailed setting area 431, a save button 432, and a back button 433.
- the geofence detailed setting area 431 includes a geofence name input field 431A, a center position input field 431B, a radius input field 431C, a playback time input field 431D, a weather input field 431E, a content element input field 431F, a playback range input field 431G, and a volume. It includes an input field 431H, a repeat playback input field 431I, a fade-in / out input field 431J, and a playback priority level input field 431K.
- the geofence name input field 431A to the playback priority level input field 431K correspond to the geofence name input field 281A to the playback priority level input field 281K in FIG. 16, and the value set as the default value is displayed as it is. There is.
- the save button 432 is operated when saving the setting contents of Geofence # 1. Further, the back button 433 is operated when returning to the activation condition setting screen.
- the user may use the setting contents of the default value of Geofence # 1 as it is, or may change it to a desired setting content. For example, when the content element input field 431F is tapped, the content element selection screen of FIG. 24 is displayed.
- the content element selection screen of FIG. 24 includes a content element display area 441, a selection button 442, and a back button 443.
- icons 441A to 441F corresponding to each content element are arranged in a tile shape in 3 rows and 2 columns.
- the selection button 442 is operated when a desired icon is selected from the icons 441A to 441F. Further, the back button 443 is operated when returning to the activation condition detailed setting screen.
- the content element # 1 is played.
- the content element editing screen of FIG. 25 includes a content reproduction partial display area 451, a content reproduction operation area 452, a song change button 453, and a back button 454.
- the waveform of the music of the content element # 1 is displayed in order to edit the content element # 1 as a music, and the part to be reproduced is specified by sliding the sliders 451a and 451b left and right. can do.
- the waveform of the music in the cut selection area 451B corresponding to the area outside the sliders 451a and 451b is regarded as the waveform to be non-reproduced, and the waveform inside the sliders 451a and 451b.
- the waveform of the music in the reproduction selection area 451A corresponding to the area of is set as the waveform to be reproduced.
- the seek bar 451c indicates the reproduction position of the music of the content element # 1 being reproduced.
- a play button, a stop button, a skip button, and the like are displayed as buttons for operating the music of the content element # 1.
- the user wants to play the music of the content element # 1 by operating the buttons, sliders 451a, 451b, etc. in the content playback operation area 452 while checking the waveform of the music in the content playback portion display area 451. Only the part can be cut out.
- the song change button 453 is operated when changing the song to be edited. Further, the back button 454 is operated when returning to the activation condition detailed setting screen.
- the user can create a desired user scenario by operating the user scenario generation tool executed by the playback device 30 such as a smartphone.
- This scenario selection screen includes a map / scenario display area 471 and a scenario list 472.
- the name is written on the pin 471A indicating the position on the map in the map / scenario display area 471, or the scenario display banner 472A is displayed as a list in the scenario list 472 in a predetermined order.
- the user can select a desired scenario by clicking the pin 471A on the desired map or the scenario display banner 472A in the scenario list 472.
- This activation condition setting screen includes a map / geofence display area 481 and a context list 482.
- the geofence area 481A indicating the activation range of the content element is displayed.
- the geofence area 481A is represented by a plurality of preset shapes such as a circle or a polygon.
- the context information given to the activation condition is displayed in text or the like in the geofence area 481A, or when the desired geofence area 481A is clicked. It is displayed in a balloon shape.
- the geofence area 481A can be moved on the screen according to a drag operation.
- the geofence area 481B is moved diagonally upward to the right (in the direction of the arrow in FIG. 28) by a drag operation. It can be moved from the position shown in FIG. 27 to the position shown in FIG. 28.
- the shape of the geofence area 481B can be transformed into a desired shape by moving the cursor to the white circle ( ⁇ ) on the thick line indicating the shape of the geofence area 481B and dragging in the desired direction. ..
- the user sets by himself / herself which position in the real life space the context corresponds to by moving or transforming the geofence area 481B based on the context information displayed in the geofence area 481B. can do.
- content elements may be presented in the form of a separate list.
- unused content elements may be deleted, or separately obtained content elements may be added to the scenario currently being edited.
- the geofence edit screen of FIG. 29 is displayed. Is displayed.
- This geofence edit screen includes a geofence detailed setting area 491, a select button 492, an update button 493, a delete button 494, and a cancel button 495.
- the geofence detailed setting area 491 includes a geofence name input field 491A, a content element input field 491B, a repeat playback input field 491C, a fade-in / out input field 491D, a playback range input field 491E, and a volume input field 491F. These setting items correspond to the setting items of the geofence detailed setting area 431 of FIG. 23.
- a desired content element can be selected by using the content element selection screen as in the selection button 282 of FIG.
- the update button 493 is operated when updating the setting item of the geofence area 481B.
- the delete button 494 is operated when deleting the geofence area 481B.
- the cancel button 495 is operated when canceling the edit.
- the user can create a desired user scenario by operating the user scenario generation tool executed by the information device such as a personal computer.
- a user interface using a map is illustrated as a user scenario generation tool, but another user interface that does not use a map may be used.
- a method for setting activation conditions without using a map will be described.
- the camera unit 306 of the playback device 30 such as a smartphone is used for the purpose.
- Settings can be made by shooting the bench.
- a voice command such as "shoot here” or “set on this bench” to shoot the target bench. It can also be set with.
- the user can take a picture including his / her hand using a camera such as eyewear, he / she performs a hand gesture in the form of surrounding the bench, and when the gesture is recognized, the object and the scenery in the fence are recorded. It can be set by doing.
- an activation condition that cannot be set by a map expression such as a user's biological state or emotion
- a "current feeling” button is displayed on a playback device 30 such as a smartphone, and the button is tapped.
- the data and the recognition result at the time of the click operation or at a certain time before and after that are recorded and can be set as the activation condition.
- the "current status" button is displayed, or it is set in advance as a voice command or a specific gesture, and when there is an input to the button, , Data such as a position, time, weather, surrounding objects, weather, biological data, and emotions specified in advance may be acquired in a batch.
- the data input by the user without using the screen is transmitted to, for example, the data management server 10 and stored in the user scenario DB 153.
- the user can display the screen of the user scenario generation tool on the playback device 30 owned by the user. Then, the user can confirm the association between the activation condition displayed on this screen and the data set of "content element-context information", and can re-edit.
- the above operation is an operation of setting only the activation condition for the content element in the scenario provided by the user, but depending on the usage condition, the content of the content such as audio data and image data constituting the content element, or The context information given to the content element may be allowed as an operation that can be changed by the user.
- the edited scenario is stored in the user scenario DB 153 as a user scenario.
- the user scenarios accumulated in the user scenario DB 153 can also be disclosed to other users by using a sharing means such as a social networking service (SNS: Social Networking Service).
- SNS Social Networking Service
- a data set of multiple "content elements-context information" included in the scenario is displayed in an editing means such as a user scenario generation tool, and the user can display the actual position and time zone of his / her living space, the environment, and his / her own.
- an editing means such as a user scenario generation tool
- FIG. 30 shows an example of setting a user scenario.
- two users set trigger conditions A and B for the delivered scenario, and each creates their own user scenario.
- the user A sets the activation condition A and the user B sets the activation condition B, so that the activation condition is different for each user.
- the same scenario can be implemented in different places for each user. That is, one scenario can be used by users living in different places.
- Another example of service is the cooperation with streaming distribution service.
- audio data of a plurality of works in an existing music format is grouped together based on a certain theme such as for each creator (creator) or for each usage scene.
- a certain theme such as for each creator (creator) or for each usage scene.
- the work itself or a part of the work expressing a specific context is extracted and used as a content element, and a music is played for the content element (for example, a station at dusk).
- Context information indicating a state for example, a tired return path
- the scenario is collectively stored in the scenario DB 152 so that it can be distributed.
- the user acquires the above scenario by the playback device 30, and refers to the specific "content element-context information" data set included in the data set in his / her own living area while referring to the given context information.
- a user scenario can be created and registered in the user scenario DB 153 by arranging the user scenarios at various positions and time zones.
- the user can also specify a part of the work itself that he / she wants to play as a content element by designating it as a playback range.
- the scenario may include a content element (another content element) as a voice character that describes the work to be reproduced during or during the reproduction of the content element.
- this voice character can be acquired not only by the same route as the scenario but also by a route different from the scenario.
- a character preferred by the user explain from a plurality of voice characters. it can.
- scenario DB 152 a combination of context information for various content elements is accumulated for the purpose of providing to the user by the creator.
- this context information when this context information is used as teacher data and a recognizer that machine-learns the melody structure of a content element is used, the context that is easily recalled from the melody structure of a certain content element is reflected in the subjective tendency of the creator. Can be estimated. Then, using this estimation result, it is possible to automate the process of giving context information to the content element, or to support the giving of context information of the creator by presenting multiple contexts having a certain correlation. it can.
- a data set of "content element-context information" associated with an activation condition consisting of the position, time, environment, physical condition, emotion, etc. of one's own living space is sequentially accumulated by the user. There is.
- the user scenario DB 153 stores a large number of data sets of "content element-context information" for which activation conditions are set by a plurality of users, machine learning or analysis of the stored information is performed. With, you can create algorithms and recognizers that automate the process.
- the information processing shown in FIGS. 31 and 32 is realized by at least linking the data management server 10 (control unit 100) and the playback device 30 (control unit 300) in the information processing system 1. That is, this information processing is executed by at least one of the control unit 100 and the control unit 300.
- context information is added to each content element, and one or more "content element-context information" data sets are stored in the scenario DB 152 as a scenario (S401). ..
- activation conditions are set according to the sensor data obtained by sensing the user for each context information given to the content element (S402).
- a user scenario consisting of context information and a data set of user-specific activation conditions is generated (S403) and stored in the user scenario DB 153 (S404).
- the activation condition it is possible to set the activation condition according to the captured image data, the characteristic operation data, and the like.
- the image data includes image data that is assumed to be visually recognized by the user.
- the characteristic operation data includes, for example, operation data of a button (current feeling button) for registering information according to the user's current emotion.
- the information processing system 1 provides a user scenario generation tool as a user interface using a map for setting a user-specific activation condition.
- this user scenario generation tool is provided as an application executed by a playback device 30 such as a smartphone or an information device such as a personal computer.
- activation conditions are set for each context information given to the content element extracted from the content (S401, S402).
- a data set of content elements and context information is presented on a map of a desired area (S441), and as an activation condition for the context information, a map of the desired area is displayed.
- S441 a map of a desired area
- S442 an interface capable of setting a predetermined area
- the fourth embodiment has been described above.
- the user's position, physical condition, and emotion are used as sensor data by the sensing means mounted on the playback device 30 possessed or worn by the user or the device (device) arranged around the user.
- Motion information on objects, structures, buildings, products, people, animals, etc. in the surrounding environment, and data such as the current time are sequentially acquired.
- the content element included in the "content element-context information" data set associated with the activation condition is specified in advance. It is reproduced from a device (for example, a playback device 30) or a combination of a plurality of devices (for example, a playback device 30 and devices arranged in the vicinity).
- the reproduction location and timing are determined by comparing the sensor data by the sensing means with the activation condition, the determination process is based on subjective elements such as context and data including subjective elements. Since the recognition device by machine learning is not directly included, stable operation with reproducibility as a system is possible.
- FIG. 33 shows an example of a combination of the activation condition and the sensing means.
- the time and time can be set, and it is possible to measure and judge using a clock or timer.
- spatial activation conditions it is possible to set positions such as latitude, longitude, and approach to a specific position, and it is possible to make a judgment by measuring using GPS, Wi-Fi (registered trademark), wireless beacon, etc. It is possible.
- authentication information such as a user ID may be set as an activation condition, and it is possible to measure and judge using proximity communication such as Bluetooth (registered trademark).
- proximity communication such as Bluetooth (registered trademark).
- the user's posture such as standing, sitting, and sleeping, and the user's behavior such as train, bicycle, and escalator may be set as activation conditions, and the judgment is made by measuring using an inertia sensor, a camera, proximity communication, or the like. It is possible to do.
- surrounding environment information such as chairs, desks, trees, buildings and rooms, scenery and scenes may be set as activation conditions, and measurements are made using cameras, RF tags, wireless beacons, ultrasonic waves, etc. Is possible.
- states such as body posture, movement, respiratory rate, pulse, and emotion may be set as activation conditions, and measurement and determination can be made using an inertial sensor, a biological sensor, or the like.
- the fifth embodiment has been described above.
- the activation conditions set for two or more content elements included in at least one or more scenarios are the same.
- two or more activation ranges are set in duplicate so as to include positions on the same map. There is.
- the geofence 661 set as a circular activation range and the geofences 662A to 662E set as a circular activation range inside the circle are For example, when they are superimposed.
- the content element can be appropriately reproduced.
- the content element is the reading of a sentence by TTS voice
- the utterance (line) by the character A is set as the activation condition A including the activation range of the entire area including the home, etc.
- the activation condition B including the activation range a case where the utterance (speech) by the character B is specified in the presentation range setting user scenario is illustrated.
- the lower layer L1 corresponds to the user scenario
- the upper layer L2 corresponds to the presentation range setting user scenario.
- the elliptical region corresponds to the activation range set by the geofence.
- the character B makes an utterance when the activation condition C1 of the user scenario is satisfied, and the character A makes an utterance when the activation condition C2 is satisfied. Make an utterance. That is, in this case, there is always one character.
- the character A or B makes an utterance when the activation condition C1 of the user scenario is satisfied. Whether character A or B speaks may be randomly determined, or specific rules may be set. Further, when the activation condition C2 is satisfied, only the character A speaks. That is, in this case, when the user is at home, there are two characters.
- the priority to be set can be set based on the sensor data. For example, when a plurality of content elements are utterances (lines) by a plurality of characters and the user's position is a position where the activation conditions of the plurality of content elements overlap, all the corresponding content elements can be played. Imagine when you are in.
- the position of the user 600 and the specific position of the activation range of the content element according to the geofences 672A to 672C when the user 600 is wearing the stereo earphone connected to the playback device 30, the position of the user 600 and the specific position of the activation range of the content element according to the geofences 672A to 672C.
- the fixed position of the reproduced sound source for example, dialogue
- the presentation of the sound source (for example, dialogue) by the desired character is directed to the body, head, or the like of the user 600. It becomes possible to select according to.
- the volume of the sound source by the character may be changed according to the position of the user 600 on the Geofence 672A.
- the volume of the sound source can be increased as the user 600 approaches the specific position 671A, while the volume of the sound source can be decreased as the user 600 moves away from the specific position 671A.
- the user scenario for setting the presentation range may be referred to here as well.
- the presentation range setting user scenario is provided with information for setting the activation range and information for designating the sound source setting positions P1 to P4 for each activation condition C1 to C4. To do so.
- the sound source setting positions P1 to P4 are not limited to the positions within the activation range in which the activation conditions C1 to C4 are specified.
- FIG. 38 four activation conditions C1 to C4 having a common activation condition region CA (diagonal lines in the figure) are shown, and sound source setting positions P1 to P4 (in the figure) are shown in the respective activation conditions C1 to C4. Black circle) is set.
- the sound source setting position is searched for all the activation conditions for which the condition is satisfied.
- the sound source setting position P2 in the viewing angle region VA calculated from the user's orientation information measured by the sensor unit 305 of the playback device 30 possessed by the user 600. Is identified. Then, the content element associated with the activation condition C2 having the specified sound source setting position P2 is reproduced.
- control is an example of control when two or more activation ranges are set so as to include positions on the same map, and other controls may be performed.
- one content element is used as the background sound and the other content elements are used as a plurality of lines, so that the user moves within the activation range as the user moves. It is possible to present an expression in which multiple lines are reproduced in the same BGM.
- control is not limited to the presentation of voice (sound), and the presentation of a character image through a display device such as an augmented reality (AR) -compatible eyeglass-type device can be similarly controlled. .. Therefore, next, a case where the arrangement of a plurality of characters can be set for the scenario will be described with reference to FIGS. 39 to 45.
- AR augmented reality
- FIG. 39 shows an example of the configuration of the information processing system 1 in the case where the arrangement of a plurality of characters can be set.
- FIG. 39 among the devices constituting the information processing system 1 of FIG. 2, the data management server 10 and the playback device 30 are shown. However, some of the processes executed by the data management server 10 may be executed by another device such as the editing device 20 or the playback device 30.
- control unit 300 includes a user position detection unit 341, a user direction detection unit 342, a voice recognition intention understanding unit 343, and a content reproduction unit 344.
- the user position detection unit 341 detects the user's position based on information related to GPS and the like.
- the user direction detection unit 342 detects the direction in which the user is facing based on the sensor data from the sensor unit 305 (FIG. 5).
- the voice recognition intention understanding unit 343 performs voice recognition / intention understanding processing based on the voice data of the user's utterance, and understands the user's utterance intention.
- this voice recognition / intention understanding process is not limited to the control unit 300, and a server on the Internet 40 may perform a part or all of the process.
- the voice data of the user's utterance is picked up by the microphone.
- the transmission data processed by the user position detection unit 341, the user direction detection unit 342, and the voice recognition intention understanding unit 343 is transmitted to the data management server 10 via the Internet 40 by the communication unit 304 (FIG. 5). Further, the communication unit 304 receives the response data transmitted from the data management server 10 via the Internet 40.
- the content reproduction unit 344 reproduces the content element based on the received response data.
- the content element not only the utterance (line) by the character can be output from the speaker 332, but also the image of the character can be displayed on the display 331.
- control unit 100 further includes an instruction character selection unit 131, a scenario processing unit 132, and a response generation unit 133. Further, the storage unit 103 (FIG. 3) further stores the character arrangement DB 161, the position-dependent information DB 162, and the scenario DB 163.
- the communication unit 104 receives the transmission data transmitted from the playback device 30.
- the instruction character selection unit 131 selects an instruction character by referring to the character arrangement DB 161 based on the received transmission data, and supplies the selection result to the scenario processing unit 132.
- the character arrangement DB 161 As shown in FIG. 40, in the character arrangement DB 161, an arbitrary system and an arrangement location corresponding to the system are set for each character.
- the scenario processing unit 132 processes the scenario by referring to the position-dependent information DB 162 and the scenario DB 163 based on the selection result from the instruction character selection unit 131, and supplies the processing result to the response generation unit 133.
- the position-dependent information DB 162 relates to the type information, the position information such as latitude and longitude, and the contents associated with the type information and the position information for each information ID having a unique value. Information is set.
- the type information and the information related to the content associated with the type information are set for each scenario ID having a unique value.
- the information related to the character and the content corresponds to the content element
- the system and type information correspond to the context information
- the position information corresponds to the activation condition. It can be said that.
- the response generation unit 133 generates response data based on the processing result from the scenario processing unit 132. This response data is transmitted to the playback device 30 via the Internet 40 by the communication unit 104 (FIG. 3).
- the user can set a plurality of desired voice characters in the scenario, and detects the position and the direction facing the user with respect to the activation condition indicating the trigger of voice reproduction. Then, the voice character can be switched according to the detection result.
- the information processing system 1 when providing the voice character service, it is possible to detect the position and direction of the user and switch the voice character according to the detection result, so that the voice character is divided into roles. It is possible to instruct the desired operation. Therefore, it becomes easy to give instructions to a plurality of voice characters.
- the user 900 simply gives instructions to the characters 700A to 700C in the virtual space, and each of the characters 700A to 700C operates according to the instructions given to them. Will be done.
- the user 600 simply asks a question by voice in the direction in which the character 700C exists in the virtual space, and the answer to the question can be obtained from the character 700C. That is, the character 700C can identify the information around the arranged position, so to speak, the user can obtain the access right to the surrounding information by the existence of the character 700C.
- a user scenario in which voice characters talk to each other can be realized, and processing that does not cause conversation may be added by exclusive processing.
- the environment information around the activation range indicated by the activation condition included in the user scenario may be acquired, and the voice may be provided to the user by the voice character specified in the activation range.
- the position of the character in the user coordinate system can be specified.
- the position of the character in the world coordinate system can be specified (designation of latitude / longitude or landmark, etc.), or the position of the character can be specified in a device such as a playback device 30 capable of displaying the character.
- the information processing shown in FIG. 46 is realized by at least linking the data management server 10 (control unit 100) and the playback device 30 (control unit 300) in the information processing system 1.
- the information processing system 1 acquires sensor data by real-time sensing (S601). It is determined whether or not the information obtained from the sensor data satisfies the activation condition of the user scenario stored in the user scenario DB 153 (S602).
- step S602 If it is determined in the determination process of step S602 that the activation condition is satisfied, it is further determined whether or not there is only one condition that satisfies the activation condition (S603).
- step S603 When it is determined in the determination process of step S603 that there is only one condition, the content element corresponding to the context information satisfying the activation condition is presented (S604).
- step S603 when it is determined in the determination process of step S603 that there are a plurality of conditions, a rule for determining the order of the content elements to be presented is referred to (S605), and according to the rule, the context information satisfying the corresponding activation condition is used. The corresponding content element is presented (S604).
- the order of the content elements to be presented can be determined from a plurality of content elements according to the orientation of the user estimated from the sensor data (S611, S605).
- only the content elements having a specific orientation may be presented according to the orientation of the user estimated from the sensor data (S621). Further, as shown in FIG. 35, only the content elements set at a specific position may be presented according to the position of the user estimated from the sensor data (S631).
- the content element corresponding to the first character is specified and presented to the user
- the content element corresponds to the second character.
- Content elements can be identified and presented to the user.
- the content element playback device 30 may be a single device or a plurality of devices may operate in conjunction with each other.
- the playback device 30 is a single device, for example, it is assumed that audio is reproduced from a stereo earphone worn by the user outdoors.
- the environmental sound around the user can be superimposed on the content element and presented at the same time, the sense of consistency and fusion between the provided content element and the real world around the user can be further enhanced.
- a means for providing the environmental sound around the user for example, an open type earphone that can directly propagate the ambient sound to the ear, or a closed type environmental sound acquired by a sound collecting function such as a microphone is superimposed as voice data. There is a way to do it.
- one device may be assigned to one content element, or a plurality of devices may be assigned to one content element.
- three speakers are arranged around the user, one is the dialogue of the character, the other is the noise of the cafe, and the other one is assigned to the background music and played to present a three-dimensional acoustic environment. Can be done.
- the voice of the voice character may be localized at a specific position, and the appearance of the voice character may be presented on the peripheral display corresponding to that position.
- This appearance presentation service may be provided as a paid service.
- the dialogue of the character A is reproduced by detecting the speaker installed at the closest position among the three speakers, and is made to follow so as to be reproduced from the closest speaker according to the movement of the user. be able to.
- the device has a means for grasping the position of the device and the position of the user or the position of another device.
- a camera having a function of communicating the blinking code of an LED is installed in each pixel installed indoors, and each playback device is encoded and emitted by at least one LED.
- LED Light Emitting Diode
- the functions that can be reproduced by the playback device 30 are registered in advance in a dedicated database such as the device function information DB or the scenario DB 152 as device function information.
- the device function describes a playback function that can be realized by a device having one ID, and one function is assigned to one device such as "voice playback" of a speaker, and a television.
- Some devices are assigned multiple functions, such as “image display” and “voice reproduction” of a receiver, and “illuminance adjustment” and “voice reproduction” of a light bulb type speaker.
- the TV receiver can be used as a device for "voice playback" only, for example.
- the function combination inside the conventional device is released, and each function is individually independent based on the external cooperation signal. Have a mechanism to make it work.
- the information processing shown in FIG. 47 is realized by at least coordinating a plurality of devices including the data management server 10 (control unit 100) and the playback device 30 (control unit 300) in the information processing system 1.
- the information processing system 1 acquires sensor data by real-time sensing (S701), and determines whether or not the information obtained from the sensor data satisfies the conditions for invoking the user scenario (S702). ).
- step S702 If it is determined in the determination process of step S702 that the activation condition is satisfied, the process proceeds to step S703. Then, in the information processing system 1, a device capable of presenting the content element is searched (S703), and at least one or more devices are controlled according to the search result (S704).
- one or more devices to be controlled present content elements corresponding to the context information satisfying the activation condition (S705).
- the sound of the agent among the content elements is output from the headphones worn by the user (the electroacoustic conversion device worn in the user's ear) (S711), and the display is displayed.
- the appearance of the agent can be displayed (S712).
- content elements can be presented by one or more output modals on one or more devices.
- the seventh embodiment has been described above.
- the contents that compose the scenario and the services using the context are coordinated. Can be provided.
- the restaurant is provided with the content of the scenario and information that the scenario is being used.
- menus such as omelet rice related to animation are prepared in advance, and it is assumed that the user who is using the scenario will display the menu for the electronic menu that opens in the restaurant. ..
- the distribution status of context information in the user's daily living space set in the scenario created by another service may be acquired, and music according to the context may be automatically provided as a content element.
- the user can receive a song or a part of the song that fits the context on a daily basis in a place with the context information set by the user, so that the user gets tired of listening to the same song every day. Can be avoided.
- At least a server or the like provided by an external service cooperates with the data management server 10 (control unit 100) and the playback device 30 (control unit 300) in the information processing system 1. It is realized by.
- At least one or more content elements are extracted from the contents composed of a plurality of media (S801), context information is given to each content element, and the content element-context information. It is accumulated in DB 151 (S802).
- one or more "content element-context information" data sets are stored in the scenario DB 152 as a scenario (S803).
- a user scenario is generated, it is stored in the user scenario DB 153 (S804).
- the data set, scenario, or user scenario of "content element-context information" accumulated in this way can be provided to an external service (S805).
- an external service such as a music streaming distribution service can control the service provided by itself to match a scenario, a user scenario, or the like (S811).
- sensor data by real-time sensing is acquired (S821), and it is determined whether or not the information obtained from the sensor data satisfies the conditions for invoking the user scenario (S822).
- step S822 If it is determined in the determination process of step S822 that the activation condition is satisfied, the content element corresponding to the context information that satisfies the activation condition is presented (S823).
- a voice character corresponding to a content element (musical piece) associated with a user scenario may be selected (S841), and introduction information may be presented as a DJ introducing the music in the service (S842). it can.
- the eighth embodiment has been described above.
- the scenario created by the user can be shared among the users by using the sharing means.
- social media such as social networking services (SNS) are used as a sharing means, and scenarios created by users (user scenarios) are published for each SNS account, for example, and the degree of similarity of content elements and the similarity of contexts. It is possible to search and classify according to the degree and the degree of similarity of the activation condition settings.
- SNS social networking services
- a map application is used as a sharing means, and a scenario including the user's current position as the activation condition is specified and presented so that the user can discover a new scenario. May be good.
- Information on the work and author that is the basis of the content element of the scenario information on the author who extracted the content element and gave the context, and information on the user who set the activation condition can be obtained in association with the scenario, and the scenario can be obtained. Users can follow their favorite authors and users.
- the information processing shown in FIG. 49 is performed by at least coordinating the data management server 10 (control unit 100) and the playback device 30 (control unit 300) in the information processing system 1 with a server or the like provided by social media. It will be realized.
- At least one or more content elements are extracted from the contents composed of a plurality of media (S901), and context information is given to each content element (S902).
- one or more "content element-context information" data sets are stored in the scenario DB 152 as a scenario (S903).
- a user scenario is generated, it is stored in the user scenario DB 153 (S904).
- the scenarios and user scenarios accumulated in this way can be uploaded to the social media server on the Internet 40 (S905).
- other users can view the scenarios and user scenarios published on social media (S906).
- the user can follow the author, the user, or the like of preference regarding the acquired scenario.
- steps S911 to S913 when the sensor data by real-time sensing satisfies the activation condition of the user scenario, the content element corresponding to the context information satisfying the activation condition is presented.
- the ninth embodiment has been described above.
- the data constituting the content element is not limited to audio and video, and for example, a moving image is reproduced using AR glass or the like. It shall include formats and data that have devices that can be presented, such as images, tactile sensations, odors, etc., such as presenting the tactile sensation of the ground using shoes with vibrating devices.
- the information processing shown in FIG. 50 is executed by the data management server 10 (control unit 100) in the information processing system 1.
- At least one or more content elements are extracted from the content composed of a plurality of media (S1001), but the plurality of media can be presented by the playback device 30.
- At least one of the tactile data and the odor data can be included.
- control may be performed to switch the user scenario to another one according to the feedback from the user.
- the user can surely receive the presentation of the content element suitable for himself / herself.
- the information processing shown in FIG. 51 is realized by at least linking the data management server 10 (control unit 100) and the playback device 30 (control unit 300) in the information processing system 1.
- At least one or more content elements are extracted from the contents composed of a plurality of media (S1101), and context information is given to each content element (S1102).
- the data set of one or more "content elements-context information" is accumulated in the scenario DB 152 as a scenario. Then, a user scenario is generated by setting the activation condition for the scenario accumulated in the scenario DB 152 (S1103).
- sensor data by real-time sensing is acquired (S1104), and it is determined whether or not the information obtained from the sensor data satisfies the activation condition of the user scenario (S1105).
- step SS1105 If it is determined in the determination process of step SS1105 that the activation condition is satisfied, the content element corresponding to the context information that satisfies the activation condition is presented (S1106).
- the user's preference for the content element is estimated (S1111), and the user scenario is recommended according to the user's preference (S1121).
- the above-mentioned steps S1104 to S1106 are repeated in a state of switching to the recommended user scenario, and a content element (for example, a favorite voice character) more suitable for the user's taste can be presented.
- the content element itself may be recommended so that the recommended content element is presented.
- the eleventh embodiment has been described above.
- the information processing system 1 is composed of the data management server 10, the editing device 20, and the playback devices 30-1 to 30-N has been described, but for example, another device may be added. Other configurations may be used.
- the data management server 10 as one information processing device may be divided into a dedicated database server and a distribution server for distribution of scenarios, content elements, etc., and configured as a plurality of information processing devices. Good.
- the editing device 20 or the playback device 30 may be configured not only as one information processing device but also as a plurality of information processing devices.
- the playback device 30 executes a part of the information processing processing by the data management server 10 described above, or is connected to a network (peripheral part of the network) close to the playback device 30. It may be executed by an edge server.
- the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems.
- each component may be connected via the Internet 40 or may be connected via a local network (LAN (Local Area Network) or WAN (Wide Area Network)). Further, each component may be connected by wire or wirelessly.
- LAN Local Area Network
- WAN Wide Area Network
- the conventional technology mainly aims to realize convenience of use by automating information retrieval work and device operation by the user.
- This type of automation typically determines if a system-defined context classification matches a context inferred by sensing user behavior or state.
- Such a system is composed of the elements shown in (a) to (d) below, and it is possible to specify the context defined by the system from the results of sensing the user's behavior, operation, and physical condition. It is a feature.
- the user's context can be defined on the system side, so that the user also It is easier to agree with the system-defined context.
- the content presented by the conventional technology is presented to the user without changing the provision format used in the conventional service.
- the data or music that is selected and provided by recognizing the context is presented to the user in the same form without changing the form of distribution to the service.
- the above-mentioned provision format is designed on the premise of the conventional viewing behavior, so that it can be a factor that hinders the free and diverse user behavior of the daily life.
- content such as movies and music is a format that requires the audience to sit in front of the screen or speakers for viewing, and if it is designed on the premise of conventional viewing behavior, it may hinder user behavior. There is.
- mobile devices such as smartphones are adapted to the user's daily activities with portability by pursuing portability, but the premise of viewing behavior centered on the screen remains the same. Therefore, for example, walking on general roads and public facilities is considered dangerous as so-called “smartphone walking” due to its characteristics of depriving sight and hearing.
- Patent Document 1 discloses a device that estimates a landmark visually recognized by the user and uses the information to provide a navigation service that indicates the direction of travel of the user.
- the triggering conditions for each user can be set for the context.
- Patent Document 2 discloses a system that extracts context information and content information from a content item, generates an index, and responds to generate a recommendation based on the user's context and the content of the user's query.
- the context information includes the time of the search, the recently accessed document, the running application, and the activity, and does not include the physical position of the user (see paragraph [0011]).
- Patent Document 3 edits that when the content includes the faces of a plurality of persons as a plurality of objects (including audio), the faces of only two persons defined as context information are enlarged to a specified size.
- Patent Document 4 based on the content broadcast schedule and broadcast history information, the correspondence relationship between the viewer's context (time zone, day, etc.) suitable for viewing the content and the feature amount of the content is learned in advance. , It is disclosed that by generating a correspondence table of "context-content feature amount", information indicating a context suitable for viewing the new content is generated and added as metadata. .. However, Patent Document 4 does not disclose that the content is cut out from the existing content.
- Patent Document 5 records all the context information extracted from the sensing data (motion, voice, heartbeat, emotion, etc.) indicating the user's state and the video that the user is watching at that time. Using the context information that shows the current user's state, the content according to the user's state is extracted, and the context information that shows that "the user was excited and pushed up his arm while playing soccer" was generated. Then, the contents recorded in the past can be extracted and provided to the user according to the keywords such as soccer and excitement, the heart rate, and the movement of the arm.
- Patent Document 5 does not disclose the extraction of content and context from existing content.
- Patent Documents 1 to 5 As described above, even if the techniques disclosed in Patent Documents 1 to 5 are used, it is hard to say that a good user experience can be provided when providing a service using context information, and a better user experience is provided. Was required to do.
- FIG. 52 is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes programmatically.
- a CPU Central Processing Unit
- ROM Read Only Memory
- RAM Random Access Memory
- An input / output interface 1005 is further connected to the bus 1004.
- An input unit 1006, an output unit 1007, a recording unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.
- the input unit 1006 includes a microphone, a keyboard, a mouse, and the like.
- the output unit 1007 includes a speaker, a display, and the like.
- the recording unit 1008 includes a hard disk, a non-volatile memory, and the like.
- the communication unit 1009 includes a network interface and the like.
- the drive 1010 drives a removable recording medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
- the CPU 1001 loads the program recorded in the ROM 1002 and the recording unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the above-mentioned series. Is processed.
- the program executed by the computer can be recorded and provided on the removable recording medium 1011 as a package medium or the like, for example. Programs can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting.
- the program can be installed in the recording unit 1008 via the input / output interface 1005 by mounting the removable recording medium 1011 in the drive 1010. Further, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the recording unit 1008. In addition, the program can be installed in advance in the ROM 1002 or the recording unit 1008.
- the processing performed by the computer according to the program does not necessarily have to be performed in chronological order in the order described as the flowchart. That is, the processing performed by the computer according to the program also includes processing executed in parallel or individually (for example, parallel processing or processing by an object). Further, the program may be processed by one computer (processor) or may be distributed by a plurality of computers.
- each step of information processing in each embodiment can be executed by one device or shared by a plurality of devices.
- the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.
- Context information is pre-associated with the content element
- at least the trigger condition can be set for the context information, and a user scenario consisting of the context information and the data set of the trigger condition can be generated.
- the sensor data obtained by sensing the user in real time satisfies the activation condition set in the user scenario
- the content element associated with the context information corresponding to the activation condition is presented to the user.
- the control unit From content consisting of multiple media Extract content elements consisting of at least some media Based on the content, generate contextual information corresponding to the content element,
- the information processing system according to (1) above which generates a corresponding database in which the content element and the context information are associated and accumulated.
- control unit generates a scenario database in which a data set composed of the content element and the context information is packaged and accumulated based on a certain theme.
- the content element is part of the streaming content and The information processing system according to (2) above, wherein information indicating the ID and playback range of the content is stored in association with the context information.
- the control unit presents another content element including a specific voice character corresponding to the context information before reproducing the content element.
- the control unit assigns content information to a new content element by machine learning the relationship between the content element stored in the corresponding database and the context information. Any of the above (2) to (5). Information processing system described in Crab.
- the control unit A scenario consisting of the content element and the data set of the context information is presented together with the map information.
- the information processing system according to (3) above which presents an interface in which a creator who creates a scenario can set a predetermined area on a map as a default value of an activation condition corresponding to the context information.
- the control unit From the content consisting of the first media A second medium different from the first medium is generated and used as a content element. Based on the content, generate contextual information corresponding to the content element, The information processing system according to any one of (1) to (7) above, which generates a corresponding database in which the content element and the context information are associated and accumulated.
- the first medium contains text and The information processing system according to (8) above, wherein the second medium includes TTS (Text To Speech) voice.
- the control unit Machine learning the relationship between the first media and the second media in advance The information processing system according to (8) or (9), wherein the second medium is generated from the first medium based on the result of the machine learning.
- the control unit For the context information Currently, it is possible to set the activation condition according to the sensor data obtained by sensing the user, and generate a user scenario database including a plurality of the context information and the data set of the activation condition (1) to (1). The information processing system according to any one of 10). (12) The information processing system according to (11) above, wherein the control unit sets activation conditions according to the captured image data.
- the information according to any one of (11) to (16) above which acquires data that can set a temporal or spatial activation condition or an activation condition according to a user's behavior as the sensor data.
- Processing system. (18)
- the control unit Along with the map information, a scenario consisting of the content element and the context information dataset associated with each other is presented.
- the information processing system according to any one of (1) and (11) to (17) above which presents an interface on which a user can set a predetermined area on a map as an activation condition corresponding to the context information.
- the control unit presents a plurality of content elements corresponding to the plurality of context information to the user according to a predetermined rule.
- the information processing system according to any one of (18).
- (20) The information processing system according to (19), wherein the control unit identifies one content element from the plurality of content elements according to the orientation of the user estimated from the sensor data, and presents the content element to the user.
- (21) The control unit When the orientation of the user estimated from the sensor data is the first orientation, the content element corresponding to the first character is specified and presented to the user.
- (22) The information processing system according to (21), wherein the control unit provides information associated with the position of the first character or the second character according to the position of the first character or the second character.
- the control unit When the sensor data satisfies the activation condition, a device capable of presenting a content element associated with the context information corresponding to the activation condition is searched around the user's current position.
- the information processing system according to any one of (1) to (22) above, which controls the device so that the content element is presented to the user.
- the control unit While controlling the electroacoustic conversion device worn on the user's ear so that the agent's voice included in the content element is presented to the user, The information processing system according to (23) above, which controls a display arranged around the user so that the appearance of the agent included in the content element is presented to the user.
- control unit provides a specific user scenario to a service provider via a communication unit.
- control unit provides a music character corresponding to the content element associated with the user scenario in the music streaming distribution service.
- the information processing system according to (25) above which is set as a disk jockey (DJ) to be introduced.
- DJ disk jockey
- the content element includes at least one of tactile data and odor data that can be presented by a device.
- the control unit switches the user scenario to another user scenario in response to feedback from the user presented with the content element.
- the control unit estimates a user's preference for the content element by analyzing the feedback.
- the control unit recommends the content element or the user scenario according to the preference of the user.
- Information processing device Context information is pre-associated with the content element
- at least the trigger condition can be set for the context information, and a user scenario consisting of the context information and the data set of the trigger condition can be generated.
- the sensor data obtained by sensing the user in real time satisfies the activation condition set in the user scenario
- the content element associated with the context information corresponding to the activation condition is presented to the user.
- Context information is pre-associated with the content element
- at least the trigger condition can be set for the context information, and a user scenario consisting of the context information and the data set of the trigger condition can be generated.
- a computer-readable recording medium that records a program for functioning as a control unit.
- 1 Information processing system 10 Data management server, 20 Editing equipment, 30, 30-1 to 30-N playback equipment, 40 Internet, 100 Control unit, 101 Input unit, 102 Output unit, 103 Storage unit, 104 Communication unit, 111 Data management unit, 112 data processing unit, 113 communication control unit, 131 presentation character selection unit, 132 scenario processing unit, 133 response generation unit, 151 content element-context information DB, 152 scenario DB, 153 user scenario DB, 161 character arrangement DB, 162 position-dependent information DB, 163 scenario DB, 200 control unit, 201 input unit, 202 output unit, 203 storage unit, 204 communication unit, 211 editing processing unit, 212 presentation control unit, 213 communication control unit, 221 mouse, 222 keyboard, 231 display, 232 speaker, 300 control unit, 301 input unit, 302 output unit, 303 storage unit, 304 communication unit, 305 sensor unit, 306 camera unit, 307 output terminal, 308 power supply unit, 311 playback processing unit, 312 presentation control unit, 313 communication
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Acoustics & Sound (AREA)
- Library & Information Science (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Remote Sensing (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Physiology (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
2.第2の実施の形態:シナリオDBの生成
3.第3の実施の形態:異なるメディアの生成
4.第4の実施の形態:ユーザシナリオDBの生成
5.第5の実施の形態:センシング手段の構成
6.第6の実施の形態:発動条件が複数のコンテキスト情報に設定された場合の構成
7.第7の実施の形態:複数の機器が連動した構成
8.第8の実施の形態:別のサービスと協調した構成
9.第9の実施の形態:シナリオを共有した構成
10.第10の実施の形態:データの他の例
11.第11の実施の形態:ユーザフィードバックを利用した構成
12.変形例
13.コンピュータの構成
図1は、本技術の概要を示した代表図である。
図2は、本技術を適用した情報処理システムの構成の例を示している。
図3は、図2のデータ管理サーバ10の構成の例を示している。
図4は、図2の編集機器20の構成の例を示している。
図5は、図2の再生機器30の構成の例を示している。
まず、図6を参照して、第1の実施の形態における情報処理の全体像を説明する。
次に、図7のフローチャートを参照して、第1の実施の形態における情報処理の詳細な流れを説明する。
次に、図8乃至図10を参照して、データ管理サーバ10により管理されるデータベースの例を説明する。
ここで、図14乃至図17を参照して、シナリオを生成するためのシナリオ生成ツールのユーザインターフェースについて説明する。このシナリオ生成ツールは、制作者等により操作される編集機器20の制御部200により実行され、各種の画面がディスプレイ231に表示される。
次に、図18を参照して、第2の実施の形態における情報処理の全体像を説明する。
次に、図19を参照して、第3の実施の形態における情報処理の全体像を説明する。
ここで、図21乃至図25を参照して、スマートフォン等の再生機器30により実行されるユーザシナリオ生成ツールのユーザインターフェースについて説明する。このユーザシナリオ生成ツールは、例えば、ユーザにより操作される再生機器30の制御部300により実行され、各種の画面がディスプレイ331に表示される。
次に、図31及び図32を参照して、第4の実施の形態における情報処理の全体像を説明する。
また、上述した制御は、音声(音)の提示に限るものではなく、拡張現実(AR)に対応した眼鏡型の機器等の表示装置を通じたキャラクタの画像提示についても同様に制御することができる。そこで、次に、図39乃至図45を参照して、シナリオに対して複数のキャラクタの配置を設定可能にする場合について説明する。
次に、図46を参照して、第6の実施の形態における情報処理の全体像を説明する。
次に、図47を参照して、第7の実施の形態における情報処理の全体像を説明する。
次に、図48を参照して、第8の実施の形態における情報処理の全体像を説明する。
次に、図49を参照して、第9の実施の形態における情報処理の全体像を説明する。
次に、図50を参照して、第10の実施の形態における情報処理の全体像を説明する。
図51を参照して、第11の実施の形態における情報処理の全体像を説明する。
(b)ユーザのアクセスしたコンテンツを認識し、当該コンテンツの属性データや内容の分析からコンテキストを認識する
(c)コンテキストとコンテンツの組み合わせのデータベースを持つ
(d)センシングデータとコンテキストを関連づけるデータベースを前提とする
コンテンツ要素にコンテキスト情報があらかじめ対応付けられ、
ユーザごとに、少なくとも当該コンテキスト情報に対して発動条件を設定可能で、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオを生成可能であり、
ユーザをリアルタイムでセンシングすることで得られたセンサデータが、前記ユーザシナリオに設定される発動条件を満たしたとき、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素がユーザに提示されるように制御する
制御部を備える
情報処理システム。
(2)
前記制御部は、
複数のメディアからなるコンテンツから、
少なくとも一部のメディアからなるコンテンツ要素を抽出し、
前記コンテンツに基づいて、前記コンテンツ要素に対応するコンテキスト情報を生成し、
前記コンテンツ要素と前記コンテキスト情報とを対応付けて蓄積した対応データベースを生成する
前記(1)に記載の情報処理システム。
(3)
前記制御部は、前記コンテンツ要素と前記コンテキスト情報からなるデータセットを、一定のテーマに基づいてパッケージ化して蓄積したシナリオデータベースを生成する
前記(2)に記載の情報処理システム。
(4)
前記コンテンツ要素は、ストリーミング配信コンテンツの一部であり、
前記コンテキスト情報に対応付けて、そのコンテンツのIDと再生範囲を示す情報が蓄積されている
前記(2)に記載の情報処理システム。
(5)
前記制御部は、前記コンテンツ要素を再生する前に、前記コンテキスト情報に対応する特定の音声キャラクタを含む他のコンテンツ要素を提示する
前記(4)に記載の情報処理システム。
(6)
前記制御部は、前記対応データベースに蓄積されるコンテンツ要素と前記コンテキスト情報との関係を機械学習することにより、新たなコンテンツ要素に対してコンテンツ情報を付与する
前記(2)乃至(5)のいずれかに記載の情報処理システム。
(7)
前記制御部は、
地図情報とともに、前記コンテンツ要素と前記コンテキスト情報のデータセットからなるシナリオを提示し、
前記コンテキスト情報に対応する発動条件のデフォルト値として、シナリオを作成する制作者が地図上に所定領域を設定可能なインターフェースを提示する
前記(3)に記載の情報処理システム。
(8)
前記制御部は、
第1のメディアからなるコンテンツから、
前記第1のメディアとは異なる第2のメディアを生成してコンテンツ要素とし、
前記コンテンツに基づいて、前記コンテンツ要素に対応するコンテキスト情報を生成し、
前記コンテンツ要素と前記コンテキスト情報とを対応付けて蓄積した対応データベースを生成する
前記(1)乃至(7)のいずれかに記載の情報処理システム。
(9)
前記第1のメディアは、テキストを含み、
前記第2のメディアは、TTS(Text To Speech)音声を含む
前記(8)に記載の情報処理システム。
(10)
前記制御部は、
前記第1のメディアと前記第2のメディアとの関係をあらかじめ機械学習しておき、
当該機械学習の結果に基づいて、前記第1のメディアから、前記第2のメディアを生成する
前記(8)又は(9)に記載の情報処理システム。
(11)
前記制御部は、
前記コンテキスト情報に対して、
現在、ユーザをセンシングすることで得られるセンサデータに応じた発動条件を設定可能であり、複数の、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオデータベースを生成する
前記(1)乃至(10)のいずれかに記載の情報処理システム。
(12)
前記制御部は、撮像された画像データに応じた発動条件を設定する
前記(11)に記載の情報処理システム。
(13)
前記制御部は、ユーザの特性操作に応じて、そのときのセンサデータに応じた発動条件を設定する
前記(11)に記載の情報処理システム。
(14)
前記制御部は、
前記コンテキスト情報と前記発動条件との関係を機械学習し、
当該機械学習の結果に応じた情報を出力する
前記(11)乃至(13)のいずれかに記載の情報処理システム。
(15)
前記制御部は、前記機械学習の結果に応じて、特定の発動条件に対して、コンテキスト情報を生成する
前記(14)に記載の情報処理システム。
(16)
前記制御部は、前記機械学習の結果に応じて、特定のコンテキスト情報に対して、ユーザに対応した発動条件を設定する
前記(14)に記載の情報処理システム。
(17)
前記センシングでは、前記センサデータとして、時間的若しくは空間的な発動条件、又はユーザの行動に応じた発動条件を設定可能なデータを取得する
前記(11)乃至(16)のいずれかに記載の情報処理システム。
(18)
前記制御部は、
地図情報とともに、あらかじめ対応付けられている前記コンテンツ要素と前記コンテキスト情報のデータセットからなるシナリオを提示し、
前記コンテキスト情報に対応する発動条件として、ユーザが地図上に所定領域を設定可能なインターフェースを提示する
前記(1)、及び(11)乃至(17)のいずれかに記載の情報処理システム。
(19)
前記制御部は、同一の発動条件が、複数のコンテキスト情報に設定されているとき、所定のルールに従って、当該複数のコンテキスト情報に対応する複数のコンテンツ要素を、ユーザに提示する
前記(1)乃至(18)のいずれかに記載の情報処理システム。
(20)
前記制御部は、前記センサデータにより推定されるユーザの向きに応じて、前記複数のコンテンツ要素から、一のコンテンツ要素を特定し、ユーザに提示する
前記(19)に記載の情報処理システム。
(21)
前記制御部は、
前記センサデータにより推定されるユーザの向きが第1の向きとなるとき、第1のキャラクタに対応するコンテンツ要素を特定して、ユーザに提示し、
ユーザの向きが第2の向きとなるとき、第2のキャラクタに対応するコンテンツ要素を特定して、ユーザに提示する
前記(20)に記載の情報処理システム。
(22)
前記制御部は、前記第1のキャラクタ又は前記第2のキャラクタの位置に応じてその場所に紐付けられた情報を提供する
前記(21)に記載の情報処理システム。
(23)
前記制御部は、
前記センサデータが前記発動条件を満たしたとき、ユーザの現在位置周辺に、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素を提示可能な機器を探索し、
前記コンテンツ要素がユーザに提示されるように、当該機器を制御する
前記(1)乃至(22)のいずれかに記載の情報処理システム。
(24)
前記制御部は、
前記コンテンツ要素に含まれるエージェントの音声が、ユーザに提示されるように、当該ユーザの耳に装着された電気音響変換機器を制御するとともに、
前記コンテンツ要素に含まれるエージェントの外観が、ユーザに提示されるように、当該ユーザの周辺に配置されるディスプレイを制御する
前記(23)に記載の情報処理システム。
(25)
前記制御部は、通信部を介して、特定のユーザシナリオをサービス提供者に提供する
前記(1)乃至(24)のいずれかに記載の情報処理システム。
(26)
前記制御部は、通信部を介して、前記特定のユーザシナリオを音楽ストリーミング配信サービス業者に提供することにより、当該ユーザシナリオに対応付けられるコンテンツ要素に対応する音声キャラクタを、音楽ストリーミング配信サービスにおいて楽曲を紹介するディスクジョッキー(DJ)として設定する
前記(25)に記載の情報処理システム。
(27)
前記制御部は、通信部を介して、前記ユーザシナリオを、ソーシャルメディアにアップロードし、他のユーザと共有可能にする
前記(1)乃至(24)のいずれかに記載の情報処理システム。
(28)
前記コンテンツ要素は、機器により提示可能な触覚データ及び匂いデータの少なくとも一方のデータを含む
前記(1)乃至(27)のいずれかに記載の情報処理システム。
(29)
前記制御部は、前記コンテンツ要素が提示されたユーザからのフィードバックに応じて、前記ユーザシナリオを、別のユーザシナリオに切り替える
前記(1)乃至(28)のいずれかに記載の情報処理システム。
(30)
前記制御部は、前記フィードバックを分析することにより、前記コンテンツ要素に対するユーザの嗜好を推定する
前記(29)に記載の情報処理システム。
(31)
前記制御部は、前記ユーザの嗜好に応じて、前記コンテンツ要素又は前記ユーザシナリオを推薦する
前記(30)に記載の情報処理システム。
(32)
情報処理装置が、
コンテンツ要素にコンテキスト情報があらかじめ対応付けられ、
ユーザごとに、少なくとも当該コンテキスト情報に対して発動条件を設定可能で、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオを生成可能であり、
ユーザをリアルタイムでセンシングすることで得られたセンサデータが、前記ユーザシナリオに設定される発動条件を満たしたとき、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素がユーザに提示されるように制御する
情報処理方法。
(33)
コンピュータを、
コンテンツ要素にコンテキスト情報があらかじめ対応付けられ、
ユーザごとに、少なくとも当該コンテキスト情報に対して発動条件を設定可能で、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオを生成可能であり、
ユーザをリアルタイムでセンシングすることで得られたセンサデータが、前記ユーザシナリオに設定される発動条件を満たしたとき、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素がユーザに提示されるように制御する制御部として
機能させるためのプログラムを記録したコンピュータが読み取り可能な記録媒体。
Claims (33)
- コンテンツ要素にコンテキスト情報があらかじめ対応付けられ、
ユーザごとに、少なくとも当該コンテキスト情報に対して発動条件を設定可能で、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオを生成可能であり、
ユーザをリアルタイムでセンシングすることで得られたセンサデータが、前記ユーザシナリオに設定される発動条件を満たしたとき、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素がユーザに提示されるように制御する
制御部を備える
情報処理システム。 - 前記制御部は、
複数のメディアからなるコンテンツから、
少なくとも一部のメディアからなるコンテンツ要素を抽出し、
前記コンテンツに基づいて、前記コンテンツ要素に対応するコンテキスト情報を生成し、
前記コンテンツ要素と前記コンテキスト情報とを対応付けて蓄積した対応データベースを生成する
請求項1に記載の情報処理システム。 - 前記制御部は、前記コンテンツ要素と前記コンテキスト情報からなるデータセットを、一定のテーマに基づいてパッケージ化して蓄積したシナリオデータベースを生成する
請求項2に記載の情報処理システム。 - 前記コンテンツ要素は、ストリーミング配信コンテンツの一部であり、
前記コンテキスト情報に対応付けて、そのコンテンツのIDと再生範囲を示す情報が蓄積されている
請求項2に記載の情報処理システム。 - 前記制御部は、前記コンテンツ要素を再生する前に、前記コンテキスト情報に対応する特定の音声キャラクタを含む他のコンテンツ要素を提示する
請求項4に記載の情報処理システム。 - 前記制御部は、前記対応データベースに蓄積されるコンテンツ要素と前記コンテキスト情報との関係を機械学習することにより、新たなコンテンツ要素に対してコンテンツ情報を付与する
請求項2に記載の情報処理システム。 - 前記制御部は、
地図情報とともに、前記コンテンツ要素と前記コンテキスト情報のデータセットからなるシナリオを提示し、
前記コンテキスト情報に対応する発動条件のデフォルト値として、シナリオを作成する制作者が地図上に所定領域を設定可能なインターフェースを提示する
請求項3に記載の情報処理システム。 - 前記制御部は、
第1のメディアからなるコンテンツから、
前記第1のメディアとは異なる第2のメディアを生成してコンテンツ要素とし、
前記コンテンツに基づいて、前記コンテンツ要素に対応するコンテキスト情報を生成し、
前記コンテンツ要素と前記コンテキスト情報とを対応付けて蓄積した対応データベースを生成する
請求項1に記載の情報処理システム。 - 前記第1のメディアは、テキストを含み、
前記第2のメディアは、TTS(Text To Speech)音声を含む
請求項8に記載の情報処理システム。 - 前記制御部は、
前記第1のメディアと前記第2のメディアとの関係をあらかじめ機械学習しておき、
当該機械学習の結果に基づいて、前記第1のメディアから、前記第2のメディアを生成する
請求項8に記載の情報処理システム。 - 前記制御部は、
前記コンテキスト情報に対して、
現在、ユーザをセンシングすることで得られるセンサデータに応じた発動条件を設定可能であり、複数の、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオデータベースを生成する
請求項1に記載の情報処理システム。 - 前記制御部は、撮像された画像データに応じた発動条件を設定する
請求項11に記載の情報処理システム。 - 前記制御部は、ユーザの特性操作に応じて、そのときのセンサデータに応じた発動条件を設定する
請求項11に記載の情報処理システム。 - 前記制御部は、
前記コンテキスト情報と前記発動条件との関係を機械学習し、
当該機械学習の結果に応じた情報を出力する
請求項11に記載の情報処理システム。 - 前記制御部は、前記機械学習の結果に応じて、特定の発動条件に対して、コンテキスト情報を生成する
請求項14に記載の情報処理システム。 - 前記制御部は、前記機械学習の結果に応じて、特定のコンテキスト情報に対して、ユーザに対応した発動条件を設定する
請求項14に記載の情報処理システム。 - 前記センシングでは、前記センサデータとして、時間的若しくは空間的な発動条件、又はユーザの行動に応じた発動条件を設定可能なデータを取得する
請求項11に記載の情報処理システム。 - 前記制御部は、
地図情報とともに、あらかじめ対応付けられている前記コンテンツ要素と前記コンテキスト情報のデータセットからなるシナリオを提示し、
前記コンテキスト情報に対応する発動条件として、ユーザが地図上に所定領域を設定可能なインターフェースを提示する
請求項1に記載の情報処理システム。 - 前記制御部は、同一の発動条件が、複数のコンテキスト情報に設定されているとき、所定のルールに従って、当該複数のコンテキスト情報に対応する複数のコンテンツ要素を、ユーザに提示する
請求項1に記載の情報処理システム。 - 前記制御部は、前記センサデータにより推定されるユーザの向きに応じて、前記複数のコンテンツ要素から、一のコンテンツ要素を特定し、ユーザに提示する
請求項19に記載の情報処理システム。 - 前記制御部は、
前記センサデータにより推定されるユーザの向きが第1の向きとなるとき、第1のキャラクタに対応するコンテンツ要素を特定して、ユーザに提示し、
ユーザの向きが第2の向きとなるとき、第2のキャラクタに対応するコンテンツ要素を特定して、ユーザに提示する
請求項20に記載の情報処理システム。 - 前記制御部は、前記第1のキャラクタ又は前記第2のキャラクタの位置に応じてその場所に紐付けられた情報を提供する
請求項21に記載の情報処理システム。 - 前記制御部は、
前記センサデータが前記発動条件を満たしたとき、ユーザの現在位置周辺に、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素を提示可能な機器を探索し、
前記コンテンツ要素がユーザに提示されるように、当該機器を制御する
請求項1に記載の情報処理システム。 - 前記制御部は、
前記コンテンツ要素に含まれるエージェントの音声が、ユーザに提示されるように、当該ユーザの耳に装着された電気音響変換機器を制御するとともに、
前記コンテンツ要素に含まれるエージェントの外観が、ユーザに提示されるように、当該ユーザの周辺に配置されるディスプレイを制御する
請求項23に記載の情報処理システム。 - 前記制御部は、通信部を介して、特定のユーザシナリオをサービス提供者に提供する
請求項1に記載の情報処理システム。 - 前記制御部は、通信部を介して、前記特定のユーザシナリオを音楽ストリーミング配信サービス業者に提供することにより、当該ユーザシナリオに対応付けられるコンテンツ要素に対応する音声キャラクタを、音楽ストリーミング配信サービスにおいて楽曲を紹介するディスクジョッキー(DJ)として設定する
請求項25に記載の情報処理システム。 - 前記制御部は、通信部を介して、前記ユーザシナリオを、ソーシャルメディアにアップロードし、他のユーザと共有可能にする
請求項1に記載の情報処理システム。 - 前記コンテンツ要素は、機器により提示可能な触覚データ及び匂いデータの少なくとも一方のデータを含む
請求項1に記載の情報処理システム。 - 前記制御部は、前記コンテンツ要素が提示されたユーザからのフィードバックに応じて、前記ユーザシナリオを、別のユーザシナリオに切り替える
請求項1に記載の情報処理システム。 - 前記制御部は、前記フィードバックを分析することにより、前記コンテンツ要素に対するユーザの嗜好を推定する
請求項29に記載の情報処理システム。 - 前記制御部は、前記ユーザの嗜好に応じて、前記コンテンツ要素又は前記ユーザシナリオを推薦する
請求項30に記載の情報処理システム。 - 情報処理装置が、
コンテンツ要素にコンテキスト情報があらかじめ対応付けられ、
ユーザごとに、少なくとも当該コンテキスト情報に対して発動条件を設定可能で、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオを生成可能であり、
ユーザをリアルタイムでセンシングすることで得られたセンサデータが、前記ユーザシナリオに設定される発動条件を満たしたとき、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素がユーザに提示されるように制御する
情報処理方法。 - コンピュータを、
コンテンツ要素にコンテキスト情報があらかじめ対応付けられ、
ユーザごとに、少なくとも当該コンテキスト情報に対して発動条件を設定可能で、前記コンテキスト情報と前記発動条件のデータセットからなるユーザシナリオを生成可能であり、
ユーザをリアルタイムでセンシングすることで得られたセンサデータが、前記ユーザシナリオに設定される発動条件を満たしたとき、当該発動条件に応じたコンテキスト情報に対応付けられたコンテンツ要素がユーザに提示されるように制御する制御部として
機能させるためのプログラムを記録したコンピュータが読み取り可能な記録媒体。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021527610A JPWO2020255767A1 (ja) | 2019-06-20 | 2020-06-08 | |
CN202080043410.8A CN114008610A (zh) | 2019-06-20 | 2020-06-08 | 信息处理系统、信息处理方法和记录介质 |
KR1020217039153A KR20220019683A (ko) | 2019-06-20 | 2020-06-08 | 정보 처리 시스템, 정보 처리 방법 및 기록 매체 |
EP20825869.9A EP3989083A4 (en) | 2019-06-20 | 2020-06-08 | INFORMATION HANDLING SYSTEM, INFORMATION HANDLING METHOD AND RECORDING MEDIA |
US17/618,682 US20220246135A1 (en) | 2019-06-20 | 2020-06-08 | Information processing system, information processing method, and recording medium |
JP2024085476A JP2024107029A (ja) | 2019-06-20 | 2024-05-27 | 情報処理プログラム、情報処理方法、及び情報処理システム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019-114626 | 2019-06-20 | ||
JP2019114626 | 2019-06-20 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020255767A1 true WO2020255767A1 (ja) | 2020-12-24 |
Family
ID=74037294
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2020/022483 WO2020255767A1 (ja) | 2019-06-20 | 2020-06-08 | 情報処理システム、情報処理方法、及び記録媒体 |
Country Status (6)
Country | Link |
---|---|
US (1) | US20220246135A1 (ja) |
EP (1) | EP3989083A4 (ja) |
JP (2) | JPWO2020255767A1 (ja) |
KR (1) | KR20220019683A (ja) |
CN (1) | CN114008610A (ja) |
WO (1) | WO2020255767A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022224523A1 (ja) * | 2021-04-21 | 2022-10-27 | ソニーグループ株式会社 | 情報処理装置、情報処理方法、及びプログラム |
WO2024161991A1 (ja) * | 2023-01-31 | 2024-08-08 | ソニーグループ株式会社 | 情報処理装置、情報処理方法、およびプログラム |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA3214519A1 (en) * | 2021-04-20 | 2022-10-27 | Jesse Dorogusker | Live playback streams |
KR102538155B1 (ko) * | 2022-08-26 | 2023-05-31 | 주식회사 스튜디오사월 | 시나리오를 창작, 공유 및 활용하는 플랫폼 서비스를 제공하기 위한 방법 및 그 장치 |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007172524A (ja) | 2005-12-26 | 2007-07-05 | Sony Corp | 情報処理装置、情報処理方法、およびプログラム |
JP2010136370A (ja) * | 2008-12-02 | 2010-06-17 | Palo Alto Research Center Inc | コンテキストベースのコンテンツをユーザに配信する方法 |
WO2013136792A1 (ja) | 2012-03-15 | 2013-09-19 | パナソニック株式会社 | コンテンツ処理装置、コンテンツ処理方法およびプログラム |
JP2014034374A (ja) * | 2012-08-10 | 2014-02-24 | Yupiteru Corp | 車両用システム及びプログラム |
JP2015210818A (ja) | 2014-04-24 | 2015-11-24 | キヤノン株式会社 | コンテキスト管理のための装置、システム、および方法 |
WO2016136104A1 (ja) | 2015-02-23 | 2016-09-01 | ソニー株式会社 | 情報処理装置、情報処理方法及びプログラム |
US20180081523A1 (en) * | 2013-12-03 | 2018-03-22 | Lenovo (Singapore) Pte. Ltd. | Devices and methods to receive input at a first device and present output in response on a second device different from the first device |
JP6463529B1 (ja) | 2018-03-20 | 2019-02-06 | ヤフー株式会社 | 情報処理装置、情報処理方法、及び情報処理プログラム |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4996361A (en) | 1987-07-30 | 1991-02-26 | The University Of British Columbia | Process for the preparation of optically active secondary arylamines |
KR20140091633A (ko) * | 2013-01-11 | 2014-07-22 | 삼성전자주식회사 | 모바일 장치에서의 상황 인지에 따른 추천 항목을 제공하기 위한 방법 및 이를 위한 모바일 장치 |
JP2019185389A (ja) * | 2018-04-10 | 2019-10-24 | 日本電信電話株式会社 | 情報処理装置、情報処理方法および情報処理プログラム |
-
2020
- 2020-06-08 JP JP2021527610A patent/JPWO2020255767A1/ja active Pending
- 2020-06-08 KR KR1020217039153A patent/KR20220019683A/ko active Search and Examination
- 2020-06-08 US US17/618,682 patent/US20220246135A1/en active Pending
- 2020-06-08 CN CN202080043410.8A patent/CN114008610A/zh active Pending
- 2020-06-08 EP EP20825869.9A patent/EP3989083A4/en active Pending
- 2020-06-08 WO PCT/JP2020/022483 patent/WO2020255767A1/ja active Application Filing
-
2024
- 2024-05-27 JP JP2024085476A patent/JP2024107029A/ja active Pending
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007172524A (ja) | 2005-12-26 | 2007-07-05 | Sony Corp | 情報処理装置、情報処理方法、およびプログラム |
JP2010136370A (ja) * | 2008-12-02 | 2010-06-17 | Palo Alto Research Center Inc | コンテキストベースのコンテンツをユーザに配信する方法 |
WO2013136792A1 (ja) | 2012-03-15 | 2013-09-19 | パナソニック株式会社 | コンテンツ処理装置、コンテンツ処理方法およびプログラム |
JP2014034374A (ja) * | 2012-08-10 | 2014-02-24 | Yupiteru Corp | 車両用システム及びプログラム |
US20180081523A1 (en) * | 2013-12-03 | 2018-03-22 | Lenovo (Singapore) Pte. Ltd. | Devices and methods to receive input at a first device and present output in response on a second device different from the first device |
JP2015210818A (ja) | 2014-04-24 | 2015-11-24 | キヤノン株式会社 | コンテキスト管理のための装置、システム、および方法 |
WO2016136104A1 (ja) | 2015-02-23 | 2016-09-01 | ソニー株式会社 | 情報処理装置、情報処理方法及びプログラム |
JP6463529B1 (ja) | 2018-03-20 | 2019-02-06 | ヤフー株式会社 | 情報処理装置、情報処理方法、及び情報処理プログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3989083A4 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022224523A1 (ja) * | 2021-04-21 | 2022-10-27 | ソニーグループ株式会社 | 情報処理装置、情報処理方法、及びプログラム |
WO2024161991A1 (ja) * | 2023-01-31 | 2024-08-08 | ソニーグループ株式会社 | 情報処理装置、情報処理方法、およびプログラム |
Also Published As
Publication number | Publication date |
---|---|
US20220246135A1 (en) | 2022-08-04 |
EP3989083A1 (en) | 2022-04-27 |
CN114008610A (zh) | 2022-02-01 |
JP2024107029A (ja) | 2024-08-08 |
KR20220019683A (ko) | 2022-02-17 |
EP3989083A4 (en) | 2022-08-24 |
JPWO2020255767A1 (ja) | 2020-12-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020255767A1 (ja) | 情報処理システム、情報処理方法、及び記録媒体 | |
US10915291B2 (en) | User-interfaces for audio-augmented-reality | |
US9838818B2 (en) | Immersive 3D sound space for searching audio | |
Rozier | Here&There: an augmented reality system of linked audio | |
US11043216B2 (en) | Voice feedback for user interface of media playback device | |
Jones et al. | ONTRACK: Dynamically adapting music playback to support navigation | |
US20160041981A1 (en) | Enhanced cascaded object-related content provision system and method | |
CN101803336B (zh) | 用于对视频进行选择性音频修改的方法和系统 | |
CN108876927B (zh) | 通过基于故事的增强和/或混合现实体验引导的物理航行 | |
US20120226706A1 (en) | System, apparatus and method for sorting music files based on moods | |
WO2002031710A1 (en) | Authoring system | |
CN107038197A (zh) | 情境及活动驱动的内容传送和交互 | |
KR20160065670A (ko) | 컨텐트를 제공하는 방법 및 디바이스 | |
Indans et al. | Towards an audio-locative mobile application for immersive storytelling | |
CN111630472A (zh) | 信息处理装置、信息处理方法和程序 | |
KR20160057122A (ko) | 비선형 쌍방향 콘텐츠 제작 시스템을 활용한 스토리 허브 시스템 | |
Kimura et al. | A Digital Platform for Sharing Collective Human Hearing. | |
CN110209870A (zh) | 音乐日志生成方法、装置、介质和计算设备 | |
Melchiorre et al. | EmoMTB: Emotion-aware music tower blocks | |
Li | Loco-Radio: designing high-density augmented reality audio browsers | |
JP5814197B2 (ja) | カラオケ装置の近辺のデジタルサイネージで映像音声による広告を出力する際の連携 | |
WO2022080164A1 (ja) | 情報処理システム、情報処理装置、及びコンテンツ再生装置 | |
WO2022102446A1 (ja) | 情報処理装置、情報処理方法、情報処理システム、及びデータ生成方法 | |
Dang et al. | Towards Accessible Musical Performances in Virtual Reality: Designing a Conceptual Framework for Omnidirectional Audio Descriptions | |
Sun | Impact of auditory modality on user experience during augmented outdoor tourism navigation and exploration tasks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20825869 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2021527610 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2020825869 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2020825869 Country of ref document: EP Effective date: 20220120 |