WO2018155354A1 - Electronic device control method, electronic device control system, electronic device, and program - Google Patents

Abstract

This electronic device control method comprises: a step (S1) for obtaining environmental information of an electronic device, the information including information about the environment surrounding the electronic device or information about the installation mode of the electronic device; and a step (S2) for performing, with reference to predetermined association between the environmental information of the electronic device and details of control regarding content presentation by the electronic device, a control regarding a content presentation by the electronic device, the content presentation being associated with the obtained environmental information. For example, the environmental information may include, as the information about the environment surrounding the electronic device, information indicating the presence or absence of a person around the electronic device, the orientation of the person around the electronic device, the direction in which the person is present around when viewed from the electronic device, the brightness in the surrounding area of the electronic device, or the sound volume in the surrounding area of the electronic device, and the environmental information may be obtained by a sensor which senses the environment surrounding the electronic device and outputs an environmental value indicating the environment.

Description

Electronic device control method, electronic device control system, electronic device, and program

The present disclosure relates to an electronic device control method, an electronic device control system, an electronic device, and a program.

Conventionally, in order to synchronize video reproduction by a plurality of computer terminals, there is a technique for selecting content according to the communication environment or computer environment of each computer terminal (see Patent Document 1).

JP 2006-108895 A

However, with the technique disclosed in Patent Document 1, content is selected only in accordance with a communication environment or a computer environment including a computer hardware factor and a software factor. Therefore, there is a problem that selection of content based on other factors, control of content presentation, and the like cannot be realized.

Therefore, the present disclosure provides an electronic device control method for controlling the presentation of content according to the environment around the electronic device, which is different from hardware or software factors of the electronic device.

An electronic device control method according to an aspect of the present disclosure obtains environmental information of the electronic device including information about an environment around the electronic device or information about an installation mode of the electronic device, and the electronic device By referring to a predetermined association between the environment information and the details of the control related to the presentation of the content by the electronic device, the control related to the presentation of the content by the electronic device associated with the acquired environment information is performed. .

Note that these comprehensive or specific modes may be realized by a recording medium such as a system, an apparatus, an integrated circuit, a computer program, or a computer-readable CD-ROM, and the system, apparatus, integrated circuit, and computer program. And any combination of recording media.

According to the present disclosure, it is possible to control the presentation of content according to the environment around the electronic device.

FIG. 1 is an explanatory diagram showing an overall image of a service. FIG. 2 is a schematic diagram illustrating an appearance of the electronic apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating a configuration of the electronic device according to the first embodiment. FIG. 4 is a detailed block diagram of the control processing unit according to the first embodiment. FIG. 5 is a detailed block diagram of the content management unit according to the first embodiment. FIG. 6 is a schematic diagram illustrating an example of an installation mode in which the electronic apparatus according to Embodiment 1 is installed on a wall. FIG. 7 is a schematic diagram illustrating an example of an installation mode in which the electronic apparatus according to the first embodiment is hung and installed on a wall. FIG. 8 is a perspective view illustrating an appearance of the electronic apparatus according to the first embodiment. FIG. 9 is an exploded perspective view of the electronic apparatus according to the first embodiment. FIG. 10 is a flowchart showing processing of the electronic device according to the first embodiment. FIG. 11 is a flowchart showing a first example of the viewing environment determination process according to Embodiment 1. FIG. 12A is a flowchart showing a second example of the viewing environment determination process according to Embodiment 1. FIG. 12B is a flowchart showing a third example of the viewing environment determination process according to Embodiment 1. FIG. 12C is a flowchart showing a fourth example of the viewing environment determination process according to Embodiment 1. FIG. 13 is a flowchart illustrating an example of the control content determination process according to the first embodiment. FIG. 14 is an explanatory diagram illustrating the state of the space and an example of output information in the first embodiment. FIG. 15 is an explanatory diagram of a table showing the estimation results of the person and space states in the first embodiment. FIG. 16 is an explanatory diagram of a table for determining control contents using the determination by the camera according to the first embodiment. FIG. 17 is an explanatory diagram of a table for determining control contents using installation environment determination by an illuminometer in the first embodiment. FIG. 18 is an explanatory diagram of a table for determining control contents using tilt determination by a three-axis sensor in the first embodiment. FIG. 19 is an explanatory diagram of a table for determining control contents using installation environment determination using buttons on the upper and lower surfaces in the first embodiment. FIG. 20 is an explanatory diagram of a content management table in the first embodiment. FIG. 21 is an explanatory diagram of a voice management table according to the first embodiment. FIG. 22A is an explanatory diagram of a table showing an example of a change in viewing environment state and content control in the first embodiment. FIG. 22B is an explanatory diagram illustrating a state in which a user is present in front of the electronic device according to Embodiment 1. FIG. 22C is an explanatory diagram illustrating a relationship between the direction in which the user exists as viewed from the electronic device according to Embodiment 1 and the output volume of the speaker. FIG. 22D is an explanatory diagram illustrating a state in which a plurality of users are present in front of the electronic device according to Embodiment 1. FIG. 22E is an explanatory diagram illustrating a relationship between the number of users existing in front of the electronic device and the output volume of the speaker in the first embodiment. FIG. 22F is an explanatory diagram of sound collection volume of the electronic device according to Embodiment 1. FIG. 22G is an explanatory diagram illustrating a relationship between the sound collection volume of the electronic device and the output volume of the speaker in the first embodiment. FIG. 23 is a block diagram illustrating configurations of an electronic device and a cloud server according to a modification of the first embodiment. FIG. 24 is a block diagram illustrating configurations of an electronic device and a cloud server according to the second embodiment. FIG. 25 is a sequence diagram illustrating processing of the electronic device and the cloud server according to the second embodiment. FIG. 26 is a block diagram illustrating an electronic apparatus according to the third embodiment. FIG. 27 is a flowchart illustrating the electronic device control method according to the third embodiment. FIG. 28 is a diagram showing service type 1 (in-house data center type). FIG. 29 is a diagram showing a service type 2 (IaaS usage type). FIG. 30 is a diagram illustrating a service type 3 (PaaS usage type). FIG. 31 is a diagram showing a service type 4 (SaaS usage type).

(Knowledge that became the basis of this disclosure)
The inventor has found that the following problems occur with respect to the technique described in the “Background Art” column.

(1) Conventionally, in order to synchronize video reproduction by a plurality of computer terminals, there is a technique for selecting content according to the communication environment or computer environment of each computer terminal (see Patent Document 1).

However, with the technique disclosed in Patent Document 1, content is selected only in accordance with a communication environment or a computer environment including a computer hardware factor and a software factor. Therefore, there is a problem that selection of content based on other factors, control of content presentation, and the like cannot be realized.

(2) Conventionally, there are electronic devices that change operation according to the user's environment. For example, dimming, illumination coloring, or sound field adjustment of audiovisual equipment such as a concert venue can be adjusted individually by a professional mechanic designing adjustment samples or arrangements. It is made by customizing.

However, this technique has the problem that a professional mechanic himself needs to set it individually according to the user's environment, and the mechanic's labor is excessive.

(3) In recent years, there are image projection type liquid crystal projectors that perform keystone correction in order to optimize the projected image. In this liquid crystal projector, only image distortion can be automatically corrected to some extent.

However, with this liquid crystal projector, the image cast by the liquid crystal projector itself is only corrected to a keystone. That is, it is not possible to adjust the playback environment of the audiovisual content according to the environment surrounding the user or the user's action. Therefore, depending on the installation method of the liquid crystal display where portability is important or the usage environment that changes depending on the user's behavior, it is difficult to see the image, it is difficult to hear the sound, or the audiovisual content that the user does not want to play There are problems such as being continued.

An electronic device control method according to an aspect of the present disclosure obtains environmental information of the electronic device including information about an environment around the electronic device or information about an installation mode of the electronic device, and the electronic device By referring to a predetermined association between the environment information and the details of the control related to the presentation of the content by the electronic device, the control related to the presentation of the content by the electronic device associated with the acquired environment information is performed. .

According to the above aspect, the electronic device performs control related to the presentation of content using the acquired environment information and a predetermined association. The predetermined association is associated in advance with the environmental information of the electronic device and the content of the control. Therefore, the electronic device can perform control related to content presentation based on the acquired environmental information of the electronic device. Thus, the electronic device can control the presentation of content according to the environment around the electronic device.

For example, the environmental information is information about the environment around the electronic device, the presence or absence of a person around the electronic device, the direction of the person around the electronic device, the direction of the person around the electronic device, Including information indicating brightness around the electronic device or sound volume around the electronic device, and obtaining the environment information senses the environment around the electronic device and outputs an environment value indicating the environment It may be made by a sensor.

According to the above aspect, the environmental information includes specific information such as the presence or orientation or the direction of the person around the electronic device, the direction of the person seen from the electronic device, or the ambient brightness or volume. Also, the electronic device can be specifically configured using a sensor that acquires environmental information. Based on such a specific configuration, the electronic device can perform control related to the presentation of content according to the environment around the electronic device.

For example, the environmental information includes the presence / absence of a person around the electronic device, the direction of the person around the electronic device, the direction of the person around the electronic device, the brightness around the electronic device, and The information indicating at least two of the volume around the electronic device may be included, and the control related to the presentation of the content by the electronic device may be performed in combination with the information indicating the at least two.

According to the above aspect, the environmental information is a composite of specific information such as the presence or orientation of a person around the electronic device, the direction of the person seen from the electronic device, or the ambient brightness or volume. Considering this, it is possible to more appropriately control the presentation of content according to the environment around the electronic device.

For example, in the control related to the presentation of the content, the content to be presented by the electronic device, which is the content of the control associated with the environment information acquired by the sensor by referring to the predetermined association Control may be performed to select and present the selected content.

According to the above aspect, the electronic device can select a content to be presented from among a plurality of contents as control related to the presentation of the contents.

For example, the predetermined association includes an association between the environment information of the electronic device and attribute information indicating the attribute of the content to be presented by the electronic device having the environment information matching the environment information. In the control related to presentation, the content having attribute information associated with the acquired environment information may be selected by referring to the predetermined association.

According to the above aspect, the electronic device can select a content based on the attribute information of the content when selecting the content to be presented from the plurality of contents.

For example, the predetermined association includes content to be presented by the electronic device and permissible information indicating a permissible presentation mode of the content, and is permitted by the permissible information in the control related to the presentation of the content. You may select the content which can be shown with a presentation aspect.

According to the above aspect, when the electronic device selects a content to be presented from among a plurality of contents, the electronic device is permitted to indicate whether editing, adjustment, or processing of the content at the time of content presentation is permitted. The content can be selected based on the information. This avoids presentation of content in an unacceptable manner.

For example, the predetermined association includes an association between information indicating a change in environmental information of the electronic device and start, stop, or end of content presentation. In the control related to content presentation, the predetermined correspondence By referring to the attachment, the presentation of content to be presented by the electronic device associated with the change in the acquired environment information may be started, stopped, or terminated.

According to the above aspect, the electronic device can start, stop, or end the presentation of content as control related to the presentation of content.

For example, the predetermined association includes an association between information indicating a change in environmental information of the electronic device and a change in volume or brightness of the content. In the control related to the presentation of the content, the predetermined association is performed. The volume or brightness of the content to be presented by the electronic device associated with the obtained change in the environmental information may be changed.

According to the above aspect, the electronic device can change the volume or brightness of the content as control related to the presentation of the content.

For example, the environment information includes information indicating an attitude of the installed electronic device as information regarding an installation mode of the electronic device, and the predetermined association includes information indicating the attitude of the electronic device and the attitude In the control relating to the presentation of the content, the control relating to the presentation of the content associated with the information indicating the attitude of the electronic device may be performed. .

According to the above aspect, the environmental information includes specific information related to the attitude of the electronic device. Then, the electronic device can perform control related to presentation of content based on the attitude of the electronic device.

For example, the environment information includes juxtaposition information that is information on whether or not the electronic device is installed side by side with other electronic devices as information on the installation mode of the electronic device, and the predetermined association is Including the correspondence between the juxtaposition information of the electronic device and the control related to the presentation of the content by the electronic device installed in the installation mode indicated in the juxtaposition information, and in the control related to the presentation of the content, the juxtaposition information of the electronic device You may perform control regarding presentation of the content matched with this.

According to the above aspect, the environment information includes specific information indicating whether or not a plurality of electronic devices are arranged side by side. Then, the electronic device can perform control related to presentation of content based on how the electronic device is arranged.

For example, the control method may further output log information related to the presentation of the content.

According to the above aspect, the electronic device outputs log information indicating the number of times content is presented, the time of presentation, and the like. The output log information is fed back to, for example, a content creator, and can be utilized when content is improved or new content is created.

In addition, an electronic device control system according to an aspect of the present disclosure includes an acquisition unit that acquires environmental information of the electronic device including information related to an environment around the electronic device or information related to an installation mode of the electronic device. And a predetermined correspondence between the environment information of the electronic device and the details of the control related to the presentation of the content by the electronic device, the content of the content by the electronic device associated with the acquired environment information is referred to A control processing unit that performs control related to presentation.

This produces the same effect as the control method of the electronic device.

Further, an electronic device according to an aspect of the present disclosure includes an acquisition unit that acquires environmental information of the electronic device, including information related to an environment around the electronic device, or information related to an installation mode of the electronic device, Control related to presentation of content by the electronic device associated with the acquired environment information by referring to a predetermined association between the environmental information of the electronic device and the details of control related to the presentation of content by the electronic device A control processing unit for performing

This produces the same effect as the control method of the electronic device.

Also, a program according to an aspect of the present disclosure is a program for causing a computer to execute the above-described electronic device control method.

This produces the same effect as the control method of the electronic device.

An electronic device according to an aspect of the present disclosure is an electronic device that reproduces audiovisual content in a home or a business facility, and includes a content management unit that manages the audiovisual content to be reproduced, and the audiovisual content displayed on the screen. An output unit for outputting as an audio signal, an upper speaker for outputting audio that is reproduced synchronously or asynchronously with audiovisual content to the outside, a lower speaker for outputting audio that is also reproduced synchronously or asynchronously to the outside downward, and the user From at least one or more users based on an upper transmission / reception unit that controls communication with external devices, a sensor that acquires information in a space in which the electronic device is disposed, and information detected by the sensor A control processing unit including a viewing environment estimation unit for estimating a usage environment state, and the control processing unit includes a viewing state estimation unit; Boss was in accordance with the state of the user or use environment, control processor of the electronic device comprises a control content determination means for determining a control content, the.

Further, the electronic device has any one of a function of generating video reproduction, audio reproduction, illumination dimming, or smell, and the control content determination unit includes a control processing unit including the viewing environment estimation unit. While the user is estimated to be in the wake-up state or when the user is assumed to change from the sleeping state to the wake-up state, the electronic device plays a predetermined video, plays a predetermined sound, and a predetermined illumination While the control processing unit including the viewing environment estimation means estimates that the user is sleeping or changes from the wake-up state to the sleeping state Then, when estimating, the electronic device stops the predetermined video or changes the content of the video, stops the predetermined audio or changes the content of the audio, stops the light output of the predetermined illumination, or sets the operation mode. Further, or changes the content of stopping or smell the occurrence of a predetermined odor.

The sensor is an imaging device provided in the electronic device or an imaging device outside the electronic device, and detects the presence of at least one user by analyzing an image acquired by the imaging device. The control processing unit including the viewing environment estimation unit analyzes the images acquired by the imaging device, thereby detecting the presence of at least one user and analyzing changes in a plurality of temporally different images. Thus, it is estimated whether or not the user is getting up.

Further, the sensor is a human sensor device provided in the electronic device, or a human sensor device outside the electronic device, and the control processing unit including the viewing environment estimating means is acquired by the human sensor device. By analyzing the sensor data, the presence of at least one user is detected, and by analyzing changes in a plurality of temporally different sensor data, it is estimated whether or not the user is getting up.

Also, the control processing unit including the viewing environment estimation unit estimates the presence and usage environment of the user, and the control content determination unit changes the control content of the electronic device according to the presence or usage environment of the user.

The electronic device includes an output unit that reproduces video and upper and lower speaker units that reproduce audio, and the control content determination unit includes a control processing unit including the viewing environment estimation unit. While the device is estimated to be in a state where the device is arranged at an angle, the audio output of the upper speaker and the lower speaker is changed and adjusted while the predetermined image is played on the electronic device. To play.

Note that these comprehensive or specific modes may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. You may implement | achieve in arbitrary combinations of a circuit, a computer program, or a recording medium.

Hereinafter, embodiments will be specifically described with reference to the drawings.

It should be noted that each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

In the following embodiments, the meanings of the following words are defined as follows unless otherwise specified.

“User” refers to one or more people who view an electronic device in an environment where the electronic device is placed. A user is also simply called a person.

“Usage environment” indicates a state in which an electronic device is placed in a space. The usage environment indicates, for example, a state of being directly placed on the floor or a state of being hung on a wall using a hook, and is also referred to as an installation mode. In addition, the usage environment includes a state that changes due to external factors such as illumination intensity in an external space. In addition, although the terms “device” and “electronic device” are described in the following description, they have the same meaning.

(Overview of services provided)
FIG. 1A shows an overall image of an electronic device and an electronic device system.

The group 100 is, for example, a company, an organization, a home, etc., and may be of any size. In the group 100, there are a device 101a, a device 101b, and a home gateway 102 included in a plurality of devices 101. The remote controller 103 is a remote controller that controls one or more devices 101. Even if the device 101 or the remote controller 103 is a device that cannot be connected to the Internet by itself, it may be connectable to the Internet via the home gateway 102. The group 100 includes a user 10 who uses a plurality of devices 101. Here, it has been described that the plurality of devices 101 include the device 101a and the device 101b, but the present invention is not limited to this. There may be many devices in the group 100, and the number of devices included in the plurality of devices 101 is not particularly limited. In the following embodiment, the devices 101a to 101e are described, but all of them are assumed to be included in the plurality of devices 101 illustrated in FIG.

The data center operating company 110 has a cloud server 111. The cloud server 111 is a virtualization server that cooperates with various devices via the Internet. The cloud server 111 mainly manages huge data (big data) that is difficult to handle with a normal database management tool or the like. The data center operating company 110 performs data management, management of the cloud server 111, and operation of the data center that performs them. Details of services performed by the data center operating company 110 will be described later. Here, the data center operating company 110 is not limited to a company that performs only data management or operation of the cloud server 111. For example, when a device manufacturer that develops and manufactures one of a plurality of devices 101 is also managing data or managing the cloud server 111, the device manufacturer corresponds to the data center operating company 110. (B in FIG. 1). The data center operating company 110 is not limited to one company. For example, when a device manufacturer and another management company jointly or share data management or operate the cloud server 111, both or one of them corresponds to the data center operating company 110 (FIG. 1). (C)).

The service provider 120 has a server 121. The server 121 stores a plurality of video contents including a tag managed as a set with a video and a plurality of sound contents including a tag managed as a set among the video and audio contents reproduced by the device 101. It shall be. Tags that are managed as a set with video are generally expressed in natural language that clearly shows the content of the video, but are not limited to natural language expression, and natural language hash tags such as high-speed search of search words It is assumed that it will be managed and operated, including those in the binary data representation. Further, the tag managed by the sound and the set is the same tag as the tag managed by the video and the set, and the tag related to the content of the sound is assumed. The server 121 referred to here is not limited in size, and includes, for example, a memory in a personal PC. In some cases, the service provider 120 does not have the server 121.

Note that the home gateway 102 is not essential in the above service. For example, when the cloud server 111 manages all data, the home gateway 102 becomes unnecessary. In addition, there may be no device that cannot be connected to the Internet by itself, as in the case where every device in the home is connected to the Internet.

Next, the flow of information in the above service will be described.

First, the device 101a or the device 101b of the group 100 transmits each log information to the cloud server 111 of the data center operating company 110. The cloud server 111 accumulates log information of the device 101a or the device 101b ((a) in FIG. 1). Here, the log information is information on the plurality of devices 101, for example, information indicating the driving status or operation date and time, information on the peripheral state of the device or the surrounding environment acquired from a camera or sensor mounted on the device. For example, as a result of analyzing an image around a device by a camera mounted on an electronic device in a state in which personal information is deleted, more specifically, which audiovisual content the user has watched on the screen side and watched in the video Log, history of whether the user was around the device by the presence sensor installed in the electronic device, or the playback history of the audiovisual content of the device or the periodicity of the lighting environment around the device by the illuminance sensor mounted on the device Although it is log information etc., it refers to all information that can be acquired from any device arranged in the electronic device main body or the peripheral space facility. The log information may be provided directly to the cloud server 111 from a plurality of devices 101 themselves via the Internet. In addition, log information may be temporarily accumulated in the home gateway 102 from a plurality of devices 101 and provided to the cloud server 111 from the home gateway 102.

Next, the cloud server 111 of the data center operating company 110 provides the collected log information to the service provider 120 in a certain unit. Here, the certain range may be a unit in which information collected by the data center operating company 110 can be organized and provided to the service provider 120, or a unit requested by the service provider 120. Although described as a fixed unit, it may not be constant in time, and the amount of information to be provided may change depending on the situation. The log information is stored in the server 121 held by the service provider 120 as necessary ((b) of FIG. 1). Then, the service provider 120 organizes the log information into information suitable for the service provided to the user, and provides the information to the user. The providing user may be a user 10 who uses a plurality of devices 101, or a creator 20 that produces audiovisual content. In addition, the service provision method to a user may be directly provided to a user from the service provider 120 ((f) and (e) of FIG. 1), for example. The service providing method to the user may be provided to the user via the cloud server 111 of the data center operating company 110 again ((c) and (d) in FIG. 1), for example. Alternatively, the cloud server 111 of the data center operating company 110 may organize the log information into information suitable for the service provided to the user and provide the information to the service provider 120.

Note that the user 10 and the creator 20 may be different people or the same person. Here, the creator 20 can use a creator community service that can be provided on the server 121 of the service provider 120, and the creator 20 registers and manages video and audio synchronously or asynchronously with respect to the video and audio content produced by the creator 20 itself. And whether to distribute to the device 101 can be set. Here, it has been described that the creator 20 can set whether or not distribution is possible for the sake of simplification. However, as another form of implementation, there is an administrator who manages the creator 20 in the creator community service, and the administrator It can also be considered to set the availability. In addition, the creator community service uses log information obtained from the device 101, provides an analysis result of log information linked to the audiovisual content registered in the creator 20, and improves or newly adds the registered audiovisual content. Feedback can be obtained and used in real time as reference information for the production of audiovisual content to be registered. The analysis result of the log information includes the evaluation result by the user of the reproduced audiovisual content, the reproduction time zone or reproduction time, the reproduction log indicating the point switched to another audiovisual content, and the location facility where the device 101 is arranged. Information, sound volume at the time of reproduction, illuminance used for reproduction setting, a gaze log indicating the degree of gaze by analyzing a person's line of sight or facial expression, and the like are included.

(Embodiment 1)
In this embodiment, an electronic device that controls the presentation of content according to the environment around the electronic device will be described.

The electronic device according to the present embodiment includes an acquisition unit that acquires environmental information about the electronic device, including information about the environment around the electronic device, or information about an installation mode of the electronic device, environmental information about the electronic device, A control processing unit that performs control related to the presentation of content by the electronic device, which is associated with the acquired environment information, by referring to a predetermined association with the content of the control related to the presentation of content by the electronic device.

In addition, the function of the electronic device can be shared by a plurality of devices connected to be communicable with each other, so that it can be configured as a control system. That is, the control system according to the present embodiment is a control system for an electronic device, and acquires environmental information about the electronic device including information about the environment around the electronic device or information about the installation mode of the electronic device. By referring to a predetermined association between the acquisition unit, the environment information of the electronic device, and the details of the control related to the presentation of the content by the electronic device, the presentation of the content by the electronic device associated with the acquired environment information A control processing unit that performs control.

In addition, the electronic device control method according to the present embodiment obtains electronic device environmental information including information related to the environment around the electronic device or information related to the installation mode of the electronic device, and the electronic device environmental information. By referring to a predetermined association with the details of the control regarding the presentation of content by the electronic device, the control regarding the presentation of the content by the electronic device associated with the acquired environment information is performed.

Hereinafter, the configuration of the electronic device and the control method of the electronic device will be described in detail. Note that an electronic device is also simply referred to as a device.

[Device configuration]
FIG. 2 is a schematic diagram showing an appearance of the device 101 according to the present embodiment. FIG. 3 is a block diagram showing a configuration of the device 101 according to the present embodiment. FIG. 4 is a detailed block diagram of the control processing unit according to the present embodiment. FIG. 5 is a detailed block diagram of the content management unit according to the present embodiment. 6 and 7 are schematic diagrams illustrating an example of an installation mode of the device 101 according to the present embodiment.

In the present embodiment, an example of a device equipped with a speaker and a display will be described and described as the device 101 in FIG. 1, but the present invention is not limited to this.

2A is a front view of the device 101. FIG. FIG. 2B is a side view of the device 101. FIG. 2C is a rear view of the device. As an example of the installation mode of the device 101, FIG. 6 shows an installation mode in which the device 101 is installed against a wall. As another example of the installation mode of the device 101, FIG. 7 shows an installation mode in which the device 101 is suspended from a wall.

2A and 3, the device 101 includes a sensor 201, an output unit 202, a control processing unit 203, a content management unit 204, an upper speaker 205, and a lower speaker 206. And an information transmission / reception unit 207. For the device 101, as shown in FIG. 2, a vertical direction, a horizontal direction, and a front-rear direction are defined.

The sensor 201 is an example of an acquisition unit that acquires environment information of the device 101 including information related to the environment around the device 101 or information related to the installation mode of the device 101. More specifically, the sensor 201 is a sensor that senses the environment around the device 101 and outputs an environmental value indicating the environment. The sensor 201 includes a camera, a button, or a sensor for detecting the posture of the device 101.

2A, a sensor 201F as an example of the sensor 201 is disposed in a part of the device 101. The position where the sensor 201F is disposed is, for example, the upper part of the device 101, but is not limited thereto.

The sensor 201F is, for example, a camera that detects a two-dimensional image as an environmental value. The two-dimensional image detected by the sensor 201F is a two-dimensional visible image or a two-dimensional infrared image (also referred to as a thermal image). The sensor 201F detects a two-dimensional image in the front direction when viewed from the device 101, and outputs the two-dimensional image to the control processing unit 203 described later. The sensor 201F is a sensor that senses the environment around the device 101 and outputs an environmental value indicating the environment. The sensor 201F includes, as information about the environment around the device 101, the presence / absence of people around the device 101, the orientation of people around the device 101, the direction of people around the device 101, and the brightness around the device 101. Alternatively, environmental information including information indicating the volume around the device 101 is acquired. The environmental information includes the presence / absence of people around the device 101, the direction of people around the device 101, the direction of people around the device 101, the brightness around the device 101, and the surroundings of the device 101 Information indicating at least two of the volume may be included. In this case, control relating to presentation of content by the device 101 is performed in a composite manner in accordance with information indicating the at least two.

Note that the sensor 201F does not necessarily have to detect a two-dimensional image. The sensor 201F may be a human sensor that detects only the presence or absence of a person or an object, or a sensor that detects a state of a space, such as a sensor that detects illuminance or a sensor that detects a stagnant odor component. It may be a thing. For example, if the sensor 201F is an odor sensor, fruits (apples, oysters, oranges, bananas, etc.), plants (roses, lilies, phalaenopsis etc.), perfumes or body odors used by humans, or spaces Can detect fragrances used in In this case, the device 101 can grasp human comfort or discomfort with respect to the space by smell, and controls the video / audio content related to the detected smell to reproduce the image related to the fruit or plant. Alternatively, it is possible to enhance the spatial value by enhancing the relaxation effect with sound and selecting audiovisual content with values close to the perfume used.

Further, the sensor 201F may be a microphone that picks up sounds around the device 101.

Note that the device 101 may include an information acquisition unit (not shown) that acquires information from a website on the Internet or the like instead of the sensor 201 or together with the sensor 201. And the apparatus 101 may acquire the information regarding a weather by the information acquisition part from the website on the internet. The acquired information can be used for control related to the presentation of content instead of the environmental value or together with the environmental value.

The output unit 202 is, for example, a liquid crystal panel disposed in front of the device 101 as illustrated in FIG. 2A, and an image or moving image that is information output from the control processing unit 203 described later is displayed on the user 10. Output or display for.

The control processing unit 203 refers to a predetermined association between the environment information of the device 101 and the details of the control related to the presentation of the content by the device 101, so that the content of the content by the device 101 associated with the acquired environment information is referred to. It is a process part which performs control regarding presentation. For example, a CPU (Central Processing Unit) or a microcomputer provided in the device 101 is used.

The control processing unit 203 acquires information from the sensor 201F, the output unit 202, or the content management unit 204, and performs processing such as determination based on the acquired information. The control processing unit 203 outputs information to the sensor 201F, the output unit 202, or the content management unit 204, and controls each configuration. FIG. 4 shows a part of the detailed functional configuration of the control processing unit 203. As shown in FIG. 4, the control processing unit 203 includes a viewing environment estimation unit 203A and a control content determination unit 203B. Processing of each functional configuration will be described later. Note that a part of the function of the control processing unit 203, for example, a function of executing part or all of the processing of the viewing environment estimation unit 203A may be included in a camera or a sensor which is an example of the sensor 201F.

The content management unit 204 temporarily records or accumulates information output from each configuration, information acquired from the outside, and the like. The content management unit 204 is, for example, a memory provided in the device 101.

FIG. 5 shows a part of the detailed functional configuration of the content management unit 204. As shown in FIG. 5, the content management unit 204 includes a viewing environment storage DB (database) 204A and a content management DB (database) 204B. Features of each functional configuration will be described later. Note that the content management unit 204 is not an essential component. For example, an external device other than the device 101 includes one or both of the viewing environment storage DB 204A and the content management DB 208 of the content management unit 204, and the device 101 and the external device exchange information by wired communication or wireless communication. May be substituted.

The upper speaker 205 is a speaker arranged at the upper part of the main body of the device 101.

The lower speaker 206 is a speaker disposed at the lower part of the main body of the device 101.

The sensor 201 may be provided with sensors 201BH and 201BL that detect the installation on the floor in a pressure sensitive manner at the upper and lower ends of the back surface of the main body. The sensors 201BH and 201BL function as sensors that detect the posture of the device 101. The sensors 201BH and 201BL output an ON signal when pressed or touched by an object, and output an OFF signal otherwise. The ON signal or OFF signal corresponds to the environmental value.

As shown in FIG. 6, when the main body of the device 101 is leaned against the wall, when the device 101 is placed on the floor, the sensor 201BL can be detected by contacting the floor. . In addition, although the example which detects with a pressure-sensitive sensor was shown here, the detection system of a sensor is not restricted to this, You may use another means like an electrostatic capacitance type or an illumination intensity collation system.

Note that the orientation of the device 101 can be detected using a three-axis sensor instead of the sensors 201BH and 201BL. The triaxial sensor detects angles with respect to predetermined three axes orthogonal to each other. Using this detected angle, an angle with respect to an arbitrary axis or surface can also be calculated. For example, the rising angle θ from the horizontal plane of the device 101 can be calculated using the angles with respect to the three axes. The posture of the device 101 can be expressed using the rising angle θ. The rising angle θ is an example of an environmental value.

Further, as shown in FIG. 2C, the device 101 has a hook hanging hole H1, and the hook F1 fixed to the wall is inserted into the hook hanging hole H1, so that the device 101 is placed on the wall. It can also be installed so as to be hung on and suspended (see FIG. 7). In this case, since the wall or floor is not detected by the sensor 201BH and the sensor 201BL, it can be determined that the device 101 is suspended from the wall.

Note that, as shown in FIG. 6, when the device 101 is installed leaning against a wall, the volume of the sound C206 output by the lower speaker 206 may be larger than the sound C205 output by the upper speaker 205. This is to provide the user 10 with an appropriate viewing environment by adjusting the volume in this way.

The device 101 may further include a log output unit (not shown) that outputs log information related to content reproduction. The log information output by the log output unit can be used as log information fed back to the creator 20.

FIG. 8 is a perspective view showing an appearance of the device 101 according to the present embodiment. FIG. 9 is an exploded perspective view of the device 101 according to the present embodiment. The configuration shown in FIGS. 8 and 9 will be described in detail below.

The device 101 includes an exterior frame P101, a structural frame P102, a back cover P103, a decorative member P104, a speaker net P111, a speaker base plate P112, a speaker P113, a cover glass P121, and a display panel P122. .

The exterior frame P101 is an exterior frame formed of a metal material such as aluminum. The exterior frame P101 may have a detachable structure. In this case, there is an advantage that the appearance of the device 101 can be changed independently of the display panel P122 and the like. Note that an LCD (Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) may be provided in the exterior frame P101.

The structural frame P102 is a structural frame formed of a metal material such as aluminum. The structural frame P102 is a framework for maintaining the mechanical structure of the device 101.

The back cover P103 is a cover as an enclosure that covers the components of the device 101 from the back side.

The decoration member P104 is a member for decoration of the back cover P103. The decorative member P <b> 104 is an arbitrary member such as cloth (fabric), metal, or wood, and is a member that produces the appearance of the device 101. The decorative member P104 can be used, for example, to ensure a beautiful appearance by hiding a functional shape such as a heat radiation hole from the outer surface. The decorative member P104 may be configured to be removable and customized according to the user's 10 preference.

The speaker net P111 is a cover member that covers the speaker P113 from the front side. The speaker net P111 is, for example, provided with fine holes in metal or resin, and makes it difficult for the speaker P113 to be visually recognized from the outer surface. In addition, the speaker net P111 can be made harder to visually recognize the internal structure by putting a ridge on the back surface. The speaker net P111 is preferably made of a sound-transmitting material, and may be customized such as using Fabric.

The speaker base plate P112 is a plate body that holds the speaker P113 and the speaker net P111.

Speaker P113 is mounted as upper speaker 205 at two locations on the upper left and right when device 101 is viewed from the front side, and as lower speaker 206 at two locations on the left and right of the lower portion. The speaker P113 outputs sound from a low frequency range of about 10 Hz to a high frequency range of about 100 kHz.

The cover glass P121 is a cover member that covers the display panel P122 from the front side. The cover glass P121 is directly bonded to the display panel P122 and bonded without an air layer, thereby achieving both a uniform appearance on the surface and display performance. In addition, by devising the glass shape of the cover glass P121, it is possible to cope with black matrixless display of an image up to the end of the cover glass P121.

The display panel P122 is a display panel that displays an image. The display panel P122 is, for example, an LCD or an OLED. The display panel P122 corresponds to the output unit 202.

As mentioned above, although each structure was demonstrated using FIGS. 2-9, the apparatus 101 does not need to be equipped with all these structures. That is, an external device may include a part of these configurations, and may be substituted by exchanging information with the device 101. The device 101 may have a configuration other than the above-described configurations.

[Device operation and processing flow]
FIG. 10 is a flowchart showing processing of the device 101 according to the present embodiment.

In step S 401, the sensor 201 detects a two-dimensional image such as a forward visible image or a thermal image of the device 101, and outputs the detected two-dimensional image to the control processing unit 203. Here, the detection timing or frequency of the two-dimensional image is not particularly limited. For example, it may be detected at predetermined intervals such as 5 minutes or 10 minutes, or the control processing unit 203 may be controlled to increase the detection frequency when the presence of the user 10 is detected. Note that the information acquired by the sensor 201 is not necessarily information related to the two-dimensional image. For example, the sensing information in the space which the sensor 201 acquired may be sufficient.

In step S402, the control processing unit 203 determines whether an operation has been received from the user 10. If an operation from the user 10 is accepted (Yes in step S402), the process proceeds to step S421. If not (No in step S402), the process proceeds to step S403.

In step S403, the control processing unit 203 determines whether or not to transition to the auto mode. Here, the auto mode is a mode in which the viewing environment is estimated based on the detection result of the sensor 201 and the control content is determined (a mode in which operations in steps S411 to S413 described later are performed). Whether or not to transition to the auto mode can be determined based on whether or not the operation received in step S402 is a predetermined operation for transitioning to the auto mode, for example, but is not limited thereto. . If it is determined to transition to the auto mode (Yes in step S403), the process proceeds to step S411. If not (No in step S403), the process proceeds to step S404.

In step S404, the control processing unit 203 reproduces the content in the normal mode. Here, the normal mode is a mode different from the auto mode, and is a mode in which one or more predetermined contents are reproduced. In the normal mode, one or more contents may be reproduced in a predetermined order, may be reproduced randomly, or may be reproduced in any other manner.

In step S405, the control processing unit 203 determines whether an instruction to turn off the power of the device 101 has been received. If the above instruction has been received (Yes in step S405), the series of processes shown in FIG. If the above instruction has not been received (No in step S405), step S402 is executed again.

In step S411, the viewing environment estimation unit 203A of the control processing unit 203 operates the sensor 201.

In step S <b> 412, the viewing environment estimation unit 203 </ b> A of the control processing unit 203 determines whether there is the user 10 in the space based on the two-dimensional image or sensing information that is output from the sensor 201, or the illuminance of the space Estimate the state of the space from the viewpoint of whether or not is below the specified value. The information estimated by the viewing environment estimation unit 203A is, for example, information related to the environment around the device 101, and the presence or absence of a person around the device 101, the orientation of the person around the device 101, or the brightness around the device 101. This is information indicating the above. Detailed processing in step S412 will be described later.

In step S413, the control content determination unit 203B of the control processing unit 203 determines whether the device 10 has been estimated by the viewing environment estimation unit 203A in step S402 according to the presence / absence of the user 10, the number of users 10, or the state of the user 10. The control content of 101 is determined. Detailed processing in step S413 will be described later.

In step S421, the control processing unit 203 receives a channel designation. Here, the channel means a set of contents related to a specific keyword or genre, or a set of contents provided by a specific provider. For example, the control processing unit 203 receives a keyword or a genre designated by the user 10 through an operation with the remote controller 103.

In step S422, the control processing unit 203 changes the content playback setting based on the channel designation received in step S421.

When step S406, S413, or S422 is finished, the series of processes shown in FIG.

Next, detailed processing in step S412 in FIG. 10 will be described with reference to four examples.

FIG. 11 is a flowchart showing a first example of the viewing environment determination process according to the present embodiment. The process shown in FIG. 11 is an example of the process included in step S412 of FIG.

In step S <b> 501, the viewing environment estimation unit 203 </ b> A of the control processing unit 203 analyzes information such as a two-dimensional image or sensing information output from the sensor 201, so that the presence or absence of the user 10, the number of users 10, and the like Judging.

There is no particular limitation on the method of determining the presence or absence of the user 10 or the number of users 10. For example, it may be determined whether or not the user 10 exists from changes in a plurality of images obtained at different times, that is, differences between the images. The acquired time is also referred to as an acquisition time. A change in a plurality of images having different acquisition times, that is, a difference between images indicates, for example, a pixel region or a difference region in which a predetermined luminance value changes between a plurality of images within a predetermined time by a threshold value or more. The difference area can be estimated to be an area with motion in the image. Thus, by detecting the difference between a plurality of images having different acquisition times, it is possible to specify the presence of the user 10 having movement in the space. Moreover, the number of users 10 can be specified by measuring the number of areas in which differences are detected in the image. As another method for determining the presence or absence of the user 10, for example, in the case of a visible image, the region of the user 10 in the image may be specified from color information, luminance characteristics, shape characteristics, or the like in the image. In the case of a thermal image, the area of the user 10 in the image may be specified based on information about the temperature in the image.

In step S502, the viewing environment estimation unit 203A determines whether or not the user 10 exists in the space from which the image is acquired based on the determination result in step S501, and branches the subsequent processing. If the user 10 exists (Yes in step S502), the process proceeds to step S503. On the other hand, if the user 10 does not exist (No in step S502), the process of step S412 is exited, and the viewing environment estimation unit 203A outputs information indicating that the user 10 does not exist to the control content determination unit 203B.

In step S503, the viewing environment estimation unit 203A specifies whether or not the line of sight of the user 10 is facing the device 101 with respect to the area specified as the user 10 in the image. Here, the method for specifying whether or not the device 101 is suitable is not particularly limited.

For example, the viewing environment estimation unit 203A analyzes facial elements for the area identified as the user 10 in step S501 in the image, and determines whether the line of sight of the user 10 is facing the device 101. May be. In this case, it is assumed that a general line of sight for the user 10 in the image can be determined in advance, for example, when the device 101 is shipped. In addition, it is desirable to improve the recognition rate that the characteristics of the user 10 living in a specific environment are recorded in the content management unit 204 or the like. Further, the user 10 who uses the device 101 and the characteristics of the user 10 may be acquired and held in advance, and it may be determined whether or not the user 10 is a specific person by comparing the user 10 in the image with the specified area. . This advance acquisition is also referred to as advance registration or calibration. This further improves the detection accuracy.

In step S504, the viewing environment estimation unit 203A sets the state of the user 10 specified in step S503 as the estimation result of the viewing environment of the device 101. A specific example of the state of the user 10 is a state relating to the gaze indicating the degree of gaze.

Here, the method for estimating the state of the user 10 is not particularly limited. For example, by analyzing changes in a plurality of images having different acquisition times, it may be estimated whether the user 10 is watching the device 101 or accidentally facing the device 101. In this case, if the temporal change is large, it is estimated as “accidental”, and if the temporal change is small, it is estimated as “gazing”. In addition, when the sensor 201 is a sensor that does not acquire a two-dimensional image, whether the sensor 201 is in a gaze state based on temporal changes in sensor information acquired by the sensor, or other states (states including sleep etc.) ) May be estimated.

After executing the process of step S504, or after determining that the user 10 does not exist in step S502 (No in step S502), the process of step S412 is completed and the process proceeds to step S413.

When it is determined in step S502 that the user 10 is present (“Yes” in step S502), when the process of step S504 is executed and the process of step S412 is completed, the viewing environment estimation unit 203A controls the control content determination unit 203A. 203B includes information on (i) the type of the user 10 or the presence or absence of a person, (ii) the number of the existing users 10, and (iii) the state of the existing users 10 (gaze or other: sleep, etc.). At least one of the information is output. Here, the type of user means the gender or age of the user, and in the case of use in a private house, whether it is the owner, the husband or the guest of the private house. Means.

When it is determined in step S502 that the user 10 does not exist (“No” in step S502), and the processing in step S412 is completed, the viewing environment estimation unit 203A sends (iv) space to the control content determination unit 203B. The information indicating that the user 10 does not exist is output.

FIG. 12A is a flowchart showing a second example of the viewing environment determination process according to the present embodiment. The process shown in FIG. 12A is an example of the process included in step S412 of FIG.

In step S <b> 511, the viewing environment estimation unit 203 </ b> A of the control processing unit 203 analyzes information such as a two-dimensional image or sensing information output from the sensor 201, so that the presence or absence of the user 10, the number of users 10, and the like Judging.

In step S512, the viewing environment estimation unit 203A acquires the illuminance around the device 101 using the illuminance sensor as the sensor 201, and determines whether the acquired illuminance is equal to or less than a specified value. Here, the prescribed value of illuminance is, for example, about 300 to 500 lux corresponding to a dimly lit room. If the illuminance is less than or equal to the specified value (Yes in step S512), the process proceeds to step S513. If not (No in step S512), the series of processes illustrated in FIG.

In step S513, the viewing environment estimation unit 203A uses the illuminance detected in step S512 as the estimation result of the viewing environment of the device 101. When step S513 is completed, the series of processes shown in FIG. 12A is ended.

FIG. 12B is a flowchart showing a third example of the viewing environment determination process according to the present embodiment. The process shown in FIG. 12B is an example of the process included in step S412 of FIG.

In step S <b> 521, the viewing environment estimation unit 203 </ b> A of the control processing unit 203 analyzes information such as a two-dimensional image or sensing information output from the sensor 201, so that the presence or absence of the user 10, the number of users 10, and the like Judging.

In step S522, the viewing environment estimation unit 203A determines whether the user 10 exists in the space from which the image is acquired based on the determination result in step S521, and branches the subsequent processing. If the user 10 exists (Yes in step S522), the process proceeds to step S523. On the other hand, when the user 10 does not exist (No in step S522), the process of step S412 is exited, and the viewing environment estimation unit 203A outputs information indicating that the user 10 does not exist to the control content determination unit 203B.

In step S523, the viewing environment estimation unit 203A identifies in which direction the user 10 is present as viewed from the device 101 with respect to the area identified as the user 10 in the image. Here, the method for specifying in which direction the user 10 exists is not particularly limited.

For example, the viewing environment estimation unit 203A analyzes the position where the area identified as the user 10 in step S521 in the image exists in the entire image, so that the user 10 can see from the device 101. It may be determined in which direction.

In step S524, the viewing environment estimation unit 203A uses the information specified in step S523, which indicates in which direction the user 10 is present as viewed from the device 101, as the estimation result of the viewing environment of the device 101.

After performing the process of step S524, or after determining that the user 10 does not exist in step S522 (No in step S502), the process of step S412 is completed and the process proceeds to step S413.

FIG. 12C is a flowchart showing a fourth example of the viewing environment determination process according to the present embodiment. The process shown in FIG. 12C is an example of the process included in step S412 of FIG.

In step S531, the viewing environment estimation unit 203A of the control processing unit 203 analyzes the information such as the two-dimensional image or the sensing information output from the sensor 201, so that the presence or absence of the user 10, the number of the users 10, etc. Judging.

In step S532, the viewing environment estimation unit 203A acquires a sound volume (also referred to as a sound collection volume) collected from around the device 101 by the microphone serving as the sensor 201, and the acquired sound collection volume is equal to or less than a specified value. It is determined whether or not. Here, the specified value of the sound collection volume is, for example, about 30 to 40 dB corresponding to a relatively quiet room. If the illuminance is less than or equal to the specified value (Yes in step S532), the process proceeds to step S533, and if not (No in step S532), the series of processes illustrated in FIG.

In step S533, the viewing environment estimation unit 203A sets the sound collection volume detected in step S532 as the estimation result of the viewing environment of the device 101. When step S533 is completed, the series of processing illustrated in FIG. 12C is terminated.

FIG. 13 is a flowchart showing an example of the control content determination process according to the present embodiment. Details of the control content determination unit 203B included in the control processing unit 203 will be described with reference to FIG.

In step S601, the control content determination unit 203B acquires a table for determining the control content from the content management unit 204. The table is a table shown in FIGS. 16 to 20 (described later).

In step S602, the control content determination unit 203B refers to the table acquired in step S601, and determines the control content related to the content presentation based on the viewing environment estimated in step S412.

In step S603, the control content determination unit 203B performs control related to content presentation based on the control content determined in step S602. Specifically, the control content determination unit 203B acquires the content determined in step S602, outputs the acquired content by the output unit 202, and presents it to the user 10.

In the control related to the presentation of content, for example, by referring to a predetermined association, the content to be presented by the device 101, which is the content of the control associated with the environment information acquired by the sensor 201, is selected and selected. You may perform control which shows the content which was performed.

FIG. 14 is an explanatory diagram showing the state of the space and an example of output information in the present embodiment. More specifically, in FIG. 14, a diagram showing the state in the space estimated by the viewing environment estimation unit 203A through the processing from steps S501 to S504 is shown in the upper stage, and is output to the control content determination unit 203B. The figure which shows the example of the information to perform is shown in the lower stage. FIG. 14A shows a state in which a user 10 and one dog who are getting up simultaneously exist in the space. FIG. 14B shows a state in which the user 10 who is getting up and two dogs are present in the space. The viewing environment estimation unit 203A outputs at least one of the information (i) to (iv) to the control content determination unit 203B based on the estimation from these states.

In step S504, the viewing environment estimation unit 203A may estimate other information instead of the state of “wake up” or “gaze”, or in addition to the state of “wake up” or “gaze” Other information may be estimated.

For example, in the case of (a) and (b) in FIG. 14, the viewing environment estimation unit 203 </ b> A estimates the orientation of the user 10 with respect to the device 101. Here, the orientation of the user 10 is, for example, front or side orientation. A method for estimating the orientation of the user 10 can employ a known method and is not particularly limited. For example, the orientation of the user 10 may be estimated by employing a face recognition technique or the like and estimating the orientation of the face. By estimating the direction, it is possible to estimate whether or not the user 10 is facing the direction of the device 101, that is, whether or not the user 10 is interested in the device 101. In the control content determination, control more suitable for the user 10 or control more suitable for the state of the user 10 can be determined. Therefore, when it is estimated that the user 10 exists in the space, it is preferable to additionally estimate the direction of the user 10.

Further, for example, as shown in FIG. 14A, the action of the user 10 may be estimated. Here, the action of the user 10 is during reading, eating, or walking. As a method for estimating the behavior, a known method can be adopted, and there is no particular limitation. By calculating the difference between a plurality of images having different acquisition times, it can be determined whether the user 10 is moving or stationary, so that it is possible to estimate an action such as reading or walking. Further, when the user is stationary at a predetermined place, an action such as whether he is eating or reading may be estimated depending on the place. It is possible to estimate whether the user 10 is interested in the device 101 also by estimating the behavior. Moreover, since the mental state of the user 10 can also be estimated, in the control content determination in the control content determination unit 203B described later, it is possible to determine a control more suitable for the user 10 or a control more suitable for the state of the user 10. . Here, the mental state of the user 10 is a mental state in which the user 10 wants to calm down or an active mental state. Therefore, when there is a user 10 getting up, it is preferable to additionally estimate the behavior.

FIG. 15 is an explanatory diagram of a table showing the estimation results of the person and space states in the present embodiment. More specifically, FIG. 15 shows an example of a table 801 in which the viewing environment estimation unit 203A manages information output to the control content determination unit 203B. These pieces of information may be stored as log information in the viewing environment storage DB 204A of the content management unit 204 as a table 801, or may be temporarily recorded and appropriately deleted. When it is stored as log information in the viewing environment storage DB 204A, a pattern related to the state of the user 10 in the space can be learned to some extent. Therefore, it is possible to contribute to a reduction in the amount of information to be processed or an improvement in estimation accuracy of the viewing environment estimation unit 203A. Further, when the viewing environment accumulation DB 204A manages the table in this way, the information estimated by the viewing environment estimation unit 203A is not directly output to the control content determination unit 203B, but the estimated information is not output directly to the content management unit. You may make it output once to 204 viewing-and-listening environment storage DB204A. In step S413, the control content determination unit 203B may acquire information from the viewing environment accumulation DB 204A and determine the control content.

In the example of FIG. 15, the detection result is output from the sensor 201 to the viewing environment estimation unit 203A of the control processing unit 203 every 15 minutes, and the estimation result of the user state every 15 minutes is tabulated. Note that the frequency detected by the sensor 201 and the frequency estimated by the viewing environment estimation unit 203A are not limited to 15 minutes. Further, in the table 801 in FIG. 15, an example of log information corresponding to each state illustrated in FIG. 14 is described as “state example”. For example, the state of FIG. 14A corresponds to the log 3 of FIG. 15, and there is one user 10, and one user 10 is getting up and reading in the sideways direction. Is managed in the table 801 of the viewing environment storage DB 204A. This “example of state” is described for explanation, and need not be stored in the table 801.

As described above, the viewing environment estimation unit 203A estimates whether or not the user 10 exists in the space, and if the user 10 exists, the number and state of the user 10 are estimated and output to the control content determination unit 203B. To do. Note that the viewing environment estimation unit 203A does not necessarily have to execute the processes in the order of steps S501 to S504. Some steps may be omitted. Moreover, you may implement each step collectively. That is, the viewing environment estimation unit 203A may use different means regarding the processing order or content as long as it can output at least one of the information (i) to (iv) to the control content determination unit 203B as a result. .

Hereinafter, a process for determining the control content will be described with reference to an example of a table for determining the control content managed by the content management unit 204.

FIG. 16 is an explanatory diagram of a table 901A for determining control contents using a camera-side determination according to the present embodiment.

The table 901A is an example of a predetermined association. The table 901A includes correspondence between information indicating a change in the environment information of the device 101 and the start, stop, or end of content presentation. In the control related to content presentation, the table 901A is referenced to start, stop, or end the presentation of content to be presented by the device 101, which is associated with the acquired change in the environment information.

More specifically, the table 901 </ b> A is an example of a table in which a camera is mounted as the sensor 201 and an operation to be executed by the output unit 202 of the device 101 in accordance with the look-ahead behavior as the state of the user 10.

When the control content determination unit 203B refers to the table 901A, when the viewing environment estimation unit 203A determines that the person is “not present” or “sleep” in the previous state and the rear state, the control content determination is performed. The unit 203B determines an operation of “stop reproduction if there is no change over a predetermined time” as an output operation of the device 101. Here, the case where it is determined that the person is “not present” or “sleeping” is, for example, the case where it is determined in step S502 in FIG. 11 that the user 10 does not exist. The “previous state” and “rear state” refer to a predetermined time point before the determination time and a predetermined time point after the determination time, respectively.

When the viewing environment estimation unit 203A determines that there is a person and that “front” continues in the front state and the rear state, the control content determination unit 203B sets “ “Continue playback of audiovisual content” operation is determined. At this time, based on the fact that the user 10 viewing the content is facing the front toward the device 101, it is determined that the user 10 is watching the content while watching the content, and the video and audio You may improve the attention point with respect to a content. This point of interest is managed in association with the audiovisual content and can be fed back to the creator 20 that produced the audiovisual content. In addition, not only the point of interest, but also any information that accompanies viewing by the user 10 such as the volume when the audiovisual content is reproduced or the total reproduction time is recorded. The creator 20 that produced the video can be used as an improvement element of the video / audio content or as an examination element of the video / audio content to be produced next time.

If the viewing environment estimation unit 203A determines that there is a person and the state of facing sideways continues between the front state and the rear state, the control content determination unit 203B As the output operation 101, “continue reproduction of the audiovisual content” operation is determined. At this time, the feedback can be recorded as in the case of “continuation of the state of facing the front”. When the user 10 continues to face sideways, it is considered that the user 10 recognizes that the content is a comfortable BGV (Back Ground Visual), and the cumulative time of the state facing sideways is the creator. 20 can be fed back.

When the viewing environment estimation unit 203A changes the state from “front or side” to “none (does not exist)”, the control content determination unit 203B sets “continue playback of audiovisual content” as the output operation of the device 101. "Determine the action. This is because there is a possibility that the fact that the user 10 no longer exists is due to a temporary exit, so that it is assumed that the user 10 will return after a relatively short time, and that the operation is performed as a state-of-view. I can say that.

On the contrary, when the user 10 has not existed for a while and the reproduction of the audiovisual content has been stopped, the state has changed from the previous state “none (not present)” to the rear state “front or side”. The control content determination unit 203 </ b> B determines the “resumption of reproduction of audiovisual content that has been stopped” operation as the output operation of the device 101.

When both the previous state and the rear state are “none (does not exist)”, if there is no change from this state for a specified time or longer, the control content determination unit 203B performs “stop playback” as the output operation of the device 101. Determine the behavior.

Note that the table acquired by the control content determination unit 203B to determine the control content is not limited to the table 901A.

FIG. 17 is an explanatory diagram of a table 901B for determining control content using installation environment determination by an illuminometer in the present embodiment. The table 901B is an example of predetermined association. The table 901B includes correspondence between information indicating changes in the environment information of the device 101 and changes in the volume or brightness of the content. In the control related to the presentation of content, the volume or brightness of the content to be presented by the device 101 associated with the change in the acquired environment information is changed by referring to a predetermined association.

The table 901 </ b> B includes an illuminometer as the sensor 201, and more specifically, an operation executed by the output unit 202 of the device 101 in accordance with an environment in which the device 101 is installed, more specifically, a change in brightness around the device 101. It is an example of a table that defines

The table 901B manages the previous state and the rear state in the same manner as the table 901A.

When the ambient brightness of the device 101 changes from “low”, that is, a dark state, to “medium or high”, that is, a relatively bright state, the control content determination unit 203B determines that “in conjunction with the change in ambient brightness, the user 10 is switched to a tone-up audiovisual content that tends to feel light, and the operation of “lightening the backlight of the output unit 202 as necessary” is determined. The brightness around the device 101 may change as described above, for example, when the user 10 wakes up or when the user 10 enters a room where the user 10 is absent. At this time, the record of the illuminance log as to how much the illuminance as the reproduction environment was can be recorded as a feedback record, and the creator 20 can arbitrarily utilize it. Similarly, the output is changed in accordance with the change between the previous state and the rear state.

When the ambient brightness of the device 101 changes from “high”, that is, a bright state, to “medium or low”, that is, a relatively dark state, the control content determination unit 203B determines “in conjunction with a change in ambient brightness. Then, the operation is switched to calm audiovisual content, and the backlight of the output unit 202 is dimmed if necessary.

Also, for example, when the brightness of the surroundings of the device 101 is “low”, that is, a dark state is maintained, the control content determination unit 203B determines the “stop playback if there is no change over a specified time” operation.

Further, when both the previous state and the rear state are “low” states, if there is no change for a predetermined time or more from this state, the control content determination unit 203B determines the “reproduction stop” operation as the output operation of the device 101. .

Further, for example, when the brightness around the device 101 changes from “medium”, that is, a medium state, to “high”, that is, a bright state, the control content determination unit 203B determines as “contents with high saturation”. Determine the "switch" action.

FIG. 18 is an explanatory diagram of a table 901C for determining control contents using inclination determination by a three-axis sensor in the present embodiment. The table 901C is an example of a predetermined association. The table 901C includes correspondence between information indicating the posture of the device 101 and control related to presentation of content by the device 101 in the posture. In the control related to the presentation of content, the control related to the presentation of content associated with the information indicating the attitude of the device 101 is performed.

The table 901C is equipped with a three-axis sensor as the sensor 201, and defines the operation to be executed by the output unit 202 of the device 101 according to the environment in which the device 101 is installed, more specifically, the tilt state of the device 101 main body. It is an example of a table.

The table 901C manages the installation state of the device 101 main body based on the rising angle θ. When the installation state of the main body of the device 101 is “the rising angle θ from the horizontal plane is 0 ° or more and less than 60 °”, the control content determination unit 203B determines that the device 10 may fall over, based on the determination. In addition, an operation of “notifying the alert using characters, sound or light” and an operation of “continuing reproduction of the audiovisual content” are determined. At this time, the actual installation status log (flat playback) can be recorded as a feedback recording as a playback environment. The creator 20 can use this feedback record arbitrarily. This installation state is also referred to as a flat placement state.

Further, when the installation state of the apparatus 101 main body is “the rising angle θ from the horizontal plane is 60 degrees or more and less than 90 degrees”, the control content determination unit 203B determines that the main body is leaned diagonally according to the determination. Since the distance to the user's 10 ear is different between the upper and lower speakers, the operation of “increasing the volume of the lower speaker than the volume of the upper speaker” is determined so as to achieve a balance for the viewing user 10. This installation state is also referred to as a standing state.

Further, when the installation state of the device 101 main body is “the rising angle θ from the horizontal plane is 90 degrees or more”, the control content determination unit 203B performs an operation of “same volume at the upper speaker and the lower speaker” by the determination. decide. This installation state is also referred to as a wall-mounted state or a roof / ceiling installation state.

FIG. 19 is an explanatory diagram of a table 901D for determining control contents using installation environment determination using upper and lower buttons in the present embodiment. The table 901D is an example of a table that defines the operation to be executed by the output unit 202 of the device 101 according to the installation state of the device 101, that is, the flat placement state, the wall hanging state, the standing state, or the roof ceiling installation state. .

Here, it is assumed that the device 101 has an upper button and a lower button that are button sensors. The upper button corresponds to the sensor 201BH, and the lower button corresponds to the sensor 201BL.

The table 901D manages the installation state of the device 101 main body based on the state of the upper button and the lower button. When the installation state of the device 101 is “ON” for both the upper button and the lower button, that is, when the installation state is a leaning state, the control content determination unit 203B determines that “the lower volume than the volume of the upper speaker” The operation of “increasing the volume of the speaker” is determined. When the main body is slanted and leans against the wall, the distance from the upper speaker 205 to the user's 10 ear is different from the distance from the lower speaker 206 to the user's 10 ear. This is to achieve equilibrium from a viewpoint.

When the installation state of the apparatus 101 main body is “OFF” for both the upper button and the lower button, that is, when the installation state is a wall-hanging state or a flat placement state, the control content determination unit 203B determines “ The operation of “same volume in speaker 205 and lower speaker 206” is determined.

FIG. 20 is an explanatory diagram of a table 1001 which is a content management table in the present embodiment. The table 1001 is an example of a predetermined association. The table 1001 includes the association between the environment information of the device 101 and attribute information indicating the attribute of the content to be presented by the device 101 having the environment information that matches the environment information. The content having the attribute information associated with the acquired environment information may be selected by referring to the association. Further, the predetermined association includes content to be presented by the device 101 and permissible information indicating a permissible presentation mode of the content. In the control related to content presentation, the predetermined association is a permissible presentation mode. The content that can be presented may be selected.

The table 1001 is managed by the content management unit 204, and is a table for managing the content to be output described with reference to FIGS. In addition, some of the contents described in the table 1001 may be held in the contents management DB 208, or some may be distributed from a broadcast station via a network or broadcast radio waves.

In the table 1001, content type, content ID (CID), name, presence / absence of sound source, availability of sound superimposition, BGM (Back Ground Music) meta information, attention rank, and permissible information are associated with each content. Are managed.

Content type is the type or category of the content. The content type corresponds to the output content determined by the control content determination unit 203B in FIG.

The content ID is an identifier that can uniquely identify the content.

Name is the name of the content.

The presence / absence of a sound source is information indicating the presence / absence of a sound source associated with the content.

The sound superimposability is information indicating whether sound can be superimposed on the content.

The BGM meta information is meta information that can be used for searching for a sound source when superimposing a sound as BGM on the content.

The attention rank is information indicating the degree of attention obtained from a rating action such as the degree of favorite of the user 10 who uses the content. The attention rank is basically managed separately from the attention point whether or not each user 10 has watched and watched, but the attention rank may be processed or corrected in conjunction with the attention point.

The permissible information is permissible information related to the output intended by the creator 20 who produced the content, and whether or not editing, adjustment, or processing of the content is permitted when the content is presented. It is the information shown for every. The allowance information is, for example, square cutout availability information that is information indicating whether or not the content can be cut into a square shape, that is, information that indicates whether or not playback is possible in a direction in which the length of the video constituting the content is in the vertical direction. Information regarding whether or not to adjust the shape or orientation of the content, such as vertical installation reproduction availability information. In addition, the permissible information includes information related to suitability for environment setting, that is, environment when content is output.

For example, if the control content determination unit 203B has determined the content playback as the output content in step S602, one of the contents such as C11 in the table 1001 is selected.

Note that the method for selecting content is not particularly limited. For example, if there is a content whose playback time is determined by the content, the one corresponding to the time or content type at that time may be selected.

Further, in the state estimation in step S412, when the detailed state such as the orientation of the user 10, the action or expression of the user 10 is estimated, the control content determination unit 203B matches the state with the BGM meta information. The content may be selected in consideration of the attention rank, environment setting, and the like. For example, as shown in FIG. 14B, when the viewing environment estimation unit 203A estimates that one user 10 and two dogs exist in the space and are facing the front with respect to the device 101 Selects the content C11 in which a keyword such as “slowly” that is expected to be browsed carefully after receiving the gaze of the user 10 is selected from the BGM meta information in the table 1001. Here, for example, the content “paper craft” of C11 is selected.

Also, as shown in FIG. 14A, when there is one user 10 and one dog in the space, a person is looking sideways with respect to the device 101, and viewing is being performed. If the environment estimation unit 203A has estimated, content C13 in which a keyword such as “slow” that matches music that does not interfere with reading is set in the meta information is selected. In this case, for example, the content “Craftsman's watch” of C13 is selected. Note that the content selection criteria are not limited to those described above, the content name may be selected by keyword search, and meta information directly indicating the content of the video content is added to the management table for management. Also good.

In the above description, the example in which the control content determination unit 203B selects the content in association with the BGM meta information associated with each content according to the presence or absence of the user 10 or the state has been described. . For example, a category of content to be output in association with a state may be set in advance in accordance with the behavior that the user 10, the service manager, or the creator 20 likes or assumes. According to this, for example, when a certain user 10 is facing the front as shown in FIG. 14B, the user 10 can play the music that the user 10 likes and the video can be superimposed using the music as a sound source. Content can also be set as a content candidate. Here, the contents of C11, C13, and C15 are set as content candidates because the superimposition of sound is possible or essential. Thus, the audiovisual content suitable for the user's 10 hobbies or preferences can be output in the state or timing set by the user 10.

When the user 10 exists, the control content determination unit 203B of the device 101 does not select the content, but displays and recommends a plurality of content candidates according to the state from the plurality of contents, and prompts the user 10 You may make it select.

In the present embodiment, since it is assumed that all contents are held in the device 101, basically, all contents are managed in the content management DB 208. Therefore, the control content determination unit 203B acquires content from the content management DB 208 in step S603.

However, as described above, each content may be a content held in the content management DB 208 or may be a content distributed from a broadcast station via a network or broadcast radio waves. The table 1001 also manages the source of each content. For example, since C11 to C52 have actual data of continuous video content of at least several seconds or more, if there is data managed in the content management DB 208, the actual data is held elsewhere, There may be a pattern in which only access path information to the held area is managed. When the content is not managed by the content management DB 208, the control content determination unit 203B controls the device 101 to acquire the content via the network. Also, the content to be played back may be a still image such as a photograph, and the control content determination unit 203B can manage still images such as a photograph in the content DB 204B of the device 101, and controls acquisition and playback as video content. be able to.

The photograph may be a photograph taken by the user 10, a photograph registered by the service manager or the creator 20, and may include everything recorded by the user 10 in the device 101. When “Photo” is selected, a plurality of photos can be controlled to be sequentially played back in a slide show format, and a playlist including the order or playback time intended by the creator 20 is finely managed and controlled. You can also Further, the slide show playback speed or playback method may be changed according to the state of the user 10. Further, it may be controlled to select and reproduce a plurality of photos to be reproduced according to the state of the user 10.

In the example of the table 1001, the table is an example of managing the output content by a combination of video content and audio content, but the management method is not limited to this. For example, audio-only content may be included, and video content may necessarily include acoustic data.

In addition, although the attention rank with respect to each audiovisual content is represented by a numerical value normalized with 1 being the maximum here, the management method is not limited to this, and integer management may be performed using cumulative addition points. Moreover, it is good also as the management method which attached importance to the relationship of a link similarly to the searchability weighting by the mutual link of a web site by linking contents with a deep relationship of a reproduction order mutually.

Note that, as “allowable information” is managed in the table 1001, not only the playback output setting of each content device but also the presence of the user 10 in front of the device 101 or the attribute or behavior of the user 10 The output method in the output unit 202 may be adjusted or changed.

In addition, although the description has been given taking the square cut out as an example of the permissible information, for example, the 16: 9 video content is not limited to whether or not it is allowed to cut out into a square, and there is a limit within a range that does not contradict the intention of the creator 20. As an expression method, for example, an editable area is set in time series in the screen area of the video content, and edit control is performed to allow the video to be cut out while retaining the editable area. You may do it.

In addition, the environment setting which the creator 20 intends among permissible information is demonstrated as an example. C11 is set to “darken the surroundings” and can be controlled in the viewing space of the user 10 such as the lighting function provided on the outer frame of the main body of the device 101 and the lighting that can be linked around the device 101 via the network System control that darkens the surroundings in conjunction with the system. Also, C12 is set to “size M or more”, and if the size of the playback screen of the device 101 is not M or more, the audiovisual content C12 cannot be played back. Also, C13 is set to “bright illuminance”, and when the backlight of the output unit 202 of the device 101 is set to be brighter than the normal setting and played back, the original video that has been a drawback of the video content is displayed. Setting that compensates for the dark problem as the intention of the creator 20 becomes possible. Also, C14 is set to “frame color: red”. For example, the frame of the outer frame of the main body of the device 101 can be played back only with a physically red frame, or the exterior frame P101 is an LED. Alternatively, in the case of a material that can express a color or a pattern by light emission such as OLED, the frame color is changed to “red” independently of the output unit 202 in conjunction with the content, and the content is reproduced by the output unit 202. It can also be controlled. Further, C15 is set as “unsuitable for accommodation facility”, and if the place where the device 101 is installed is an accommodation facility, the audiovisual content can be controlled to be unsuitable and excluded from reproduction candidates.

FIG. 21 is an explanatory diagram of a table 1101 that is a voice management table in the present embodiment. A table 1101 shown in FIG. 21 shows a table in which output methods corresponding to audio categories are linked. The table 1101 is also managed by the content management unit 204.

In the table 1101, music ID (Music ID), text and keywords indicating the outline of the sound source, and path information to the corresponding file are managed according to the category corresponding to each sound source type. Categories include, for example, dignity, tradition, or curiosity.

For example, when a content in which “quality” is set in the table 1001 is selected, the control content determination unit 203B refers to the table 1101 and selects and outputs a sound source that can be matched with a keyword from sound source candidates that are optimal for the quality. .

With this configuration, the device 101 can automatically control the content to be presented based on the current state of the user 10. For example, the device 101 can optimize the viewing environment of the user 10 by detecting the state where the device 101 is leaned against a wall and automatically adjusting the volume of the lower speaker 206 to be larger than that of the upper speaker 205. . Further, the device 101 can analyze the user's attention and select and switch the audiovisual content according to the degree of attention of the user 10. Also, the device 101 manages the viewing state of the user 10 as a log, and can feed back to the creator 20 to improve the video / audio content or produce a new video / audio content. It is possible to provide a comfortable audiovisual content reproduction environment and a continuous audiovisual content production environment by the creator 20 without any operation and trouble.

It should be noted that various indicators can be used to determine the combination of video content and audio content to be played simultaneously. As this index, for example, an index associated with content and an index expressed by a continuous value can be adopted. More specifically, an estimation index indicating the degree of attention that the user 10 is directed to each of the video content and the audio content may be employed. In this case, the user 10 can specify the index range, and both the video content and the audio content associated with the index included in the range specified by the user 10 can be reproduced. By doing in this way, it is possible to reproduce both the video content and the audio content that are estimated that the user 10 pays the same attention.

In the example of the present embodiment, the method of estimating the state or action of the user 10 and controlling the device in accordance with the state or action has been described. However, as shown in FIG. It may be set and prompted to change the action of the user 10. FIG. 22A shows a table 1201 created by setting in advance a target action for the user 10 to promote an action every 15 minutes in a state where the user 10 is known to exist. As shown in the table 1201, when the user 10 sets the target action in advance, the control content determination unit 203B determines the content, completes the table 1201, and sets the output unit 202 to reproduce the content according to the table 1201. Control. Thereby, the user 10 can be guided by the reproduction of the content so that the user 10 performs the target action. For example, if the table 1201 is set, if it is obvious that the user 10 will re-enter the room at 12:00, the content of C15 will be used as video and music that would make something up-tempo at the time of re-entry. Is played. Thereby, for example, a child's action can be induced and controlled by audio-visual content that is automatically reproduced while the parent is out.

In addition, although the example which set the target action for every 15 minutes was demonstrated, it is not restricted to this, The user 10 should just set for every arbitrary predetermined time. Further, the user 10 may directly set the playback content every predetermined time in advance. Moreover, the recommended pattern regarding these target actions may be held in the content management unit 204 of the plurality of devices 101. Here, the recommended pattern includes an education priority pattern, an exercise priority pattern, or a sleep priority pattern. Thereby, the user 10 can promote or control the action of the user 10 by the audiovisual content only by selecting one from a plurality of recommended patterns recommended by the device 101 when going out.

That is, the control content determination unit 203B of the device 101 has a schedule function for managing the order of the control content in time series, and the schedule defines the output according to the target action that the user 10 wants to instruct, A predetermined video or audio along the target action is output at a specified time described in time series. According to such a configuration, predetermined video or audio can be output in the order stipulated in the schedule. For example, even when a parent is absent, audiovisual content that encourages study, exercise, or sleep can be played, and a life rhythm is formed. By doing so, it is possible to guide the user 10 to an assumed life pattern.

In the present embodiment, control is performed such that the control content determination unit 203B determines the control content corresponding to the current state estimated by the viewing environment estimation unit 203A, but is not limited thereto. For example, the control content determination unit 203B may control the output unit 202 by determining optimal control content according to the timing at which the state estimated by the viewing environment estimation unit 203A changes.

Note that the environment information may include juxtaposition information that is information regarding whether or not the device 101 is installed side by side with other devices 101 as information regarding the installation mode of the device 101. The predetermined association may include association between juxtaposition information of the device 101 and control related to presentation of content by the device 101 installed in the installation mode indicated by the juxtaposition information. And in control regarding presentation of content, you may perform control regarding presentation of content matched with the juxtaposition information of the apparatus 101. FIG.

In addition, when the device 101 specifies in which direction the user 10 is seen from the device 101, the output volume of the upper speaker 205 and the lower speaker 206 may be controlled. This will be described below.

FIG. 22B is an explanatory diagram of a state in which the user 10 exists in front of the device 101 in the present embodiment. FIG. 22C is an explanatory diagram showing the relationship between the direction in which the user 10 is seen from the device 101 and the output volume of the speaker according to the present embodiment.

FIG. 22B shows a case where one user 10 is in front of the device 101 as viewed from above the device 101. Here, the angle between the normal line of the output unit 202 of the device 101, that is, the display panel P122, and the line from the center of the output unit 202 toward the user 10 is defined as θ. Also, θ is positive when a person is on the right side of the normal line as viewed from the device 101, and θ is negative when the person is on the left side. Also, the output volume of the left upper speaker 205 is H1, the output volume of the right upper speaker 205 is H2, the output volume of the left lower speaker 206 is L1, and the output volume of the right lower speaker 206 is L2. .

At this time, the device 101 controls the output volume of each speaker as shown in FIG. 22C according to the direction in which the user 10 is seen from the device 101. A numerical value such as “5” shown in FIG. 22C is a numerical value indicating the relative relationship of the output volume.

For example, when the user 10 is present in front of the device 101 (−30 degrees <θ <30 degrees), the output volume of all speakers is set to 5 and the same. When the user 10 is present on the right side of the device 101, the output volume of the right upper speaker 205 is increased to 7, and the output volume of the lower speaker 206 on the right is increased to 6. Further, the output volume of the left upper speaker 205 is maintained at 5, and the output volume of the left lower speaker 206 is lowered to 4. As described above, the device 101 increases the output volume of the speaker close to the user 10 according to the direction of the user 10 viewed from the device 101, and decreases or maintains the output volume of the speaker far from the user. For example, in the case of content that encloses a space depending on the type of content, the output volume of the upper speaker 205 may be increased. In addition, in the case of content that sounds a low tone, the output volume of the lower speaker 206 may be increased. As described above, the device 101 controls the presentation of content according to the surrounding environment.

If there is one or more users 10 in front of the device 101, the output volume of the speaker can be controlled as follows.

FIG. 22D is an explanatory diagram showing a state in which a plurality of users 10 exist in front of the device 101 in the present embodiment. FIG. 22D illustrates a case where a plurality of users 10 are present in front of the device 101 as viewed from above the device 101.

FIG. 22E is an explanatory diagram showing the relationship between the number of users 10 existing in front of the device 101 and the output volume of the speaker in the present embodiment. Here, the operation mode of the device 101 when there is only one user 10 existing in front of the device 101 is also referred to as a simple mode, and the operation mode of the device 101 when there are a plurality of users 10 is also referred to as a party mode.

When only one user 10 exists in front of the device 101 and the line of sight of the one user 10 is directed toward the device 101, the device 101 is shown in FIG. 22C. The output volume of each speaker shown is controlled.

When only one user 10 exists in front of the device 101 and the line of sight of the one user 10 is not directed toward the device 101, the device 101 Control is performed to lower the value at regular intervals.

In addition, when there are a plurality of users 10 in front of the device 101 and the line of sight of the plurality of users 10 is directed to the device 101, the device 101 outputs each speaker. Control the volume. For example, the volume of the upper speaker 205 is 6 and the volume of the lower speaker 206 is 7. By doing in this way, even when there are a plurality of users 10 in front of the device 101, it is possible to easily reach the volume of each person.

Further, when there are a plurality of users 10 in front of the device 101 and the line of sight of the plurality of users 10 is not directed to the device 101, the device 101 sets the volume to a certain time. Control to lower each time.

In this way, the device 101 controls the presentation of content according to the surrounding environment.

Further, when the device 101 collects sound around the device 101 by the sensor 201F, which is a microphone, the output volume of the speaker may be controlled according to the collected sound volume.

FIG. 22F is an explanatory diagram of sound collection volume in the present embodiment. FIG. 22F illustrates a situation in which sound around the device 101 is collected by the sensor 201 </ b> F that is a microphone when a plurality of users 10 exist in front of the device 101.

FIG. 22G is an explanatory diagram showing the relationship between the sound collection volume and the speaker output volume in the present embodiment.

When the sound collection volume is relatively high or medium, the output volume of each speaker is set to 5, and the luminance of the display panel P122 as the output unit 202 is gradually changed to medium.

When the sound collection volume is medium or relatively low, the output volume of each speaker is gradually changed to 2, and the luminance of the display panel P122 as the output unit 202 is gradually changed to a low state.

Also, when the sound collection volume is low, the output volume of each speaker is gradually changed to 0 after a predetermined time has elapsed, and the power of the display panel P122 which is the output unit 202 is turned off.

In this way, the device 101 controls the presentation of content in a complex manner according to the number of people around and the sound collection volume of the microphone. In this way, the device 101 can more appropriately control the presentation of content according to the surrounding environment.

Note that, as described above, the device 101 selects content and presents the selected content. Here, the content presentation is controlled in accordance with the environment around the electronic device by selecting the content without modifying the content itself. Therefore, music data different from that selected in the past may be presented in combination with video data that has been selected in the past.

(Modification of Embodiment 1)
FIG. 23 is a block diagram illustrating configurations of the device 101 and the cloud server 111 according to the present modification. FIG. 23 is a configuration diagram of each device in a configuration in which a part of the functions of the control processing unit 203 according to the first embodiment is processed by an external device such as the cloud server 111, the server 121, or the home gateway 102. . In this modification, the device 101 performs processing while exchanging information with the cloud server 111. It should be noted that the same description can be established by using the server 121 or the home gateway 102 instead of the cloud server 111.

Hereinafter, the description will focus on the points different from the configuration diagram shown in the first embodiment. The same components as those described in Embodiment 1 are denoted by the same reference numerals and description thereof is omitted.

23 differs from the device 101 in FIG. 3 in that the control processing unit 1301 does not include the viewing environment estimation unit 203A and includes the information transmission / reception unit 1302.

The information transmission / reception unit 1302 is a configuration for transmitting / receiving information to / from the cloud server 111 via a network, and is an essential configuration in the present modification. The information transmission / reception unit 1302 may adopt a general communication module or the like, and its specification is not particularly limited. Further, there is no particular limitation on a communication method for transmitting / receiving information.

The cloud server 111 includes a control processing unit 1303, an information transmission / reception unit 1304, and a content management unit 1305.

The control processing unit 1303 includes a viewing environment estimation unit 1303A. The operation of the viewing environment estimation unit 1303A is the same as the operation described in Embodiment 1.

The information transmission / reception unit 1304 is configured to transmit / receive information to / from the device 101 via a network.

The content management unit 1305 includes a viewing environment accumulation DB 204A and a content DB 204B. The basic functions of the viewing environment accumulation DB 204A and the content DB 204B, the data to be recorded and managed, and the like are as described in the first embodiment.

The operation or process flow of each device and each component in the present modification is the same as the operation or process flow described with reference to FIGS. 4 to 12C in the first embodiment. That is, in the present modification, information is appropriately transmitted and received by the information transmission / reception unit 1302 of the device 101 and the information transmission / reception unit 1304 of the cloud server 111, and processing is basically performed in the order shown in FIG. Therefore, the same function and effect as in the first embodiment are also exhibited in this modification.

The main difference between the first embodiment and the modification is that the cloud server 111 includes the viewing environment estimation unit 203A in the first embodiment as the viewing environment estimation unit 1303A, and basic information storage is performed in the cloud. This is a point performed by the content management unit 1305 of the server 111. As a result, processing performed on the device 101 side can be reduced, so that it is not necessary to mount a high-performance processor on the device 101 side. Further, since it is not necessary to record and manage each table or content on the device 101 side, the recording capacity on the device 101 side can be reduced. That is, even when the processing capability of the device 101 is low, optimal device control that estimates the state of the user 10 and the like can be performed.

Furthermore, since the cloud server 111 can acquire log information from the device 101, further information analysis, learning using a large amount of information, and utilization of information can be expected. Furthermore, it is possible to provide a new valuable service by analyzing together the information acquired from devices held by other users 10 in the cloud server 111.

Also in this modification, the device 101 and the cloud server 111 may have a configuration other than the above-described configurations.

(Embodiment 2)
[Configuration of each device]
FIG. 24 shows configurations of the device 101c, the device 101d, the device 101e, and the cloud server 111 according to the second embodiment. The configuration of each device will be described below with the same reference numerals for the same configurations as those of the first embodiment and the modification of the first embodiment. A part of the description of the configuration similar to that of the first embodiment and the modification of the first embodiment is omitted.

It should be noted that, similarly to the modification of the first embodiment, the same explanation can be realized by using the server 121 or the home gateway 102 instead of the cloud server 111.

The device 101c includes a sensor 201c and an information transmission / reception unit 1302c. The basic functions of these configurations are the same as those of the sensor 201 and the information transmitting / receiving unit 1302 described with reference to FIG. Here, any device may be used as the device 101c. Similar to the first embodiment, it may be an electronic device capable of reproducing audiovisual content, or a dedicated terminal having only a sensor and an information transmission / reception unit, or a terminal worn by the user 10, a so-called wearable terminal. When a wearable terminal is used as the device 101c, the sensor 201c may be a sensor that acquires biological information such as an activity meter, a pedometer, a sphygmomanometer, a heart rate monitor, a sweat measurement, or an electroencephalogram measurement. Here, the biological information includes vital information, behavior information, stress information, and the like.

Also, a sound collecting microphone that directly or indirectly acquires sound such as roaring or crying may be used. In addition, the device 101c may have other configurations not shown.

The device 101d includes an output unit 202d and an information transmission / reception unit 1302d. In the present embodiment, for example, the device 101d is assumed to be a display that does not include a sensor, a device that only performs LED (Light Emitting Diode) notification, or a speaker that outputs only sound. Here, the display, LED, speaker, or the like corresponds to the output unit 202d. The information transmitting / receiving unit 1302d is the same as the information transmitting / receiving unit 1302 described in FIG. Note that the device 101d may have other configurations not shown.

The device 101e includes an input / output unit 202e and an information transmission / reception unit 1302e. In the present embodiment, for example, an electronic device including an operation screen (GUI: Graphic User Interface) that allows an operation input is assumed as the device 101e. The information transmission / reception unit 1302e is the same as the information transmission / reception unit 1302 described in FIG. Note that the device 101e may have other configurations not shown.

The cloud server 111 includes a control processing unit 1401, an information transmission / reception unit 1304, and a content management unit 1404.

The control processing unit 1401 is basically the same as the control processing unit 203 provided in the device 101a described in the first embodiment, and includes a viewing environment estimation unit 1402 and a control content determination unit 1403. A specific user state estimation method and control content determination method of the viewing environment estimation unit 1402 and the control content determination unit 1403 will be described later.

The information transmitting / receiving unit 1304 has the same configuration as that described in FIG.

The content management unit 1404 includes a viewing environment accumulation DB 204A, a content management DB 208, and a preference tendency DB 1405. Since the viewing environment storage DB 204A and the content management DB 208 are the same as those described in the first embodiment, description thereof will be omitted. Information managed by the preference tendency DB 1405 will be described later.

[Operation and processing flow of each device]
FIG. 25 is a sequence diagram showing the operation of each device and the flow of processing in the present embodiment. The following description will focus on differences from the flowchart of the first embodiment shown in FIG.

In step S1501, the sensor 201c of the device 101c detects a state in the space. Step S1501 is basically the same as step S401 shown in FIG. However, when the sensor 201c is a biosensor of a terminal worn by the user 10 as described above, the information to be acquired is not a two-dimensional image in space but biometric information detected by the biosensor.

In step S1502, the information transmitting / receiving unit 1302c of the device 101c transmits the information acquired in step S1501 to the cloud server 111. The information transmission / reception unit 1304 of the cloud server 111 receives information from the device 101c.

Here, the frequency at which the sensor 201 detects in step S1501 or the timing at which the information transmitting / receiving unit 1302c transmits information in step S1502 is not particularly limited, and information may be detected and transmitted at predetermined time intervals. .

In step S1503, the viewing environment estimation unit 1402 of the control processing unit 1401 of the cloud server 111 estimates the state in the space and the state of the user 10 based on the received information. Here, in the case where the sensor 201 detects a two-dimensional image by the imaging device, the presence or absence of the user 10 in the space, the number of users 10, Alternatively, the state, orientation, action, or the like of the user 10 is estimated.

Further, when the sensor 201 is a biometric sensor or the like of a terminal worn by the user 10, the state or emotion of the user 10 is estimated from the acquired biometric information. In this case, if it is managed which user 10 is wearing the device 101c in the cloud server 111, it can be grasped in advance. In this case, as described in the first embodiment, it is not necessary to estimate whether the user 10 exists. As the state or emotion estimation method, a known method may be used and detailed description thereof is omitted. Further, the state estimated at this time may be recorded or stored in the viewing environment storage DB 204A of the content management unit 1404. Further, when the sensor 201 is a sound collecting microphone, the degree of stress or tension may be estimated from a sound such as a beat or cry. Also, the audiovisual content to be played back is switched by detecting the frequency of the conversation, or the excitement or demotion of the conversation, grasping the characteristics of the musical instrument or voice, the progress of the music, the actions of the user 10 such as applause or ringing fingers. It may be.

In step S1505, the information transmitting / receiving unit 1304 of the cloud server 111 transmits the control content determined in step S1504, that is, the control command to each device. Here, the timing at which the information transmission / reception unit 1304 transmits the control content in step S1505 is desirably transmitted without delay every time the control content is determined in step S1504, but is not limited thereto. For example, the control content may be periodically transmitted to each device at predetermined time intervals.

In step S1506, the devices 101b, 101c, and 101d perform an output operation according to the control content transmitted from the cloud server 111, that is, the control command.

As described above, in the present embodiment, the cloud server 111 estimates the state of the user 10 from the spatial information detected by the device 101c or the biological information of the user 10, and the cloud server 111 uses a device other than the device 101c. That is, the contents for controlling the devices 101d and 101e are determined, and each device is controlled. According to such a configuration, as in the first embodiment, it is possible to provide an optimal audiovisual content reproduction environment for the user 10 without requiring a cumbersome operation for the user 10. In the present embodiment, the cloud server 111 performs processing related to state estimation or control content determination, information accumulation, and table management. Therefore, it is not necessary to mount a processor or memory with high processing capability on the device side. In the present embodiment, control contents are determined in cooperation with a plurality of devices and cloud servers. Therefore, it becomes possible to optimally control a plurality of devices only from information on the sensors mounted on one device, and each device does not need to be mounted with a sensor.

It should be noted that a preference tendency DB 1405 may be constructed in the content management unit 1404 to learn the preference regarding the control content according to the state of the user 10. That is, by analyzing the log information related to the control command to the device of the user 10, the table for determining the control content is updated to an optimal one as appropriate. In this case, a sensor that senses the usage status of the device may be used as the sensor 201c. For example, the usage status of each device in FIG. 24 is sensed and accumulated in the preference tendency DB 1405.

The cloud server 111 can manage and analyze a huge amount of information, that is, big data. Therefore, since it is possible to accumulate device log information or sensing information in the preference tendency DB 1405 and learn the preference of the user 10 or the user 10, the service is more comfortable than that of the first embodiment. Can provide. Further, by analyzing together information acquired from devices held by other users 10 in the cloud server 111, (a) a playback order or timing using a tendency of audiovisual content preferred by similar users 10 (B) Form a compatibility or popularity ranking between video and audio and use it for the next playback, (c) Feed back to the screen or community of the creator 20 that produces the audiovisual content, It is possible to provide a new and valuable service such as being useful for video or sound in consideration of the environment where the device is placed or the reproduction condition.

In addition, although the server which controls an electronic device was demonstrated here as the cloud server 111 connected via a network, the structure of the device control server which controls an electronic device is not restricted to this, The same network as the electronic device in a house A local server in a local environment connected to the network may serve as a gateway, and a device control server function for controlling the electronic device may be implemented in the electronic device itself.

Note that, here, the electronic device to be controlled has been described by taking a display equipped with a speaker as an example, but the electronic device to be controlled, its determination criteria, and control contents are not limited thereto. Although it is different from the described shape, it controls projectors that project images on the wall, terminals with screens dedicated to user 10, display signage embedded on the floor, mobile displays that perform automatic driving or automatic flight, etc. It is good. Further, a humidifier or a dehumidifier capable of controlling the scent of air or aroma, or a curtain or blind electronic control device capable of controlling the lighting of a window or an entrance may be set as a control target. In addition, the determination criterion has been described in terms of whether or not an image is output to a sleep state or a behavior state for the sake of simplification. However, the control criterion is not limited to this, and the behavior history of the user 10 is not limited to this. The magnitude of the state, the desire estimated from the cry, the stress response estimated from the vibration state of the user 10 may be used as the determination criterion. The action history state of the user 10 includes, for example, information indicating that the user 10 is not moving much today compared to usual or that the movement is intense today compared to usual. In addition, the control content may be connected to stress relief or divergence in the user 10 by controlling the sound output of video or scent on the screen of one electronic device, and lighting dimming on / off.

Here, an example has been described in which current data obtained from a sensor, specifically a camera, a luminometer, a three-axis sensor, or a pressure-sensitive button, is analyzed to estimate the current state of the user 10. However, the input data used for the criterion for determining the control content is not limited to the current state estimation. For example, the input data has a track record of comparing the state change from the past state to the current state with the past data, and transitioning the state from the defective state to the good state in the past in a situation similar to the past data. The content based on the user type or the user-specific preference may be selected and adopted according to the current state. In addition, the selection may be controlled according to the time zone, weather or season, the number of users 10, and the like. For example, when a sudden evening or lightning strike is predicted, a change in the outside appearance is adopted as a video, or a video that changes from a small sound to a loud sound in advance in preparation for a sudden sound such as thunder. You may do it. Moreover, it is good also as what gives a change to the image | video to employ | adopt, such as employ | adopting the image | video which encourages relief if the said user 10 is a child, and the image which makes a hunting go bravely if it is an adult. The past data may be actual data of growth records of the user 10's own family or child, or the actual value of the same generation or an average value thereof may be used. In addition, when the transition to the assumed state is not made within a certain period of time or when the transition to the defective state occurs, the output contents are prioritized according to the past data, and are controlled sequentially from the one with the highest priority. May be executed.

In addition, although the specific example centered on the input from the sensor mounted on the electronic device or the external dedicated terminal to be linked is described here, the input data is not limited to the sensor, for example, the electronic device or the linkage If an external dedicated terminal is placed in a restaurant private room, hotel room, condominium party room, wedding reception hall, or a delivery-type catering destination ordered by an individual or corporation, a restaurant is reserved User 10 attribute information, past reservation history or order history, profile of invited user 10, party such as celebration or anniversary, or purpose of stay, contents of course meal on that day, reserved private room or hotel room Static or dynamic information about the user 10 or space, such as a spatial theme or feature It may also include a. Here, the attribute information of the user 10 includes the age, sex, family structure, presence / absence of child, occupation, and the like of the user 10. The order history includes food or liquor preferences. In the control related to the content, for example, when the child is present, the pop audiovisual content may be preferentially reproduced, and the reproduction content of the audiovisual content may be changed in accordance with the progress of the course meal provision. More specifically, the Vienna court preferentially selects the music or artwork of an artist belonging to the region associated with the selected dessert or tea, switching to a video related to the origin of the ingredients, while playing the sound seamlessly. In the case of cooking, priority may be given to images or music related to Austria.

The dedicated terminal is a cooking device such as a roasting machine, a coffee maker, an electromagnetic cooking machine, or an oven, for example, determining the place of origin of the roasted green beans, the sound during roasting, the sound during milling, You may make it produce | generate the audiovisual content at that time using the sound at the time of drip on the spot. In addition, by incorporating a process of boiling with an electromagnetic cooking machine or a process of slowly baking in an oven as an audiovisual content, it is possible to provide an effect of enjoying the background of the provided food or beverage with the five senses. In addition, if there is a wine bottle cooler that can specify the brand of wine or sake as an external dedicated terminal, the audio / video content related to the place of production or raw material, and the particular way of drinking can be obtained from the brand information of the drink. It may be controlled to play back or play audiovisual content explaining how to make sake in conjunction with the insertion of a bottle. In addition, if the sensor of the external dedicated terminal is a sensor that can recognize a wide range of space conditions, such as a ceiling camera, the chef at the time when the course dish is served on the table or until it is served. By automatically inserting and playing back stories of dishes on the table in a timely manner, such as the message video from or the origin or breeding environment of the ingredients used for cooking, the 3-day preparation process, etc. By avoiding this, the deliciousness of cooking can be produced at the climax by the information of audiovisual content.

In addition, when it is determined that the conversation frequency is high and the conversation frequency is high and the laughter or the tone of the voice is high as an index by detecting the conversation frequency or the conversation excitement or decline, You may make it preferentially select the image | video with little flickering or switching of a screen, or an acoustic music piece with little intensity of sound, in which sound does not insist too much. In addition, when the frequency of conversation is low, laughter is low, and tension or rise is judged from the tone of the voice, the video or audio content is devised, pop and humorous video, You may make it preferentially select a certain video, a video that can provide a topic of change or unexpectedness by carefully watching, or an acoustic music that lightens the atmosphere of the place.

In addition, when electronic devices or linked external dedicated terminals are placed in the shared space of a building common area, conference room, music studio, or share house, install multiple electronic devices and external dedicated terminals. Also good. For example, if the degree of congestion in a building common area is obtained from a sensor mounted on an electronic device or an external dedicated terminal or a visitor number recording system, and the space is quiet, the arrangement relationship of multiple electronic devices can be grasped and The distribution control may be performed by cutting out the video in consideration of the arrangement relationship of each electronic device so that one large video is formed and reproduced by the device. Further, if the space is congested, control may be performed so that the audiovisual content is optimized and optimized for each electronic device so that each visitor can visually recognize the audiovisual content. In addition, in order to welcome important customers with hospitality at the building common use department, you may preferentially play audiovisual content related to your business. In addition, by registering the facial features of the customer in advance and detecting the customer's visitor, an effect of changing the audiovisual content dramatically or naturally may be provided.

Also, for example, in the case of a conference room, the playback order of audiovisual content is designed according to the set time of the conference, and the phase of deriving discussion, sharing phase, wrap-up and approval, and action item confirmation from the initial objective sharing The audiovisual content may be reproduced according to the above. In addition, when the frequency of conversation is high and active discussions can be performed, the main body of the electronic device reproduces audiovisual content indicating an active meeting, and further, as an external output device, lighting Control may be performed to increase the spatial value by changing the hue or outputting a scent that promotes a relaxation or cool-down effect. In addition, when the conversation frequency is low and active discussions are not being developed, audiovisual content that can be used as a trigger for discussions, or that has a refreshing effect that breaks down the thoughts of thought, can be refreshed. Or may be controlled to output a scent that has an effect of stimulating thought. Further, the playback method may be changed and controlled so as to indicate that the conference room use end time is close from 15 minutes before the conference room use end time. For example, an alarm sound is inserted step by step from 15 minutes ago, the screen is gradually reduced, the sound volume is increased, or the screen is stained red. Can be devised so that changes in

Further, for example, in an acoustic room such as a music studio or karaoke, a player who plays music or a performer who utters voice as vocal music emits music, so that only video content may be played back. To create an environment in which a player or performer can relax and practice, the player or performer recognizes the player or plays a pre-registered player or performer's favorite video content, or the voice sound feature of the instrument played or the vocal sound The feature may be grasped and video content corresponding to the feature may be reproduced. In particular, a player or performer who is practicing plays a video content indicating that the fitness level is high if the fitness level is high with respect to the scale that should be played, or if the fitness level is low, For example, it may be possible to play back video content including content to be dealt with at a low level. This video content includes, for example, a video in which an object moves upward as it rises to guide the pitch in the direction of increasing the pitch. In addition to music studios, when multiple electronic devices are installed in concert halls with performers or performers, high-end restaurants, hotel lobbies, shared buildings, or commercial complex facilities, each operates individually. Rather than electronic devices connected via a network via a communication unit, they are designed and implemented so that they move in a unified and synchronized manner. It is also possible to produce a video content that is synchronized with a plurality of electronic devices according to the voiceprint of the performer. For example, music such as classical music may be specified, and video content designed for starting and transferring music may be controlled to be played on at least one electronic device according to the progress.

In addition, not only the characteristics of the sound produced by the performer or performer, but the non-occurrence caused by the action of the user 10 such as applause or finger whistle of the audience who listens to the performance or vocal music, single sound produced by the operation of the facilitator who manages the place ringing the finger It is also possible to determine the characteristics of language audio or environmental sound and change the playback content or playback order of the audiovisual content.

Also, for example, in the case of a shared space in a share house, the number of tenants living together in the shared space with the time series needs to increase or decrease, and the creation of the atmosphere of the place needs to be changed. Therefore, for example, the number of people in the shared space is grasped with sensors of electronic devices or external dedicated terminals, and when there are few people, switch to calm audiovisual content, or the number of people increases and conversations between people increase, If the atmosphere is close, it may be controlled to switch to bright and pop audiovisual content. Furthermore, when a certain user 10 takes out an analog record and starts playback on the record player, it is determined that there is external music, and the audio content output from the electronic device or the external dedicated terminal is stopped, and only the video content is played back. You may control to change. Here, the number of persons can be grasped by grasping the number of warm objects using, for example, a two-dimensional temperature sensor. In addition, the external music feature may be grasped by collating with a database on the cloud, and the video content may be switched to a category that matches the external music feature. Here, the audiovisual content may be preset in an electronic device or an external dedicated terminal, or may be downloaded and added via a network via a communication unit.

(Embodiment 3)
In this embodiment, a structure of an electronic device and a method for controlling the electronic device described in each of the above embodiments or modifications will be described.

FIG. 26 is a block diagram showing the device 101 according to the present embodiment.

As shown in FIG. 26, the device 101 includes an acquisition unit 301 and a control processing unit 302.

The acquisition unit 301 acquires environment information of the device 101 including information related to the environment around the device 101 or information related to the installation mode of the device 101.

The control processing unit 302 refers to a predetermined association between the environment information of the device 101 and the details of the control related to the presentation of the content by the device 101, so that the content of the content by the device 101 associated with the acquired environment information is displayed. Control the presentation.

FIG. 27 is a flowchart showing a method for controlling the device 101 according to the present embodiment.

As shown in FIG. 27, the control method of the device 101 includes steps S1 and S2.

In step S <b> 1, the acquisition unit 301 acquires the environment information of the device 101 including information about the environment around the device 101 or information about the installation mode of the device 101.

In step S <b> 2, the control processing unit 302 refers to a predetermined association between the environment information of the device 101 and the content of the control related to the presentation of content by the device 101, so that the device is associated with the acquired environment information. 101 performs control related to the presentation of content.

Thus, the electronic device can control the presentation of content according to the environment around the electronic device, which is different from hardware or software factors.

The technology described in the above aspect can be realized, for example, in the following types of cloud services. However, the type in which the technique described in the above embodiment is realized is not limited to this.

(Service type 1: In-house data center type)
FIG. 28 shows service type 1 (in-house data center type). This type is a type in which the service provider 120 acquires information from the group 100 and provides a service to the user 10. In this type, the service provider 120 has a function of a data center operating company. That is, the service provider 120 has a cloud server 111 that manages big data. Therefore, there is no data center operating company.

In this type, the service provider 120 operates and manages the data center 2203 (cloud server 111). The service provider 120 manages the OS 2202 and the application 2201. The service provider 120 performs service provision 2204 using the OS 2202 and the application 2201 managed by the service provider 120.

(Service type 2: IaaS usage type)
FIG. 29 shows service type 2 (IaaS usage type). Here, IaaS is an abbreviation for infrastructure as a service, and is a cloud service provision model that provides a base for constructing and operating a computer system as a service via the Internet.

In this type, the data center operating company operates and manages the data center 2203 (cloud server 111). The service provider 120 manages the OS 2202 and the application 2201. The service provider 120 performs service provision 2204 using the OS 2202 and the application 2201 managed by the service provider 120.

(Service type 3: PaaS usage type)
FIG. 30 shows service type 3 (PaaS usage type). Here, PaaS is an abbreviation for Platform as a Service, and is a cloud service provision model that provides a platform serving as a foundation for constructing and operating software as a service via the Internet.

In this type, the data center operating company 110 manages the OS 2202 and operates and manages the data center 2203 (cloud server 111). Further, the service provider 120 manages the application 2201. The service provider 120 provides the service 2204 using the OS 2202 managed by the data center operating company and the application 2201 managed by the service provider 120.

(Service type 4: SaaS usage type)
FIG. 31 shows service type 4 (SaaS usage type). Here, SaaS is an abbreviation for software as a service. For example, a function that allows applications provided by a platform provider who owns a data center (cloud server) to be used via a network such as the Internet by a company / individual (user) who does not have a data center (cloud server). This is a cloud service provision model.

In this type, the data center operating company 110 manages the application 2201, manages the OS 2202, and operates and manages the data center 2203 (cloud server 111). Further, the service provider 120 provides the service 2204 using the OS 2202 and the application 2201 managed by the data center operating company 110.

In any of the above types, it is assumed that the service provider 120 performs a service providing act. Further, for example, the service provider or the data center operating company may develop an OS, an application, a big data database, or the like, or may be outsourced to a third party.

As described above, according to the electronic device control method of each embodiment, the electronic device performs control related to presentation of content using the acquired environment information and a predetermined association. The predetermined association is associated in advance with the environmental information of the electronic device and the content of the control. Therefore, the electronic device can perform control related to content presentation based on the acquired environmental information of the electronic device. Thus, the electronic device can control the presentation of content according to the environment around the electronic device.

Also, the environmental information includes specific information such as the presence or orientation of a person around the electronic device, the direction of the person around the electronic device, or the brightness or volume of the surroundings. Information about people around the electronic device may be environmental information by capturing a person's line of sight and expression using a camera. Also, the electronic device can be specifically configured using a sensor that acquires environmental information. Based on such a specific configuration, the electronic device can perform control related to the presentation of content according to the environment around the electronic device.

In addition, environmental information is combined with specific information such as the presence or orientation of people around the electronic device, the direction of people around the electronic device, or the brightness or volume of the surroundings. Control related to presentation of content according to the environment around the electronic device can be performed more appropriately.

Also, the electronic device can select a content to be presented from among a plurality of contents as control related to the presentation of the contents.

In addition, when the electronic device selects content to be presented from a plurality of content, the electronic device can select the content based on the attribute information of the content.

In addition, when the electronic device selects content to be presented from among a plurality of content, the electronic device is based on permission information indicating whether editing, adjustment, or processing of the content at the time of content presentation is permitted. The content can be selected. This avoids presentation of content in an unacceptable manner.

In addition, the electronic device can start, stop, or end the presentation of content as control related to the presentation of content.

In addition, the electronic device can change the volume or brightness of the content as control related to the presentation of the content.

Also, the environmental information includes specific information on the attitude of the electronic device. Then, the electronic device can perform control related to presentation of content based on the attitude of the electronic device.

Also, the environmental information includes specific information indicating whether or not a plurality of electronic devices are arranged side by side. Then, the electronic device can perform control related to presentation of content based on how the electronic device is arranged.

In addition, the electronic device outputs log information indicating the number of times content is presented, the time of presentation, and the like. The output log information is fed back to, for example, a content creator, and can be utilized when content is improved or new content is created. At this time, it is more preferable that the installation status and environment information where the electronic device is installed together with the log information is fed back to the content creator.

In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the electronic device of each of the above embodiments is a program as follows.

In other words, this program is a method for controlling an electronic device, and obtains environmental information of the electronic device including information on an environment around the electronic device or information on an installation mode of the electronic device. The presentation of content by the electronic device associated with the acquired environment information by referring to a predetermined association between the environmental information of the electronic device and the details of the control related to the presentation of the content by the electronic device A control method for performing control related to the above is executed.

As mentioned above, although the electronic device etc. which concern on the one or several aspect were demonstrated based on embodiment, this indication is not limited to this embodiment. Unless it deviates from the gist of the present disclosure, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

The device control method for an electronic device according to the present disclosure can be installed in any electronic device in and outside the home. In addition, the electronic device and the device control method according to the present disclosure can be mounted on a vehicle such as an automobile or a moving body such as an airplane.

10 user 20 creator 100 group 101, 101a, 101b, 101c, 101d, 101e equipment 102 home gateway 103 remote control 110 data center operating company 111 cloud server 120 service provider 121 server 201, 201c, 201F, 201BH, 201BL sensor 202, 202d output Unit 202e Input / output unit 203, 302, 1301, 1303, 1401 Control processing unit 203A, 1303A, 1402 Viewing environment estimation unit 203B, 1403 Control content determination unit 204, 1305, 1404 Content management unit 204A Viewing environment storage DB
204B Content DB
205 Upper speaker 206 Lower speaker 207, 1302, 1302c, 1302d, 1302e, 1304 Information transmission / reception unit 301 Acquisition unit 801, 901A, 901B, 901C, 901D, 1001, 1101, 1201 Table 1405 Preference trend DB
2201 Application 2202 OS
2203 Data center 2204 Service provision C205, C206 Audio F1 Hook H1 Hook hook P101 Exterior frame P102 Structure frame P103 Back cover P104 Decoration member P111 Speaker net P112 Speaker base plate P113 Speaker P121 Cover glass P122 Display panel

Claims (14)

1. An electronic device control method comprising:
   Obtaining environmental information of the electronic device, including information on the environment around the electronic device, or information on an installation mode of the electronic device,
   The presentation of content by the electronic device associated with the acquired environment information by referring to a predetermined association between the environmental information of the electronic device and the details of control related to the presentation of the content by the electronic device Control method to control.
2. The environmental information is information about the environment around the electronic device, such as the presence / absence of a person around the electronic device, the direction of the person around the electronic device, the direction of the person around the electronic device, the electronic Including information indicating the brightness around the device or the volume around the electronic device,
   Acquisition of the environmental information
   The control method according to claim 1, wherein the control method is performed by a sensor that senses an environment around the electronic device and outputs an environment value indicating the environment.
3. The environmental information includes the presence / absence of a person around the electronic device, the direction of the person around the electronic device, the direction of the person seen from the electronic device, the brightness around the electronic device, and the electronic Including information indicating at least two of the volume around the device,
   The control method according to claim 2, wherein the control related to the presentation of content by the electronic device is performed in response to information indicating the at least two.
4. In the control related to the presentation of the content,
   By referring to the predetermined association, the content to be presented by the electronic device, which is the content of the control associated with the environment information acquired by the sensor, is selected, and the selected content is The control method according to claim 2 or 3, wherein control for presenting is performed.
5. The predetermined association includes association between environment information of the electronic device and attribute information indicating an attribute of content to be presented by the electronic device having environment information matching the environment information,
   In the control related to the presentation of the content,
   The control method according to claim 4, wherein content having attribute information associated with the acquired environment information is selected by referring to the predetermined association.
6. The predetermined association includes content to be presented by the electronic device and permission information indicating a permitted presentation mode of the content,
   The control method according to claim 5, wherein in the control related to the presentation of the content, a content that can be presented in a presentation mode permitted by the permission information is selected.
7. The predetermined association includes association between information indicating a change in environmental information of the electronic device and start, stop, or end of presentation of content,
   In the control related to the presentation of the content,
   The presentation of content to be presented by the electronic device associated with the change in the acquired environment information is started, stopped, or terminated by referring to the predetermined association. 2. The control method according to item 1.
8. The predetermined association includes an association between information indicating a change in environmental information of the electronic device and a change in volume or brightness of content,
   In the control related to the presentation of the content,
   The volume or brightness of the content to be presented by the electronic device associated with the change in the acquired environment information is changed by referring to the predetermined association. The control method according to item.
9. The environment information includes information indicating an attitude of the installed electronic device as information on an installation mode of the electronic device,
   The predetermined association includes an association between information indicating the posture of the electronic device and control related to presentation of content by the electronic device in the posture;
   The control method according to any one of claims 1 to 8, wherein in the control related to the presentation of content, the control related to the presentation of content associated with information indicating the attitude of the electronic device is performed.
10. The environmental information includes juxtaposition information that is information on whether or not the electronic device is installed side by side with other electronic devices as information on the installation mode of the electronic device,
    The predetermined association includes an association between juxtaposition information of the electronic device and control related to presentation of content by the electronic device installed in an installation mode indicated by the juxtaposition information,
    The control method according to any one of claims 1 to 9, wherein in the control relating to the presentation of the content, the control relating to the presentation of the content associated with the juxtaposition information of the electronic device is performed.
11. In the control method, further,
    The control method according to any one of claims 1 to 10, wherein log information relating to presentation of the content is output.
12. An electronic device control system,
    An acquisition unit for acquiring environmental information of the electronic device, including information on an environment around the electronic device, or information on an installation mode of the electronic device;
    The presentation of content by the electronic device associated with the acquired environment information by referring to a predetermined association between the environmental information of the electronic device and the details of control related to the presentation of the content by the electronic device A control system comprising a control processing unit that performs control.
13. Electronic equipment,
    An acquisition unit for acquiring environmental information of the electronic device, including information on an environment around the electronic device, or information on an installation mode of the electronic device;
    The presentation of content by the electronic device associated with the acquired environment information by referring to a predetermined association between the environmental information of the electronic device and the details of control related to the presentation of the content by the electronic device An electronic device comprising a control processing unit that performs control.
14. A program for causing a computer to execute the electronic device control method according to claim 1.

Images