DE112017003398T5 - INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING PROCESS AND PROGRAM - Google Patents

Abstract

[Problem] To provide a mechanism capable of suppressing a fluctuation in user satisfaction with the operation object in accordance with the situation of the user. [Solution] An information processing apparatus includes: a detection unit that obtains information on a body shape of an operator; and a display control unit that controls the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object existed in a real space based on body shape information. An information processing method includes, using a processor: obtaining information about a body shape of an operator; and controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object were present in a real space on the basis of body shape information. A program realizes the functions of the information processing apparatus.

Description

Technical area

The present disclosure relates to an information processing apparatus, an information processing method, and a program.

State of the art

In recent years, with the development of information communication technologies, a variety of interfaces for device control have been researched and developed. As such, there is an interface in which a virtual object is displayed.

For example, an invention in which operations of an information processing apparatus corresponding to the movements of a user relative to a projected image are executed is disclosed in Patent Document 1. Further, a change of the projection image corresponding to the user operations performed on the projected image is described in Patent Document 1 (US Pat. 41 and the like). In the following, the projected image is also called an operation object.

List of quotations

patents

Patent document 1: JP 2011-188023A

Disclosure of the invention

Technical problem

However, according to Patent Document 1, regardless of the situation of a user, a predetermined operation object is projected. This also applies to the operating object, which changes in response to operations. This is because the operation object changes in the same way when the same operation is performed. However, it is desired that the operability and the like of the operation object be maintained even if the situation of the user changes. Therefore, the present disclosure proposes a mechanism capable of suppressing a variation of user satisfaction with the operation object according to the situation of the user.

Technical solution

According to the present disclosure, there is provided an information processing apparatus including: a detection unit that obtains information about a body shape of an operation subject; and a display control unit that controls the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object existed in a real space based on body shape information.

In addition, according to the present disclosure, there is provided an information processing method that includes, using a processor, obtaining information about a body shape of an operation subject; and controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object were present in a real space based on body shape information.

In addition, according to the present disclosure, a program is provided for causing a computer system to realize: a detection function of acquiring information on a body shape of an operation subject; and a display control function of controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object were present in a real space based on information about the body shape.

Advantageous Effects of the Invention

According to the present disclosure, there is provided a mechanism capable of suppressing a variation of user satisfaction with the operation object according to the situation of the user as described above. It should be noted that the effects described above are not necessarily limiting. With or instead of the above effects, any of the effects described in this specification or other effects that can be taken from this specification can be achieved.

list of figures

• 1 FIG. 10 is a diagram for describing features of an information processing system according to an embodiment of the present disclosure. FIG.
• 2 FIG. 10 is a flowchart illustrating principles of processing by the information processing system according to an embodiment of the present disclosure.
• 3 FIG. 12 is a block diagram illustrating an example of a functional configuration of an information processing system according to a first embodiment. FIG Embodiment of the present disclosure schematically illustrated.
• 4 Fig. 12 is a diagram for describing an example of a first device selection by which a device to be operated is selected in the information processing system according to the embodiment.
• 5 Fig. 12 is a diagram for describing an example of a first device selection by which a plurality of devices to be operated are selected in the information processing system according to the embodiment.
• 6 FIG. 12 is a diagram for describing an example of a second device selection by which an operation target device is selected on the basis of an operation performed on the selection object in the information processing system according to the embodiment.
• 7 FIG. 12 is a diagram for describing an example of priority information in the information processing system according to the embodiment. FIG.
• 8th FIG. 12 is a diagram illustrating an example of a message to be operated devices selected by the first device selection in the information processing system according to the embodiment. FIG.
• 9 Fig. 12 is a flowchart conceptually illustrating an example of the overall processing by the information processing system.
• 10 FIG. 10 is a flowchart conceptually illustrating an example of the processing of the selection object display by the information processing system according to the embodiment. FIG.
• 11 FIG. 12 is a diagram for describing an example of a first device selection in an information processing system according to a first modification example of the embodiment. FIG.
• 12 FIG. 12 is a diagram for describing another example of the first device selection in the information processing system according to the first modification example of the embodiment. FIG.
• 13 Fig. 12 is a diagram for describing still another example of the first device selection in the information processing system according to the first modification example of the embodiment.
• 14 FIG. 10 is a flowchart conceptually illustrating an example of the processing of the selection object display by an information processing system according to a second modification example of the embodiment. FIG.
• 15 FIG. 12 is a diagram illustrating an example of displaying selection items in an information processing system according to a third modification example of the embodiment. FIG.
• 16 FIG. 12 is a diagram illustrating another example of the display of the selection objects in the information processing system according to the third modification example of the embodiment. FIG.
• 17 FIG. 10 is a flowchart conceptually illustrating an example of the processing of the selection object display by an information processing system according to a fourth modification example of the embodiment. FIG.
• 18 FIG. 10 is a flowchart conceptually illustrating another example of the processing of the selection object display by the information processing system according to the fourth modification example of the embodiment. FIG.
• 19 FIG. 12 is a diagram illustrating an example of a message to be operated devices selected by the first device selection in an information processing system according to a fifth modification example of the embodiment. FIG.
• 20 FIG. 12 is a diagram illustrating another example of a message to be operated devices selected by the first device selection in the information processing system according to the fifth modification example of the embodiment. FIG.
• 21 FIG. 12 is a diagram illustrating still another example of a message to be operated devices selected by the first device selection in the information processing system according to the fifth modification example of the embodiment. FIG.
• 22 FIG. 10 is a diagram illustrating an example of the operation control display control in an information processing system according to a second embodiment of the present disclosure. FIG.
• 23 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to the embodiment. FIG.
• 24 FIG. 10 is a flowchart conceptually illustrating an example of processing by the information processing system according to the embodiment. FIG.
• 25 FIG. 12 is a diagram illustrating an example of the operation control display control in an information processing system according to a first modification example of the embodiment. FIG.
• 26 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to the first modification example of the embodiment. FIG.
• 27 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to a second modification example of the embodiment. FIG.
• 28 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to the second modification example of the embodiment. FIG.
• 29 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to a third modification example of the embodiment. FIG.
• 30 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to the third modification example of the embodiment. FIG.
• 31 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to a fourth modification example of the embodiment. FIG.
• 32 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to a fifth modification example of the embodiment. FIG.
• 33 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to a sixth modification example of the embodiment. FIG.
• 34 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to the sixth modification example of the embodiment. FIG.
• 35 FIG. 12 is a diagram illustrating another example of the display control for the operation objects in the information processing system according to the sixth modification example of the embodiment. FIG.
• 36 FIG. 10 is a flowchart conceptually illustrating an example of the processing of an information processing system according to a sixth modification example of the embodiment. FIG.
• 37 FIG. 12 is a diagram illustrating an example of the display control for message operation objects in an information processing system according to a seventh modification example of the embodiment. FIG.
• 38 FIG. 12 is a diagram illustrating an example of the display control for message operation objects for a plurality of users in an information processing system according to the seventh modification example of the embodiment.
• 39 FIG. 12 is a diagram illustrating an example of the display control for message operation objects that are visually recognized only by a specific user in the information processing system according to the seventh modification example of the embodiment.
• 40 FIG. 12 is a diagram illustrating another example of the display control of the notification operation objects that are visually recognized only by the specific user in the information processing system according to the seventh modification example of the embodiment.
• 41 FIG. 12 is a diagram illustrating an example of the display control for operation objects in an information processing system according to an eighth modification example of the embodiment. FIG.
• 42 FIG. 10 is a diagram illustrating an example of the operation control display control in an information processing system according to a ninth modification example of the embodiment. FIG.
• 43 FIG. 12 is a diagram illustrating an example of an operation performed on an operation object in an information processing system according to a tenth modification example of the embodiment. FIG.
• 44 FIG. 10 is a diagram illustrating an example of a displacement control of operation objects in an information processing system according to a third embodiment of the present disclosure. FIG.
• 45 FIG. 12 is a diagram illustrating an example of the followability of the operation objects relative to a reference of a display location in the information processing system according to the embodiment. FIG.
• 46 FIG. 12 is a diagram illustrating an example of the followability of the operation objects in a case where the reference of the display location in the information processing system according to the embodiment is a person.
• 47 Fig. 12 is a flowchart conceptually illustrating an example of the overall processing by the information processing system.
• 48 FIG. 10 is a flowchart conceptually illustrating an example of the processing of controlling a reference of the display location in the information processing system according to the embodiment. FIG.
• 49 FIG. 12 is a diagram illustrating an example of a predetermined operation in an information processing system according to a first modification example of the embodiment. FIG.
• 50 FIG. 12 is a diagram illustrating another example of a predetermined operation in the information processing system according to the first modification example of the embodiment. FIG.
• 51 FIG. 12 is a diagram illustrating still another example of a predetermined operation in the information processing system according to the first modification example of the embodiment. FIG.
• 52 FIG. 12 is a diagram illustrating still another example of a predetermined operation in the information processing system according to the first modification example of the embodiment. FIG.
• 53 FIG. 12 is a diagram illustrating still another example of a predetermined operation in the information processing system according to the first modification example of the embodiment. FIG.
• 54 FIG. 12 is a diagram illustrating an example in which a reference of a display location in an information processing system according to a second modification example of the embodiment is changed. FIG.
• 55 Fig. 10 is a flowchart conceptually illustrating an example of the processing of controlling a reference of a display location in an information processing system according to a third embodiment.
• 56 FIG. 12 is a diagram illustrating an example in which a goal of changing a reference of a display location of an operation object in an information processing system according to a fourth modification example of the embodiment is expressly given. FIG.
• 57 FIG. 12 is a diagram illustrating an example of mixing operation items in an information processing system according to a fifth modification example of the embodiment. FIG.
• 58 FIG. 10 is a diagram illustrating an example of the operation control display control in an information processing system according to a sixth modification example of the embodiment. FIG.
• 59 FIG. 12 is a diagram illustrating an example of the operation control display control in an information processing system according to a seventh modification example of the embodiment. FIG.
• 60 FIG. 12 is a diagram illustrating an example of the display control for operation items for reproduction in an information processing system according to a seventh modification example of the embodiment. FIG.
• 61 FIG. 10 is an explanatory diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment of the present disclosure. FIG.

Description of the embodiments

Hereinafter, a preferred embodiment (s) of the present disclosure will be described in detail with reference to the attached drawings. It should be noted that in this specification and the appended drawings, structural elements having substantially the same function and structure are denoted by the same reference numerals, and repeated explanation of these structural elements is omitted.

In addition, in some cases, various numbers are appended to the same reference numerals in the specification and the drawings to distinguish a plurality of elements having substantially the same functions. For example, a plurality of elements having substantially the same functions are selected as a selection object as needed 31A , Selection object 31B and the like. However, in cases where it is not necessary to distinguish elements that are essentially have the same function, assign the same reference numerals. In a case where it is not necessary, the selection object 31A and the selection object 31B to distinguish, for example, the selection object 31A and the selection object 31B simply as a selection object 31 designated.

In addition, information processing devices 100 according to the first to third embodiments, as an information processing apparatus 100 - 1 to information processing device 100 - 3 by adding, for convenience of description, the numbers according to the embodiments at the end.

1. 1. Einführung
2. 2. Grundzüge eines Informationsverarbeitungssystems gemäß einer Ausführungsform der vorliegenden Offenbarung
3. 3. Erste Ausführungsform der vorliegenden Offenbarung (Auswahl der zu bedienenden Vorrichtung)
   • 3-1. Konfiguration des Systems
   • 3-2. Verarbeitung durch das System
   • 3-3. Kurzdarstellung der ersten Ausführungsform
   • 3-4. Modifikationsbeispiele
4. 4. Zweite Ausführungsform der vorliegenden Offenbarung (Anzeige von Bedienungsobj ekten)
   • 4-1. Konfiguration des Systems
   • 4-2. Verarbeitung durch das System
   • 4-3. Kurzdarstellung der zweiten Ausführungsform
   • 4-4. Modifikationsbeispiele
5. 5. Dritte Ausführungsform der vorliegenden Offenbarung (Verlagerung von Bedienungsobj ekten)
   • 5-1. Konfiguration des Systems
   • 5-2. Verarbeitung durch das System
   • 5-3. Kurzdarstellung der dritten Ausführungsform
   • 5-4. Modifikationsbeispiele
6. 6. Hardwarekonfiguration gemäß einer Ausführungsform der vorliegenden Offenbarung
7. 7. Schlussfolgerung

It should be noted that the description is given in the following order.

1. 1. Introduction
2. 2. Principles of an information processing system according to an embodiment of the present disclosure
3. 3. First embodiment of the present disclosure (selection of the device to be operated)
   • 3-1. Configuration of the system
   • 3-2. Processing by the system
   • 3-3. Brief description of the first embodiment
   • 3-4. modification Examples
4. Second Embodiment of the Present Disclosure (Display of Operating Objects)
   • 4-1. Configuration of the system
   • 4-2. Processing by the system
   • 4-3. Brief description of the second embodiment
   • 4-4. modification Examples
5. 5. Third Embodiment of the Present Disclosure (Relocation of Operating Objects)
   • 5-1. Configuration of the system
   • 5-2. Processing by the system
   • 5-3. Outline of the third embodiment
   • 5-4. modification Examples
6. 6. Hardware configuration according to an embodiment of the present disclosure
7. 7. Conclusion

<Introduction>

First, technologies related to information processing systems according to the respective embodiments of the present disclosure will be described.

Electronic devices (hereinafter also referred to as devices to be operated), such as household electrical appliances, are generally operated using remote controls. For example, remote controls are provided for the respective devices to be operated, and a user operates a device to be operated which is an operation target by using a remote control for the device to be operated.

However, as the number of remote controls increases as the number of devices to be operated increases, there is a concern that it may be complicated for the user to operate the devices to be operated. For example, the effort of the user to find a remote control for a desired device to be operated increases. Also, the handling of the remote controls becomes an additional burden.

Meanwhile, the operation of a variety of devices to be operated by means of a device corresponding to the remote controls, conceivable. For example, the switching of GUIs (Graphical User Interfaces) for operating the devices to be operated according to the devices to be operated in a device in which a GUI is displayed can be exemplified.

However, it remains complicated for the user in this case. For example, the user must manually pick up said device to operate the devices to be operated. In addition, since a desired GUI is selected from the GUIs for the plurality of devices to be operated, as the number of devices to be operated increases, so does the number of GUIs, and the selection of GUIs becomes complicated. In addition, therefore, it takes time until it becomes possible to perform an operation. Further, in a case where there are a plurality of users, it is necessary to provide said devices according to the number of users.

In consideration of the above points, it is conceivable to provide a user interface in which said devices are not used. For example, devices to be operated may be exemplified, which may be a so-called NUI (natural user interface) such as voice, line of sight or recognize gestures of the user and perform an operation that the user desires.

However, in a case where a plurality of devices to be operated exist, it is generally difficult to select a desired device to be operated via a NUI. For example, when a user attempts to select a device to be operated by means of a line of sight, it may be that due to the line of sight deviation, a line of sight is detected not only to the desired device to be operated but also to devices to be operated in the environment and a misoperation occurs. Also, limiting the deviation in a line of sight is a burden on the user. Further, in a case where an operation of a device to be operated, in particular, a detailed operation such as the setting of parameters and the like is performed, the load further increases even if the desired device to be operated is selected.

However, the use of a projected GUI instead of the above-mentioned NUI can be conceivable. For example, projecting a GUI in a user's environment and operating a device to be operated in accordance with operations performed on the projected GUI may be exemplified.

However, the difficulty in selecting a device to be operated is still not solved in this case.

Therefore, an embodiment of the present disclosure proposes a mechanism capable of reducing a user's burden in selecting a device to be operated that the user wants to operate. In addition, there is also proposed a mechanism capable of performing operation objects suitable for user's conditions and a mechanism capable of performing operation on a display location of the operation objects in a sense of moving real objects. provide.

<Features of an information processing system according to an embodiment of the present disclosure>

Next, features of an information processing system according to an embodiment of the present disclosure will be described with reference to FIG 1 described. 1 FIG. 12 is a diagram for describing the principles of the information processing system according to an embodiment of the present disclosure. FIG.

The information processing system according to the one embodiment of the present disclosure includes an information processing apparatus 100 comprising a user shape recognition function, a projection control function, a device control function, and a communication function, a projection device, an imaging device, and devices to be operated. The user shape recognition function is a function of recognizing a body shape of the user. The projection control function is a function of controlling a shape, a projection location and the like of an image to the projection of which the projection apparatus is caused. The device control function is a function of controlling the processing of the devices to be operated. The communication function is a function of communicating information to devices or devices outside the information processing device 100 , Therefore, the information processing apparatus can 100 to control serving devices, which are communicatively connected, in response to an operation using the body of a user on the image to operate the devices to be operated, to the projection of which the projection device is caused (hereinafter also referred to as operation objects).

Further, the information processing apparatus has 100 in addition to the above-mentioned functions, a selection function for the device to be operated, an operation object shape control function and an operation object movement function. The selection function for the device to be operated is a function of selecting a device to be operated, which is an operation target, from a variety of devices to be operated. The operation object display function is a function of controlling a display form of operation items to be displayed. The operation object moving function is a function of controlling a displacement of the operation objects to be displayed.

Such as in 1 illustrates, includes the information processing system 1 an information processing device 100 , a projection imaging device 10 , a display device 20 and an air conditioning device 21 , The information processing device 100 is over a network such as the Internet with the projection imaging device 10 , the display device 20 and the air conditioning device 21 connected. It should be noted that the projection imaging apparatus 10 may be a projection device and an imaging device as separate devices.

First, the information processing apparatus selects 100 the display device 20 and the air conditioning device 21 as to-use devices. Next, the information processing apparatus causes 100 the projection imaging device 10 , Operation objects for operating the selected display device 20 or the air conditioning device 21 to project. In addition, the projection imaging device forms 10 an area in which the user falls, and sends image information on the image obtained by the image to the information processing apparatus 100 , The information processing device 100 recognizes the operation of a user from a body shape of the user, which is recognized on the basis of the received image information. Then, the information processing apparatus causes 100 a display location of the operation objects to move in accordance with the operation by the user. In addition, the information processing apparatus controls 100 the processing of the display device 20 or the air conditioning device 21 based on the operation by the user performed on the operating objects.

Furthermore, principles of processing by the information processing system 1 with reference to 2 described. 2 is a flowchart that outlines the processing by the information handling system 1 in accordance with an embodiment of the present disclosure.

The information processing system 1 selects a device to be operated representing an operation target (step S201 ). In particular, the information processing apparatus selects 100 an operator-to-use device that represents an operator's goal, from a variety of devices to be operated, which are communicatively connected.

Next, the information processing system 1 determines whether one or more devices to be operated have been selected (step S202 ). In particular, the information processing system provides 100 determines whether one or more devices to be operated have been selected as operating targets.

When it is determined that one or more devices to be operated have been selected, the information processing system displays 1 Operating objects (step S203 ). In particular, the information processing apparatus causes 100 the projection imaging device 10 To project operation objects for the device to be operated selected as the operation target.

Next, the information processing apparatus causes 1 the operating objects to move (step S204 ). In particular, the information processing apparatus causes 100 a projection location of the operation objects to move according to the operation by a user who has been performed on the projected operation objects.

In addition, the information processing system operates 1 a device to be operated representing the operation target (step S205 ). In particular, the information processing apparatus causes 100 the device to be operated, which corresponds to an operation object, to execute processing in accordance with the operation by the user performed on the projected operation object.

Then put the information processing system 1 determines if the operation is finished (step S206 ). In particular, the information processing system provides 100 determines whether the operation of the device to be operated is completed by the operation object. It should be noted that the information processing system 1 the completion of the display of the operating object causes when it is determined that the operation is completed.

The above is a description of the principles of the information processing system according to an embodiment of the present disclosure. Next, the above-mentioned selection function for the device to be operated, the operation object shape control function, and the operation object motion function will be described.

<First Embodiment of the present disclosure (Selection of the device to be operated)>

First, an information processing system 1 according to an embodiment of the present disclosure. In the first embodiment of the present disclosure, an embodiment of the above-mentioned selection function for the device to be operated in the information processing system will be mainly described 1 described.

<System configuration>

A functional configuration of the information processing system 1 according to the embodiment, with reference to 3 described. 3 Fig. 12 is a block diagram illustrating an example of the functional configuration of the information processing system 1 according to the first Embodiment of the present disclosure schematically illustrated.

The information processing system 1 includes an information processing device 100 - 1 , a projection imaging device 10 and a display device 20 and an air conditioning device 21 as devices to be operated as in 3 illustrated. Hereinafter, functions of the information processing apparatus will be described 100 - 1 and the projection imaging device 10 described in detail.

(Information processing device)

The information processing device 100 - 1 includes a recognition unit 101 , a device selection unit 102 , a projection controller 103 , a device control unit 104 , a communication unit 105 and a storage unit 106 ,

(Detection Unit)

The recognition unit 101 recognizes a form of a user as the operator. In particular, the recognition unit recognizes 101 a body shape of the user based on viewing information. More specifically, the viewing information is image information about an image in which the user appears, and the recognition unit 101 recognizes the body shape of the user by analyzing an image related to the image information. For example, the recognition unit recognizes 101 a face of the user or eyes of the user appearing in an image, and recognizes a line of sight of the user based on the arrangement, shape, and the like of the recognized face or eyes. It is to be noted that the viewing information may be measurement information on a movement, position or the like of the user and the recognition unit 101 can recognize the body shape of the user based on the measurement information. The body shape includes a form of visual recognition such as a line of sight or a field of view. In addition, the measurement information may be acquired from a sensor worn by the user on the body, a sensor mounted on an object present in the user's environment, or the like.

In addition, the recognition unit recognizes 101 the operation by a user based on the recognized form of the user. In particular, the recognition unit recognizes 101 an operation performed on a device to be operated based on the shape of the user with respect to an operation object to project the projection device 103 is initiated. For example, when an operation of touching an operation object is detected, the detection unit recognizes 101 in that an operation has been carried out on the operating object.

(Device selection unit)

The device selection unit 102 selects a device to be operated representing an operation target (hereinafter also referred to as an operation target device) on the basis of the body shape of the user. In particular, the device selection unit selects 102 the operation target device based on the body shape of the user who is directed to the device to be operated in the first device selection. For example, the device selection unit selects 102 an operator aiming device based on a visual recognition detected form of a user who is directed to a device to be operated. Further, the first device selection will be described with reference to FIG 4 described in detail. 4 Fig. 12 is a diagram for describing an example of the first device selection by which a device to be operated in the information processing system 1 is selected according to the embodiment.

The device selection unit 102 sets a device selection area based on the user's line of sight information from the recognition unit 101 is recognized. For example, the device selection unit sets 102 an area in a real space that corresponds to the user's field of vision as in 4 as the device selection area based on line of sight information provided by the recognition unit 101 to be provided. It should be noted that the device selection area may be smaller than that of the user's estimated view.

Next, the device selection unit selects 102 a device to be operated, which is determined to fall in the device selection area according to the decision operation of the user. For example, when the user's decision operation for the device selection area is made by the recognition unit 101 is detected sets the device selection unit 102 determines whether a device to be operated exists within the specified range. Then the device selection unit selects 102 the device to be operated, whose presence is detected in the specified range, for example, the display device 20 as in 4 illustrated as an operation target device. It should be noted that position information of the device to be operated in the real space of the recognizer 101 can be provided or provided by an external device.

In addition, the device selection unit selects 102 in a case where a plurality of devices to be operated are included in the device selection area, the plurality of devices to be operated are selected as candidates for the operation target device (hereinafter also referred to as candidate devices). Further, in a case where a plurality of devices to be operated are selected as candidate devices, the first device selection will be explained with reference to FIG 5 described in detail. 5 Fig. 12 is a diagram for describing an example of a first device selection by which a plurality of devices to be operated in the information processing system 1 is selected according to the embodiment. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

The device selection unit 102 Defines a device selection area based on line of sight information about the user's line of sight, that of the recognition unit 101 is recognized.

Next, the device selection unit selects 102 to-use devices that are determined to fall into the device selection area. For example, the device selection unit provides 102 determines whether devices to be operated exist in the specified area. The device selection unit 102 selects the plurality of serving devices whose presence is detected in the specified range, for example, all of the display device 20 , the air conditioning device 21 and a fan device 22 , such as in 5 illustrated as candidate devices.

Although the example in which all the devices to be operated that fall in the device selection area are selected as the candidate devices is described above, a part of the devices to be operated that fall within the device selection area may be selected as candidate devices. For example, only devices to be operated at distances from the line of sight equal to or smaller than a threshold can be selected as candidate devices from the devices to be operated in the device selection area.

Although the example in which the device to be operated actually exists in the region determined from the user's body shape (ie, the device selection area) is selected by the first device selection is described above, a device to be operated, which is associated with the device selection area. In this case, it is possible to select the device to be operated even in a case where the device to be operated can not be directly visually recognized. For example, markers associated with devices to be operated are located in a real space, and the device selection unit 102 selects a device to be operated or devices associated with a mark or markers in the device selection area, as candidate devices, or as an operation target device. In addition, for example, a particular region in real space is associated with a device or devices to be operated, and the device selection unit 102 selects the device (s) to be operated associated with the particular region in the device selection area as candidate devices or as an operation target device. With the above markings provided, the user can reliably recognize the device to be operated and then perform the first device selection. If the above-mentioned particular region is provided, it is also possible to omit the above-mentioned markings and reduce the hassle or cost of a prior preparation or change in linkage.

Further, the device selection unit selects 102 an operator target device from candidate devices selected by the first device selection. In particular, the device selection unit selects 102 from the candidate devices selected by the first device selection, in a second device selection based on the selection operation by the user, the on of the projection exposure device 10 projected selection objects is performed, an operation target device. Further, the second device selection will be described with reference to FIG 6 described in detail. 6 FIG. 14 is a diagram for describing an example of the second device selection by which an operation target device is selected on the basis of an operation performed on a selection object in the information processing system according to the embodiment.

First, information representing a plurality of candidate devices selected by the first device selection is presented to the user as a selection object after the first device selection. For example, the selection objects become 31A . 32A and 33A , each one of the fan device 22 , the display device 20 and the air conditioning device 21 as in 6 respectively, selected by the first device selection, respectively, from the projection imaging device 10 in a selection object display region 30 projected.

Next, the device selection unit selects 102 the selected candidate device from the candidate devices as an operation target device when an operation of selecting a candidate device from the plurality of candidate devices from the recognition unit 101 is recognized. For example, when an operation of touching the selection object 32A from the detection unit 101 is detected, selects the device selection unit 102 the display device 20 to object to the selected selection 32A corresponds, as an operation target device.

(Projection control unit)

The projection control unit 103 serves as a display controller and controls the projection passing through the projection imaging device 10 is carried out. In particular, the projection control unit controls 103 the projection of selection objects relating to candidate devices selected by the first device selection. More specifically, the project control unit arranges 103 the projection imaging device 10 To project selection objects that indicate the candidate devices selected by the first device selection. Further, selecting objects are referred to 6 and 7 described in detail. 7 Fig. 12 is a diagram for describing an example of priority information in the information processing system 1 according to the embodiment.

When a plurality of candidate devices are selected by the first device selection, the projection controller sets 103 Forming selection objects for the plurality of candidate devices based on priority information. The priority information includes information that is determined based on a body shape in the first device selection. In particular, the projection control unit decides 103 the arrangement of the selection objects based on a line of sight of the user in the first device selection. For example, the projection controller sets 103 in accordance with the distances from the user's line of sight to the candidate devices in a three-dimensional space in the first device selection, shapes of the selection objects. In detail, the projection control unit calculates 103 a distance d1 from the user's line of sight to the fan device, 22 , a distance d2 from the line of sight of the user to the air conditioning device 21 , a distance d3 from the line of sight of the user to the display device 20 as in 7 illustrated. Then put the projection controller 103 the orientation of the selection object in ascending order or descending order of the calculated distances. It should be noted that the distances from the line of sight to the candidate devices may, of course, be spacings in a two-dimensional space. In addition, the selection objects may be placed in locations that are closer to the user with higher priority, that is, with shorter intervals as mentioned above.

It also sets the projection control unit 103 in accordance with the user's decision-making operation in the first device selection, sets a location as the projection location of the selection objects. The location according to the decision operation includes a body part of the user determined by the decision operation. In a case where, for example, a tap of the thigh of the user is performed as a decision operation for a device selection area in the first device selection, the projection control unit sets 103 a region 30 on the thigh of the user as the projection location of the selection objects. It should be noted that the projection control unit 103 determines the display location of the selection objects according to an operation of determining the display location of the selection objects that is different from the above decision operation in the first device selection.

Then, the projection controller causes 103 the projection imaging device 10 to project the selection objects in a defined form to the specified projection location. For example, the projection controller causes 103 the projection exposure apparatus 10 for the projection of the selection objects 31A . 32A and 33A in the form of a list of a specific arrangement in the region 30 on the user's thigh as a projection location as in 6 illustrated.

It should be noted that selection objects may be projected for devices to be operated related to candidate devices selected by the first device selection. In particular, the projection control unit detects 103 the devices to be operated, which operate in conjunction with candidate devices, and cause the projection imaging device 10 to project the selection items for the captured devices to be served along with the candidate object selections. For example, selection objects for a recording device, a sound output device, a lighting device or the like, which can be used in conjunction with the display device 20 works, be projected. In addition, selection objects for devices to be operated having functions similar to the functions of the candidate devices can be projected. In this case, it is possible to improve the usability by projecting selection objects for devices which the user is likely to operate.

Furthermore, the projection control unit 103 serve as a message control unit and control a message of the devices to be operated selected by the first device selection when the selection objects are projected. In particular, the projection control unit controls 103 the projection indicating a link between the devices to be served selected by the first device selection and the selection objects. Further, the message of the devices to be operated selected by the first device selection will be referred to with reference to FIG 8th described in detail. 8th Fig. 12 is a diagram showing an example of the message of the devices to be operated selected by the first device selection in the information processing system 1 illustrated according to the embodiment.

When the decision operation is detected in the first device selection, the projection control unit causes 103 the projection imaging device 10 to project a display indicating a link between a plurality of respective candidate devices selected by the first device selection and the location where the decision operation was performed. When the decision operation in the first device selection is detected as being performed on the thigh of the user, an animation of an image (including simple light) with a track following a line that is each from the display device 20 , the air conditioning device 21 and the fan device 22 according to the decision service with the region 30 on the thigh of the user joins, projected. For example, an animation in which an image from the candidate devices to the region 30 be moved, projected and the selection objects can be projected if the image is the region 30 reached.

Above, projection control has been described in the selection of devices. In addition, the projection controller controls 103 the projection of operator objects for a device to be operated selected by the second device selection. Specifically, when a selection operation of a candidate device is detected in the second device selection, the projection control unit causes 103 the projection imaging device 10 To project operator objects for the selected candidate device. For example, when a selection operation performed on a selection object is detected, the projection of the selection objects is made to end, and the operation objects may be projected to the place where the selection objects were projected.

(Device control unit)

The device control unit 104 controls a device to be operated. In particular, the device control unit controls 104 the processing of the device to be operated on the basis of an operation by the user, by the recognition unit 101 is recognized. For example, the device control unit decides 104 for the processing of the display unit 20 in accordance with an operation of an operation object for the display unit 20 and causes the communication unit 105 , a processing execution request to the display device 20 to request execution of the specified processing.

(Communication unit)

The communication unit 105 communicates with devices outside the information processing device 100 - 1 , In particular, the communication unit transmits 105 Image information to the projection imaging device 10 and receives image information from the projection exposure apparatus 10 , In addition, the communication unit sends 105 a processing execution request to the display device 20 and the air conditioning device 21 , It should be noted that the communication unit 105 can perform the communication either in a wired communication concept or in a wireless communication concept.

(Storage unit)

The storage unit 106 stores information used in the processing of the information processing apparatus. In particular, the storage unit stores 106 Information necessary for the analysis of viewing information in the recognition processing by the recognition unit 101 is performed. It also stores the storage unit 106 Image information about an image, to the projection of which the projection control unit 103 the projection imaging device 10 causes. It should be noted that information stored in an external device may be detected via communication instead of information in the memory unit 106 get saved.

(Projection imaging device)

The projection imaging device 10 projects an image based on an instruction from the information processing apparatus 100 - 1 , In particular, the projection imaging device projects 10 an image to the image information provided by the information processing device 100 - 1 be provided to a specific location. For example, the projection imaging device 10 a projector capable of rotating a projection direction with respect to two axes. It should be noted that the projection imaging device 10 may be an omnidirectional projector, a hologram video device, or a display device having an object (e.g., a table or a sofa) located in the user's environment. In addition, the projection imaging device 10 Project different images at a variety of locations simultaneously.

In addition, the projection imaging device forms 10 the environment of the device itself. In particular, the projection imaging device forms 10 the environment of the device itself at predetermined time intervals or in response to a request from the information processing device 100 - 1 from. Then the projection imaging device sends 10 Image information about the captured image by mapping to the information processing device 100 - 1 , It should be noted that an area available for imaging may be the same as an area available for projection, or may be wider than an area available for projection. In addition, an imaging area may be made to correspond to a projection area. Also, there can be a variety of imaging areas.

<Processing by the system>

Next, the processing by the information processing system 1 described according to the embodiment.

(Overall processing)

First, the overall processing by the information processing system 1 with reference to 9 described. 9 Fig. 10 is a flowchart showing an example of the overall processing by the information processing system 1 conceptually illustrated according to the embodiment.

The information processing system 1 estimates the body shape of a user (step S301 ). In particular, the recognition unit recognizes 101 the body shape of the user using image information or the like.

Next, the information processing system 1 determines whether a decision operation has been made in the first device selection (step S302 ). In particular, the detection unit sets 101 determine a device selection area based on the recognized body shape of the user. In addition, the recognition unit tries 101 to recognize the decision operation in the first device selection on the basis of the recognized body shape of the user. Then the device selection unit will turn off 102 determines whether the decision operation in the first device selection from the recognition unit 101 was detected.

When it is determined that the decision operation has been made in the first device selection, the information processing system sets 1 determines whether to-use devices have been selected (step S303 ). For example, when it is determined that the decision operation for the device selection by the recognition unit 101 is detected sets the device selection unit 102 determines whether one or more devices to be operated are present in the device selection area.

When it is determined that the devices to be operated have been selected, the information processing system provides 1 determines whether a variety of devices to be operated has been selected (step S304 ). In particular, the device selection unit provides 102 determines whether two or more devices to be operated are present in the device selection area.

When it is determined that a plurality of devices to be operated have been selected, the information processing system shows 1 Selection objects (step S305 ). If it is determined that there are two or more devices to be operated, the device selection unit selects 102 the two or more devices to be operated as a candidate device. Then, the projection controller causes 103 the communication unit 105 ; Image information on the selection objects for the candidate devices to the projection imaging device 10 to send. Then the projection imaging device projects 10 the selection objects to the received image information to a specific location. It should be noted that details will be described later.

Next, the information processing system 1 determines whether a decision operation has been made in the second device selection (step S306 ). In particular, tries the recognition unit 101 to recognize the decision operation in the second device selection. Then the device selection unit will turn off 102 determines whether the decision operation in the second device selection from the recognition unit 101 was detected.

When it is determined that the decision operation has been made in the second device selection, the information processing system shows 1 Operating objects for the device to be operated on (step S307 ). In particular, when it is determined that the decision operation in the second device selection by the recognition unit 101 was detected causes the device selection unit 102 the communication unit 105 , Image information on the operation objects to the projection imaging device 10 to send. Then the projection imaging device projects 10 the operating objects to the received image information instead of selecting objects.

(Overall processing)

Subsequently, processing of the selection object display by the information processing system 1 with reference to 10 described. 10 Fig. 10 is a flowchart showing an example of the processing of the selection object display by the information processing system 1 conceptually illustrated according to the embodiment.

The information processing system 1 collects information about the body shape of a user (step S311 ). In particular, the projection control unit detects 103 Information about the body shape of a user, that of the recognition unit 101 is recognized.

Next, the information processing system decides 1 on a form of the selection objects on the basis of the body shape information (step S312 ). In particular, the projection control unit decides 103 on priority information based on the acquired body shape information. Then the projection control unit decides 103 on the orientation of the selection objects based on the priority information.

In addition, the information processing system decides 1 about a display location corresponding to a decision operation in the first device selection (step S313 ). In particular, the projection control unit decides 103 about a place where the decision operation was made in the first device selection as a display location of the selection objects.

Then the information processing system shows 1 select objects in the specified form at the specified location (step S314 ). In particular, the projection control unit causes 103 the communication unit 105 , Image information on the selection objects in the fixed orientation together with an instruction of the projection location to the projection imaging device 10 to send. Then the projection imaging device projects 10 the selection objects to the received image information to the specific location.

<Brief description of the first embodiment>

The information processing system controls this way 1 ie the information processing device 100 - 1 according to the first embodiment of the present invention, the display of display objects to-be-operated devices selected by the first device selection based on information from which a body shape of an operation subject directed to the device-to-be-operated is estimated. Then, the information processing apparatus controls 100 - 1 a device to be operated selected by the second device selection based on information on a selection operation performed by the operator on the selection objects.

Conventionally, either a first interface serving the device to be operated has been provided near the device or a second interface remotely controlling the device to be operated. However, the user must move toward the device according to the first interface. Also, it is difficult to accurately select the device to be operated with NUI operation such as gesture operation in the second interface. Furthermore, in a case where a plurality of devices to be operated exist, an operation of remote controls has been troubled to find remote controls corresponding to the devices to be operated.

Meanwhile, according to the embodiment, candidates for a device to be operated selected by the first device selection based on the body shape of a user are presented to the user, and the user can select a device to be operated from the candidates. Therefore, it is not necessary for the user to move to the device to be operated first. In addition, it is possible to suppress a wrong selection of the device to be operated and a re-selection of the device to be operated by the user who selects an operation target from the presented candidates for the device to be operated prevent. Also, it is possible to operate the device to be operated without a special device such as a remote control and to suppress the effort for searching for the remote control or the like by operating the device to be operated based on the selection from the selection objects. Therefore, it is possible to reduce a load of the user for selecting the device to be operated that the user wants to operate.

In addition, the above-mentioned selection objects include objects indicating the devices to be operated selected by the above-mentioned first device selection. Therefore, it is possible for the user to reliably select the device to be operated that the user intends by expressly indicating the candidate devices selected by the first device selection.

In addition, the above selection objects are displayed so that the selection objects are visually recognized by a form based on the priority information. Although a plurality of candidate devices are selected by the first device selection, in practice the user generally wishes to select a device to operate. Thus, it is possible to improve the operability of the selection objects by the projection imaging device 10 is caused to project the selection objects so that the desired device to be operated is simply selected. In addition, a form of the selection objects controlled on the basis of priority information includes an arrangement of the selection objects. Therefore, it is possible to prompt the user to intuitively touch the device to be operated that the user wants to operate. Accordingly, it is possible to improve the operability of the selection objects.

In addition, the above-mentioned priority information includes information set based on information from which the above-mentioned body shape of the user is estimated in the above-mentioned first device selection. The user has already decided at the time of the first device selection on the device to be operated, which would like to operate the user. Therefore, the selection objects for the device to be operated are more likely to be selected, and the operability can be further improved by setting the shape of the selection objects according to the likelihood of being the desired device to be operated the body shape is estimated in the first device selection.

In addition, the information processing apparatus controls 100 - 1 the display of operation items for the device to be operated selected by the above-mentioned second device selection. Therefore, it is possible to operate the desired device to be operated in accordance with the user's intention. Therefore, it is possible to improve the usability in the operation of the selected device to be operated.

In addition, the above-mentioned selection objects are displayed at a location corresponding to a decision operation performed by the above-mentioned operation subject in the first device selection according to the decision operation. Therefore, it is possible to promote the selection of the desired device to be operated as a candidate device by performing the first device selection according to the user's intention. In addition, it is possible to improve the operability of the selection objects by projecting the selection objects to a location which the user intends.

In addition, the location according to the above-mentioned decision operation includes a body part of the operation subject determined by the above-mentioned decision operation. Therefore, it is possible to project the selection object to a place where the user can easily operate the selection objects even in a case where a projection space for the selection objects in the user's environment is not ensured by placing the selection objects on the body be projected by the user. Also, in a case where the projection location is tracked, it is possible to continuously operate the projected selection objects because the selection objects move even when the user moves.

In addition, the information processing apparatus controls 100 - 1 a message of the devices to be operated selected by the above-mentioned first device selection when the above-mentioned selection objects are displayed. Therefore, it is possible for the user to check the candidate devices selected by the first device selection. For example, in a case where the desired device to be operated has not been selected, the user may try the first device selection again. Therefore, it is possible to suppress a wrong selection of the operation target device and to improve the efficiency in the selection of the operation target device.

In addition, the above message includes a display output that is a link between the above-mentioned selected device to be operated and the above-mentioned selection objects. Therefore, it is possible to prompt the user to check the selected candidate devices and guide the user to the selection objects. In particular, in a case where the first device selection is performed on the basis of a visual recognition form of the user, it is possible to guide the line of sight of the user from the devices to be operated to the selection objects. Therefore, it is possible to smoothly guide the user from the first device selection to the second device selection and make it easy for him to select the operation target device.

In addition, the above-mentioned body shape includes a visual recognition form of the above-mentioned operation subject, and operated devices which are determined to be at least partially within the field of view of the operation subject are selected by the above-mentioned first device selection. Therefore, it is possible to select the devices to be operated without moving a body part of the user such as a user's hand or leg. Therefore, the user can select a device to be operated that the user wishes to operate and operate the device while performing another task or the like. In addition, in a case where the user operates the device to be operated, the line of sight of the user is generally directed to the device to be operated. Therefore, it is possible to improve the likelihood that the candidate device is the device to be operated that the user wants to operate by performing the first device selection on the basis of the visual recognition form.

<Modification Examples>

The first embodiment of the present invention has been described above. It should be noted that the embodiment is not limited to the above-mentioned example. Hereinafter, the first to fifth modification examples of the embodiment will be described.

(First Modification Example)

In a first modification example of the embodiment, the information processing system 1 perform the first device selection based on a different body shape. In particular, the information processing device performs 100 - 1 the first device selection based on an attitude of the user. More specifically, the device selection unit selects 102 in the first device selection, a device or devices to be operated, which are determined to fall within an area determined by the user's posture, as an operation target device or a candidate device. Further, the processing in the modification example will be described with reference to FIG 11 described. 11 Fig. 12 is a diagram for describing an example of the first device selection in the information processing system 1 according to the first modification example of the embodiment.

The recognition unit 101 recognizes an attitude of at least a part of the body of the user on the basis of image information or the like. For example, the recognition unit recognizes 101 an orientation of the user's face or body appearing in an image based on image information to a three-dimensional image taken by the projection imaging device 10 was received.

Next, the detection unit decides 102 via a device selection area based on the attitude of the user. For example, the device selection unit decides 102 via a device selection area as in 11 illustrated with reference to the recognized orientation of the user's face or body.

Then the device selection unit selects 102 an operation device or devices that fall within the specified selection range, as operation target device or candidate devices. For example, the device selection unit selects 102 the display device 20 that falls within the designated device selection area, as an operation target device as in 11 illustrated. It should be noted that in a case where a plurality of devices to be operated fall within the designated device selection range, the plurality of devices to be operated are selected as candidate devices.

In addition, the information processing apparatus 100 - 1 in another example, perform the first device selection based on movement of the user. In particular, the device selection unit selects 102 in the first device selection, an operation device or devices that are determined to fall within an area determined by the user's movement as an operation target device or a candidate device. Further, the processing in this other example of the modification example will be described with reference to FIG 12 described. 12 FIG. 12 is a diagram for describing another example of the first device selection in FIG Information processing system 1 according to the first modification example of the embodiment.

The recognition unit 101 detects a movement of the user on the basis of image information or the like. For example, the recognition unit recognizes 101 a gesture or action of the user appearing in an image based on image information to a three-dimensional image obtained from the projection imaging device 10 was received. The gesture includes, for example, a movement of describing a circle, double tapping, shuffling, clapping, contact with an object, and the like.

Next, the detection unit decides 102 via a device selection area based on the movement of the user. For example, if grasping a hand that is aimed at a device to be operated as in 12 is detected sets the device selection unit 102 the device selection area with reference to the hand as in 12 illustrated.

Then the device selection unit selects 102 an operation device or devices that fall within the specified selection range, as operation target device or candidate devices. In particular, the device selection unit selects 102 the display device 20 that falls within the designated device selection area, as an operation target device as in 12 illustrated. It should be noted that in a case where a plurality of devices to be operated fall within the designated device selection range, the plurality of devices to be operated are selected as candidate devices.

In addition, the information processing apparatus 100 - 1 in yet another example, perform the first device selection based on user speech generation. In particular, the device selection unit selects 102 in the first device selection, a device or devices to be operated, which are determined to fall within an area determined from the user's speech production, as an operation target device or a candidate device. Further, the processing in this still further example of the modification example will be described with reference to FIG 13 described. 13 Fig. 12 is a diagram for describing still another example of the first device selection in the information processing system 1 according to the first modification example of the embodiment.

In particular, the recognition unit recognizes 101 the speech production of the user on the basis of sound information. For example, the recognition unit recognizes 101 the presence of the user's speech production or the content of the speech production based on sound information received from a sound input device comprising the information processing system 1 included separately.

Next, the detection unit sets 102 a device selection area based on the user's speech production. For example, if the content "living room" of the user's speech production as in 13 is detected, sets the device selection unit 102 the living room as a device selection area.

Then the device selection unit selects 102 an operation device or devices that fall within the specified selection range, as operation target device or candidate devices. In particular, the device selection unit selects 102 the display device 20 , the air conditioning device 21 and the fan device 22 which are present in the designated device selection area, namely in the living room, as candidate devices. It is to be noted that in a case where only one device to be operated exists in the designated device selection area, the device to be operated is selected as the operation target device.

According to the first modification example of the embodiment, the body shape of the operation subject includes a posture of the operation subject, and the devices to be operated which are determined to fall in a region determined from the posture of the user are selected by the first device selection as above described. In a case where the user operates a device to be operated, the user is generally in an attitude corresponding to the device to be operated. Therefore, it is possible to select suitable candidate devices in the first device selection by selecting the candidate devices according to the user's posture. In addition, since there is a case where the line-of-sight detection uses the processing with a higher load than the attitude detection, it is possible to reduce the processing load as compared to the first device selection based on the visually recognized shape. Therefore, it is possible to increase the reactivity.

In addition, the body shape of the operator includes movement of the operator and devices to be operated which are determined to fall within a region determined by the movement of the user. are selected by the first device selection in another example of the modification example. Therefore, it is possible to recognize the first device selection that corresponds to the user's intention by selecting from the candidate devices according to the user's explicit movement with respect to the first device selection. Therefore, it is possible to further improve the operability in the first device selection.

In addition, the body shape of the operation subject includes voice generation of the operation subject, and operated devices which are determined to fall in a region determined from the user's speech production are selected by the first device selection in still another example of the modification example. Therefore, it is possible for the user to select a device to be operated without moving his body. Therefore, it is possible to select a device to be operated which the user wishes to operate and to operate the device while performing another task separately or without directing the line of sight to the device.

It should be noted that the first device selection may be performed in place of the user's body based on a shape of an object that the user serves. In particular, the information processing device sets 100 - 1 determines a device selection area from a shape of an object that the user operates, and selects devices to be operated in the first device selection, which fall into the device selection area according to a decision operation performed on the object.

For example, the object that the user operates may be a device such as a smartphone and the recognition unit 101 detects an orientation or the like of the smartphone. The device selection unit 102 sets a device selection area according to the recognized orientation or the like of the smartphone. Then the recognition unit recognizes 101 an operation such as a down-trimming operation performed on the smartphone. When the down-sweeping operation is detected, the device selection unit selects 102 devices to be operated in the specified device selection area. It should be noted that the detection unit 101 Can detect information on the shape of the object from the object or from another external device, instead of recognizing the shape of the above object from image information or the like.

In this case, it becomes easier to determine if the shape of the user is a shape for the first device selection compared to the user's body shape. Therefore, it is possible to eliminate the concern of wrong handling of processing in the first device selection. In addition, in a case where information on the shape of the object is obtained from the above-mentioned object, it is possible to more accurately recognize the shape of the object. Therefore, it is possible to improve the accuracy of the processing in the first device selection.

Second Modification Example

In a second modification example of the embodiment, selection objects may be displayed without a decision operation in the first device selection. In particular, sets the projection control unit 103 the body of the operation subject or the environment of the operation subject, which has a region in which the selection objects are displayed so that the selection objects are visually recognized by the operator (hereinafter also referred to as the projection-available region), and causes the projection imaging device 10 to project the selection items to the specified location.

More specifically, the projection controller searches 103 after the region available for projection, so that the selection objects are visually recognized by the user. The region available for projection may be a region of maximum width on the user's body or in the user's environment. In addition, the region available for the projection may be determined according to a degree of surface irregularity, a surface color, a texture, the presence of a pattern, or the like. For example, the region available for projection may be a flat surface or a surface having irregularity within an allowable range, a surface having color or texture uniformity within an allowable range, or a surface having no pattern. More specifically, a white flat surface may be selected as a region available for projection.

Further, the processing in the modification example will be described with reference to FIG 14 described. 14 Fig. 10 is a flowchart showing an example of the processing of the selection object display by the information processing system 1 conceptually illustrated in accordance with a second modification example of the embodiment. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

The information processing system 1 captures body shape information (step S312 and sets a shape of the selection object on the basis of the detected body shape information (step S322 ).

Next sets the information processing system 1 Sets a display location according to a region available for the display (step S323 ). In particular, the projection control unit searches 103 after a region available for projection on the body of the user or in the user's environment. When a region satisfying the conditions is found, the projection controller sets 103 set the region as the region available for projection.

Then the information processing system shows 1 the selection objects in a specified form at the specified location (step S314 ). In particular, the projection control unit causes 103 the projection imaging device 10 to project the selection objects into the region on the user's body or in the user's environment that has been set as the region available for projection.

According to the second modification example of the embodiment, the selection objects are displayed on the body of the operation subject or in the environment of the operation subject having a region in which the selection objects are displayed so that the selection objects are visually recognized by the operation subject as described above. Therefore, it is possible to project the selection objects without the user's decision operation in the first device selection. Therefore, it is possible to perform the first device selection even in a situation where the user can not make the decision operation, for example, when the user performs a task.

(Third Modification Example)

In a third modification example of the embodiment, the display form of the selection objects may be another form. In particular, the display location of the selection objects may be another location. For example, the selection objects may be displayed on a display unit determined by a decision operation in the first device selection. Specifically, the projection controller causes 103 in a case where an operation target of the decision operation in the first device selection is the display device, the display device displays the selection objects. Further, the processing in the modification example will be described with reference to FIG 15 described. 15 is a diagram showing an example of the display of the selection items in the information processing system 1 according to the third modification example of the embodiment.

The recognition unit 101 recognizes an operation target of the decision operation in the first device selection. For example, when the decision operation is detected in the first device selection, the recognition unit recognizes 101 the operation target for the decision operation.

In a case where the recognized operation target is the display device, the projection controller sets 103 determines whether it is possible to control the display device. In a case where the recognized service destination is, for example, a smartphone 70 is like in 15 illustrates, represents the projection controller 103 determines whether it is possible to have at least one display function of the smartphone 70 to control.

In a case where it is determined that it is possible to control the display device, the projection control unit causes 103 the display device for displaying the selection objects. For example, if it is determined that it is possible the display function of the smartphone 70 to control causes the projection controller 103 the communication unit 105 to send image information about the selection objects to the smartphone 70 , The smartphone 70 shows the selection objects 31A to 33A on the basis of the received image information on the display unit.

It should be noted that operation objects for the device to be operated correspond to the selected selection object on the display unit of the smartphone 70 displayed when a selection operation on the selection objects displayed on the display unit of the smartphone 70 be displayed is performed. Then, the operation target is controlled in accordance with an operation performed on the displayed operation items.

In addition, the shape of the selection objects corresponding to priority information may be another form. In particular, the sizes of the selection objects may be set according to the priority information. For example, the projection controller sets 103 determine the sizes of the selection objects based on the body shape in the first device selection. Further, another processing in the modification example will be described with reference to FIG 16 described. 16 Figure 12 is a diagram illustrating another example of the display of the selection objects in the information processing system 1 according to the third modification example of the embodiment.

The projection control unit 103 sets the sizes of the selection objects corresponding to the distances from the user's line of sight to the candidate devices in three-dimensional space in the first device selection. For example, the projection controller calculates 103 in each case the distances from the distances from the line of sight of the user to the display device 20 , the air conditioning device 21 and the fan device 22 , Then put the projection controller 103 the sizes of the selection objects are fixed so that the sizes increase with shorter calculated distances. Specifically, the projection control unit sets 103 the size of the selection object 32B for the display device 20 with the shortest distance calculated among the candidate devices, as the largest size among the candidate devices as in 16 illustrated. It also sets the projection control unit 103 the size of the selection object 31B for the fan device 22 with the longest distance calculated among the candidate devices as the smallest size among the candidate devices as in 16 illustrated.

According to the third modification example of the embodiment, the location according to the decision operation in the first device selection includes the display unit determined by the decision operation as described above. Therefore, it is possible to ensure the visibility of the selection objects by displaying the selection objects on the display unit. In particular, in a case where the selection objects are projected, the configuration according to the modification example is advantageous because it becomes difficult to project the selection objects when an object exists between the projection location and the projection device.

In addition, the shape of the selection objects, which is controlled on the basis of priority information, includes the sizes of the selection objects. Therefore, it is possible to make it easier to detect the device to be operated that the user wants to operate. Accordingly, it is possible to improve the operability of the selection objects.

(Fourth Modification Example)

In a fourth modification example of the embodiment, the priority information on the display form of the selection objects may be other information. In particular, the priority information may be information that is determined based on biological information of the operator. More specifically, the biological information includes information about the user's pulse, body temperature, sweating, brain waves, and the like, and the projection controller 103 estimates a device to be operated that the user wishes to operate from the biological information. Then put the projection controller 103 the orientation or sizes of the selection objects so that the selection objects for the estimated device to be operated are easily selected. In a case where the body temperature of the user is lower than a normal temperature, that is, in a case where it is estimated that the user is cold, for example, selection objects are projected with such an orientation or in such sizes that selection objects for a candidate device such as an air-conditioning device or a heater can be easily selected.

In addition, the priority information may be information set based on information on an environment of the operator (hereinafter also referred to as environment information). For example, the environmental information includes information on temperature, humidity, brightness, noise, and the like, and the projection control unit 103 estimates an operator-to-use device that the user wants to operate from the environmental information. For example, in a case where the volume of a sound is higher than a threshold value, that is, in a case where it is estimated that the user feels it to be loud, selection objects having such an orientation or sizes are projected that selection objects for a candidate device such as For example, a video player or a sound output device can be easily selected.

Further, the processing in the modification example will be described with reference to FIG 17 described. 17 Fig. 10 is a flowchart showing an example of the processing of the selection object display by the information processing system 1 according to the fourth modification example of the embodiment conceptually illustrated. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

The information processing system 1 Determines if biological information or environmental information has been collected (step S331 ). In particular, the projection controller represents 103 Determines whether biological information or environmental information has been collected. It should be noted that the biological information about the communication unit 105 can be detected while the environment information about the communication unit 105 can be recorded or at the Recognition processing of the recognition unit 101 can be generated.

When it is determined that the biological information or the environmental information has been detected, the information processing system sets 1 determine a form of the selection objects on the basis of the biological information or the environment information (step S332 ). In particular, the projection control unit estimates 103 a user-to-use device that the user wishes to operate from among candidate devices from the biological information or the environmental information as the respective candidate device. Then put the projection controller 103 the shape of the candidate object's selection object, which is estimated to be the device to be operated that users want to operate, is determined to be simply selectable shape.

If it is determined that neither the biological information nor the environmental information has been detected, the information processing system detects 1 also body shape information (step S333 and sets a shape of the selection objects on the basis of the acquired body shape information (step S334 ).

Next sets the information processing system 1 determines a display location according to the decision operation in the first device selection (step S335 ) and displays the selection objects in the specified form at the specified location (step S336 ).

In addition, the priority information may be information that has been set on the basis of information on operations of devices to be operated in the past. In particular, the projection control unit detects 103 from the storage unit 106 an operation history of the user to be operated devices in the past, that of the recognition unit 101 has been detected, and estimates an operable device that the user wishes to operate from the operation history. For example, a probability with which the user wishes to operate a candidate device is estimated on the basis of time zones and operating points and operation commands to operating devices detected from the operation history, a time zone, a user's location, and a candidate device belonging to one current time is selected. Then, the selection objects are projected with such an orientation or in such sizes that the selection objects with a higher estimated probability are more easily selected.

Further, the processing in the modification example will be described with reference to FIG 18 described. 18 Fig. 13 is a flowchart showing another example of processing the selection object display by an information processing system 1 according to a fourth modification example of the embodiment conceptually illustrated. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

The information processing system 1 Determines if there are operating histories of operating devices (step S341 ). In particular, the projection controller represents 103 determines whether operations of the devices to be operated in the storage unit 106 available. It should be noted that the presence of operation histories of the candidate devices can be detected under operating histories of the devices to be operated.

If it is determined that there are service histories, the information processing system sets 1 a form of selection objects based on the operation histories (step S342 ). In particular, the projection control unit estimates 103 on the basis of the operational histories for the respective candidate devices, probabilities with which the candidate devices are the device to be operated which the user wishes to operate. Then put the projection controller 103 determine the shape of the candidate object selection object as such a form that the higher-probing devices to be operated with which the candidate devices are the device to be operated that the user wishes to operate are more easily selected.

If it is determined that neither the biological information nor the environmental information has been detected, the information processing system detects 1 also body shape information (step S343 and sets a shape of the selection objects on the basis of the acquired body shape information (step S344 ).

Next sets the information processing system 1 determines a display location according to the decision operation in the first device selection (step S345 ) and displays the selection objects in the specified form at the specified location (step S346 ).

According to a fourth modification example of the embodiment, the priority information includes information based on the biological information of the operator or the environment information of the operator Operator as described above. Therefore, it is possible to set the shape of the selection objects according to the body conditions or the body feeling of the user. Accordingly, the selection objects for the desired device to be operated are more easily selected, and it is possible to further improve the operability.

In addition, the priority information includes information that has been set on the basis of information on operations of the devices to be operated in the past. Therefore, it is possible to set a shape of the selection objects according to a tendency, a habit or the like of operations of the user. Therefore, the probability that the selection objects displayed in an easy-to-select manner are the selection objects for the desired device to be operated increases, and it is possible to further improve the operability.

It should be noted that a plurality of sets of priority item information may be combined. For example, the projection control unit 103 determine the shape of the selection objects by using a combination of at least two of the user's body shape information, biological information, environment information, and operating history. In addition, priority information can be converted into scores or the like, and the selection objects can be displayed in such a form that the candidate scores with higher scores are more easily selected.

(Fifth Modification Example)

In a fifth modification example of the embodiment, the notification of the candidate device may be realized by another method. In particular, the projection control unit causes 103 the projection imaging device 10 to perform such a projection that the devices to be operated selected by the first device selection, ie, the candidate device or the environment of the candidate devices, appear clear when the selection objects are displayed. Further, an example of the notification of the candidate devices in the modification example will be described with reference to FIG 19 described. 19 Fig. 12 is a diagram showing an example of the message of the devices to be operated selected by the first device selection in the information processing system 1 according to the fifth modification example of the embodiment.

When a decision operation is detected in the first device selection, the projection control unit causes 103 the projection imaging device 10 to perform a projection that is visually recognized so that the plurality of respective candidate devices selected by the first device selection or the vicinity of the plurality of respective candidate devices emit light. For example, when a decision operation in the first device selection is detected as being performed, the projection control unit causes 103 the projection imaging device 10 to visually project a visual effect that is visually recognized by the user, so that each of the display device 20 , the air conditioning device 21 and the fan device 22 Light emits as in 19 illustrated. It should be noted that an image or the like indicating the candidate devices, such as arrows, may be projected instead of a light emission.

It should be noted that the information processing device 100 - 1 in a case where the candidate devices have light-emitting functions that can cause candidate devices to emit light. For example, when a decision operation is detected in the first device selection, the device control unit causes 104 the communication unit 105 to send to the candidate devices directed light emission requirements. The candidate devices that have received the light emission requests cause light emitting substances that have the devices themselves to emit light for a predetermined period of time based on the light emission request. In this case, it is possible to reduce the burden of the projection processing by emitting the candidate devices themselves light as compared with the case where projection is used.

In addition, can the projection control unit 103 the projection imaging device 10 cause to perform a projection indicating locations of the candidate devices of the selection objects when the selection objects are displayed. In particular, an image indicating locations of the candidate devices may be displayed in a display region for the selection objects. Further, an example of the notification of the candidate apparatuses according to the modification example will be described with reference to FIG 20 described. 20 Fig. 12 is a diagram illustrating another example of the notification of the devices to be operated by the first device selection in the information processing system 1 were selected according to the fifth modification example of the embodiment.

When a decision operation is detected in the first device selection, the projection control unit causes 103 the projection imaging device 10 To project selection objects having images with a location of the candidate devices. For example, when a decision operation in the first device selection is detected as being performed, the projection control unit causes 103 the projection imaging device 10 , the selection objects 31C . 32C and 33C having images such as arrows representing the respective locations of the fan device 22 , the display device 20 or the air conditioning device 21 in the region 30 specify to project as in 20 illustrated.

In addition, the message from the candidate device may be an audible message. In particular, the device control unit 104 control the sound output from a region into which the devices to be operated fall. Further, another example of the notification of the candidate apparatuses according to the modification example will be described with reference to FIG 21 described. 21 Fig. 4 is a diagram illustrating still another example of the message of the devices to be operated by the first device selection in the information processing system 1 are selected according to the fifth modification example of the embodiment.

When a decision operation is detected in the first device selection, the device control unit causes 104 in a case where the candidate devices have sound output functions, the communication unit 105 for sending a sound output request directed to the candidate devices. Each of the candidate devices that has received the sound output request sounds on the basis of the sound output request, as in FIG 21 illustrated. It should be noted that the output sound may be a simple tone, music or speech. In addition, the output tone may be different or the same for each of the devices to be operated. In addition, in a case where the devices to be operated have a function of adding a directionality to the sound outputs, the sound may be output to a user.

It should be noted that, despite the above description of the case in which the candidate devices have the sound output function, the information processing system 1 may use a phenomenon such as the reflection of sound output from a separately provided sound output device to allow the user to perceive a sound as if the candidate devices output the sound.

According to the fifth modification example of the embodiment, the notification of the candidate devices includes such a display output that the candidate devices or the environment of the candidate device appear clear as described above. Therefore, it is possible for the user to grasp at a glance the devices to be operated which have been selected as candidate devices. Accordingly, it is possible for the user to determine at a glance whether the devices to be operated have been selected as intended by the user and to quickly redial the device to be operated even if a wrong selection is made in the first device selection.

In addition, the message about the candidate devices includes a display output indicating locations of the candidate devices in the selection objects. Therefore, the user can check the devices to be operated that have been selected as candidate devices based on the operation items. Therefore, the user can suppress the occurrence of an erroneous selection by checking before selecting whether to select objects for the desired device to be operated. For example, in a case where a plurality of devices of the same kind to be operated are selected as candidate devices, it may be difficult for the user to make the determination based only on the selection objects. However, it is possible for the user to select the desired device to be operated also in this case by presenting to the user the locations of the candidate devices.

In addition, the message from the candidate devices includes a sound output from a region into which the candidate devices fall. Therefore, it is possible for the user to acoustically check the devices to be operated which have been selected as candidate devices. Accordingly, even in a case where it is not possible to direct the line of sight to the candidate devices or the selection objects, it is possible to detect the candidate devices and improve the operability.

Second Embodiment of the Present Disclosure (Display of Operating Objects)

The first embodiment of the present invention has been described above. Next, a second embodiment of the present disclosure will be described. In the second embodiment of the present disclosure, an embodiment of the above-mentioned control function for the shape of the operation object in the information processing system will mainly become 1 described.

<System configuration>

First, a functional configuration of the information processing system 1 described according to the embodiment. It should be noted that only one information processing device 100 - 2 since the projection imaging device 10 is substantially the same as in the first embodiment. In addition, descriptions of functions that are substantially the same as functions in the first embodiment will be among the functions of the information processing apparatus 100 - 2 omitted.

(Detection Unit)

The recognition unit 101 serves as a detection unit and detects a body shape of the operator of the devices to be operated. In particular, the recognition unit recognizes 101 an attitude of the user as a body shape and generates body shape information to the recognized posture of the user. It is to be noted that a detailed description will be omitted because the functions are substantially the same as those of the recognition unit 101 in the first embodiment.

In addition, the recognition unit recognizes 101 a predetermined operation of the operator. In particular, the recognition unit recognizes 101 an operator's instruction operation for the operation object display. For example, the instruction operation for the operation object display may be a touch operation, a touch operation, a pointing operation, or the like.

(Projection control unit)

The projection control unit 103 serves as a display controller and controls the display of operating objects for the devices to be operated. In particular, the projection control unit controls 103 On the basis of the recognized body shape of the operating subject, a complexity of the operation objects for the devices to be operated, which are visually recognized, as if the device to be operated in a real space available. For example, the projection control device causes 103 the projection imaging device 10 , Operation objects that are visually recognized, in a complexity corresponding to one of the recognition unit 101 projected attitude of the user to project. Further, a display control for the operation objects will be described with reference to FIG 22 and 23 described. 22 Fig. 12 is a diagram showing an example of the display control for the operation objects in the information processing system 1 illustrated according to the embodiment. 23 Fig. 12 is a diagram showing another example of the display control for the operation objects in the information processing system 1 illustrated according to the embodiment.

First, the attitude of the user of the detection unit 101 detected when a given movement is performed by the user. For example, when an instruction operation of the user for the selection object display is recognized, the recognition unit recognizes 101 the attitude of the user. It should be noted that the above predetermined movement may be the selection operation in the second device selection in the first embodiment.

The projection control unit 103 sets operation objects associated with a target of the given movement as a display destination. In particular, sets the projection control unit 103 Operation objects associated with an operation target of the recognized object display selection instruction operation are set as display targets. For example, when a tap operation as an instruction operation for the selection object display is recognized as performed on the thigh of the user, the projection control unit sets 103 Operating objects that correspond to the user's thigh are set as display targets. It should be noted that, in a case where the above-mentioned specific operation is the selection operation in the second device selection in the first embodiment, the operation objects set as display targets may be operation objects for the operation target device.

It also sets the projection control unit 103 a complexity of the operation objects according to the recognized attitude of the user. The complexity of the operating objects involves a complexity of displays or complexity of operations. For example, in a case where the upper body of the seated user is tilted backward to a predetermined extent (in a so-called reclined state) as in the right graph in FIG 22 illustrates reduces the projection controller 103 the amount of display of the operation objects or selects an operation object that can be easily operated compared to a case where the user sits with the upper body tilted forward as in the right diagram in FIG 23 illustrates (in a forward tilted state). In this way, it is possible to control the complexity of the operation objects accordingly, whether the user is in a relaxed state. In other words, it is possible to arrange the operation items according to a degree of concentration of the User to control operations of a device to be operated. This is because, for example, in a case where the user is in the reclining state, the user is in a relaxed state, and it is considered that the user does not want detailed operation of a device to be operated.

It also sets the projection control unit 103 determine a location according to the body shape of the user as the display location of the operation objects. In particular, sets the projection control unit 103 a location corresponding to a predetermined movement performed by the user as the display location of the operation objects. More specifically, the display location of the selection objects may be the user's body. For example, if a tap operation is detected on the thigh of the user, the projection controller lays down 103 the region on the thigh of the user as in the left graphs in 22 and 23 Illustrates as the display region of the operating objects.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation objects with the specified complexity to the specified location. For example, in a case where the user is in the reclined state, the projection control unit causes 103 the projection imaging device 10 , the operating objects 41A . 42A and 43A as in the right diagram in 22 illustrated in the region 40 to project on the thigh of the user. The operating object 41A is an object for providing operations of activating and deactivating the display device. The operating object 42A is an object for providing operations of changing channels. The operating object 43A is an object for providing operations of changing the volume.

In a case where the user is in the front leaning state, the projection control unit causes 103 Meanwhile, the projection imaging device 10 , the operating objects 41B . 42B and 43B as in the right diagram in 23 illustrated in the region 40 to project on the thigh of the user. The operating objects 41B . 42B and 43B provide operations that are essentially the same as those of the operating objects 41A . 42A respectively. 43A , However, the operating object becomes 42B unlike the operating object 42A accompanied by a thumbnail, for example, belongs to a target channel after the change. In addition, the operating object 43B unlike the operating object 43A to change the volume as you like by moving a slider instead of changing the volume gradually.

<Processing by the system>

Next, the processing by the information processing system 1 with reference to 24 described. 24 Fig. 10 is a flowchart showing an example of processing by the information processing system 1 conceptually illustrated according to the embodiment. It is to be noted that the description of the processing which is substantially the same as the processing in the first embodiment is omitted.

The information processing system 1 captures body shape information (step S401 ) and determines a complexity of operation objects based on the acquired body shape information (step S402 ). In particular, when a predetermined movement of the user from the detection unit 101 was detected, sets the projection controller 103 from a target of the given movement projected operating objects. Then put the projection controller 103 the complexity of the display or operations of the set operation objects based on the body shape at the time when the predetermined movement is detected.

In addition, the information processing system sets 1 determine a display location of the operation objects based on the body shape information (step S403 ). In particular, sets the projection control unit 103 a location corresponding to the predetermined movement of the user by the recognition unit 101 is recognized as the display location of the operating objects.

Next, the information processing system shows 1 the operating objects with the specified complexity at the specified location (step S404 ). In particular, the projection control unit causes 103 the projection imaging device 10 to project the selected objects with the specified complexity to the specified location.

<Brief description of the second embodiment>

According to the second embodiment of the present disclosure, the information processing system gains 1 ie the information processing device 100 - 2 Information on the body shape of the operator of the devices to be operated and controls the complexity of the operating objects for the devices to be operated, which are visually recognized as if the devices to be operated in real space present, based on the body shape information as described above.

Conventionally, virtual objects are displayed that are set regardless of the user's conditions. Therefore, there is a concern that displayed virtual objects differ from virtual objects that the user desires. For example, if the same virtual objects are displayed both in a case where the user wants detailed operations and in a case where the user wants to have rough operations, the operability may be deteriorated or an operating load may increase.

However, according to the embodiment, it is possible to display the operation object in a complexity corresponding to the states of the user estimated from the body shape of the user. Therefore, it is possible to improve the possibilities that operation items that are required for an operation that the user desires are displayed in each state of the user. Accordingly, it is possible to suppress the variations in accordance with the user's conditions regarding satisfaction with the operation items.

In addition, the above-mentioned body shape includes a posture of the above-mentioned operation subject, and the above-mentioned operation objects are displayed so that the operation objects are visually recognized in a complexity corresponding to the operator's posture information. A degree of concentration of the user on an operation is generally represented in the attitude of the user. Therefore, it is possible to improve the operability of the operation objects by controlling the complexity of the operation objects according to the degree of concentration of the user on the operation estimated from the attitude of the user.

In addition, the above-mentioned selection objects are displayed so that the selection objects on the body of the above-mentioned operation subject are visually recognized. Therefore, it is possible to project the operation objects to a place where the user can easily operate the operation objects, even in a case where a projection space for the operation objects in the user's environment is not ensured by the operation objects on the body be projected by the user. Also, in a case where the projection location is tracked, it is possible to continuously operate the projected operation objects because the operation objects move even when the user moves.

In addition, the above operation items are displayed at a location corresponding to the body shape of the above-mentioned operation subject. Therefore, it is possible to improve the operability of the operation objects as compared with a case in which the operation objects are fixedly projected to a predetermined location by controlling the display location of the operation objects according to the shape of the user.

In addition, the body shape of the above-mentioned operation subject includes a predetermined movement of the above-mentioned operation subject, and the above-mentioned operation objects are displayed at a location corresponding to the predetermined movement. Therefore, it is possible to display the operation items in a location as intended by the user. Accordingly, it is possible to improve the usability of the display of the operation objects.

In addition, the above operation objects include operation objects associated with a target of the above given movement. Therefore, the user can select an operation object for the device to be operated which the user wants to operate by selecting the destination of the predetermined movement. Accordingly, it is possible to omit processes related to the display of the operation objects as compared with a case where an apparatus to be operated is selected on the basis of an additional operation.

<Modification Examples>

The second embodiment of the present invention has been described above. It should be noted that the embodiment is not limited to the above-mentioned example. Hereinafter, the first to ninth modification examples of the embodiment will be described.

(First Modification Example)

In a first modification example of the embodiment, the information processing system 1 perform the compilation of the operation objects based on information about another body shape. In particular, the information processing apparatus causes 100 - 2 the projection imaging device 10 to project the operation objects in a complexity according to biological information of the operator. For example, the biological information includes information about the user's pulse, body temperature, sweating, brain waves and the like. Further, the modification example will be described with reference to FIG 25 described in detail. 25 is a chart that one Example of the display control for operating objects in the information processing system 1 according to the first modification example of the embodiment.

When an operation operation display instruction operation is performed, the projection control unit detects 103 biological information. For example, when the instruction operation for the operation object display is performed by the recognition unit 101 is detected, the projection control unit detects 103 Information about a pulse and a body temperature as biological information of the user. The biological information is from an external device 80 captured by the user worn on the body as in 25 illustrates and biological information about the communication unit 105 generated. It should be noted that the biological information can be detected at predetermined time intervals.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation objects with the complexity set based on the biological information. For example, the projection controller causes 103 the projection imaging device 10 to display the operation objects in a complexity based on whether the acquired pulse and body temperature information is equal to or greater than the threshold values. Specifically, in a case where both the heart rate and the body temperature are lower than the threshold values, the user is regarded as being in a relaxed state and becomes the operation objects 41A to 43A with a simple display content and operating functions as in the left diagram in 25 illustrated projects. On the other hand, in a case where at least one of the pulse or the body temperature is equal to or greater than the threshold value, the user is regarded as being in a nervous state and becomes the operation objects 41B to 43B with a complicated display content and operating functions projected as in the right diagram in 25 illustrated.

In addition, the information processing apparatus causes 100 - 2 the projection imaging device 10 to project the operation objects in a complexity according to information on an action of the operation subject (hereinafter referred to as action information). For example, the action of the user includes an action that is not accompanied by a displacement, such as walking, cooking or reading, and an action that is accompanied by a displacement, such as cycling or transporting. Further, another example of the modification example will be described with reference to FIG 26 described in detail. 26 Fig. 12 is a diagram showing another example of the display control for operation objects in the information processing system 1 according to the first modification example of the embodiment.

When an operation operation display instruction operation is performed, the projection control unit detects 103 Action information. When an instruction operation for the operation object display from the recognition unit 101 is detected, the projection control unit detects 103 For example, action information about the user's transport process as in 26 illustrated by the detection unit 101 was detected.

Next, the projection controller causes 103 the projection imaging device 10 to project the operation objects in a complexity based on the action information. For example, when action information about the transportation operation is detected, the projection control unit causes 103 the projection imaging device 10 , an operating object 44 to project, indicating that an operating means as in 26 illustrates the language is. In this case, the complexity of operation becomes easier than with the manual input and the user can operate the devices to be operated even if the user has carried something and has his hands full.

According to the first modification example of the embodiment, the body shape information of the operation subject includes biological information of the operation subject, and the operation objects are displayed so that the operation objects are visually recognized in a complexity corresponding to the biological information of the operation subject, as described above. Therefore, it is possible to determine the complexity of the operation objects according to whether the user is in the relaxed state or in the nervous state, in other words, according to a degree of concentration on operations. Therefore, it is possible to reduce a feeling of strangeness, tension or the like of the user against the operation objects by adjusting the complexity of the operation object according to the condition of the user.

In addition, the above-mentioned body shape includes an action of the operation subject, and the operation objects are displayed so that the operation objects are visually recognized in a complexity corresponding to the action information of the operation subject. That's why it is the user is able to operate the operation objects smoothly without interrupting the action by displaying the operation objects in the composition suitable for the action of the user.

Second Modification Example

In a second modification example of the embodiment, the information processing system 1 perform the compilation of the operation objects on the basis of information different from the body shape information. In particular, the information processing apparatus controls 100 - 2 the composition of the operation objects based on information from which the operation subject is specified. For example, the projection controller controls 103 the composition of the operation objects based on information from which the user is specified (hereinafter also referred to as user specification information). Further, the modification example will be described with reference to FIG 27 described in detail. 27 Fig. 12 is a diagram showing an example of the display control for operation objects in the information processing system 1 according to the second modification example of the embodiment.

When an operation operation display instruction operation is performed, the projection control unit detects 103 User specification information. For example, when an operator's instruction operation for the operation object display is performed by the recognition unit 101 is detected, the projection control unit detects 103 Face recognition information as user's user specification information. The face recognition information is detected by face recognition processing of the recognition unit 101 based on image information obtained on an image in which the user appears. For example, the users U1 and U2 as in 27 Illustrates detected and facial recognition information to the users U1 and U2 detected. It should be noted that the user specification information may be information such as an ID (ID) or a name instead of the face recognition information.

Next, the projection controller causes 103 the projection imaging device 10 To project operation objects in a complexity set based on the user specification information. For example, the projection controller causes 103 the projection imaging device 10 to display the operation objects in a complexity according to the acquired user specification information. In detail, for example, for the user U1 the operating objects 41A to 43A in a complexity associated with the user whose face is in the face recognition information of the user U1 with simple display contents and operating functions as in the left diagram in 27 illustrated projects. Meanwhile, for example, for the user U2 the operating objects 41B to 43B in a complexity associated with the user whose face is in the face recognition information of the user U2 with complicated display contents and operating functions as in the right diagram in 27 illustrated projects.

In addition, the information processing apparatus 100 - 2 control the complexity of the operation objects on the basis of information from which an attribute of the operation subject is specified. In particular, the projection control unit controls 103 the complexity of the operation objects based on information specifying an attribute of the user (hereinafter also referred to as user attribute information). Further, the modification example will be described with reference to FIG 28 described in detail. 28 Fig. 12 is a diagram showing another example of the display control for operation objects in the information processing system 1 according to the second modification example of the embodiment.

When an operation operation display instruction operation is performed, the projection control unit detects 103 User attribute information. For example, if the operator's instruction operation for the operation object display is from the recognition unit 101 is detected, the projection control unit detects 103 Face recognition information as user attribute information of the user. The user attribute information is determined by attribute recognition processing of the recognition unit 101 based on image information obtained on an image in which the user appears. The user attribute includes an age, a gender, a nationality, a race, a dominant hand, or the like.

Next, the projection controller causes 103 the projection imaging device 10 to project the operation objects in a complexity set based on the user specification information. For example, the projection controller causes 103 the projection imaging device 10 for displaying the operation objects with the complexity according to the detected user attribute information. Specifically, in a case where one dominant hand indicated by the user attribute information, the operating objects 41C to 43C , the operating objects corresponding to those in the left diagram in 27 illustrated operating objects 41A to 43A reversed, projected as in 28 illustrated.

It should be noted that the display or operations of at least a portion of the service objects may be disabled based on the user specification information or the user attribute information. For example, in a case where the age of the user indicated by the user specification information belongs to the childhood, the projection controller displays 103 if applicable, at least a portion of the adult-related controls that do not address or may disable operations on the portion of the display's control objects.

According to the second modification example of the embodiment, the information processing apparatus controls 100 - 2 the composition of the operation objects based on the information from which the operation subject is specified, as described above. Therefore, it is possible to adjust the complexity of the displayed operation items to a complexity suitable for individual users. Accordingly, it is possible to further improve the operability or usability for the individual users.

In addition, the information processing apparatus controls 100 - 2 the complexity of the operation objects on the basis of information from which an attribute of the operation subject is specified. Therefore, it is possible to improve the operability or usability as compared with a case where the complexity is uniformly determined by controlling the complexity of the operation objects according to the characteristics of the user. In addition, it is possible to alleviate a concern that the security of information deteriorates because of the complexity of the operation objects being controlled without information from which the individual users are specified.

(Third Modification Example)

In a third modification example of the embodiment, the information processing system 1 control the composition of the operation objects based on information other than information of the operation subject. In particular, the information processing apparatus controls 100 - 2 the complexity of the operation objects based on information about the places where the operation objects are visually recognized. For example, the projection controller controls 103 the complexity of the operation objects based on information about the locations where the operation objects are displayed (hereinafter also referred to as display location information). Further, the modification example will be described with reference to FIG 29 and 30 described in detail. 29 Fig. 12 is a diagram showing an example of the display control for the operation objects in the information processing system 1 according to the third modification example of the embodiment. 30 Fig. 12 is a diagram showing another example of the display control for the operation objects in the information processing system 1 according to the third modification example of the embodiment.

When an operation operation display instruction operation is performed, the operation control unit 103 acquires display location information. For example, when an operator's instruction operation for the operation object display is performed by the recognition unit 101 is detected, sets the projection controller 103 a location to which the instruction operation for the display has been performed as the display location of the operation objects.

Next, the projection controller causes 103 the projection imaging device 10 to project the operation objects in a complexity determined based on the display location information. In particular, the projection control unit causes 103 the projection imaging device 10 to project the operation objects with the complexity set from a form of the designated display location of the operation objects. The shape of the display location includes a width of the display location, a degree of the irregularity, a color, a texture, a presence of a pattern, and the like. In a case where the display location of the operation objects is, for example, a palm of the user and an area of a region available for projection is smaller than a threshold, becomes a simple operation object 42D in a complexity corresponding to the area smaller than the threshold, for example, the simple operation object 42D only for entering a channel number, as in 29 illustrated projects. Meanwhile, in a case where the display location of the operation objects is a table and the area of the region available for projection is equal to or greater than a threshold, a complicated operation object becomes 45 in a complexity corresponding to the area equal to or greater than the threshold, such as the operating object 45 with which channels can be selected from a program table as in 30 illustrated on a table 60 projected.

According to the third modification example of the embodiment, the information processing apparatus controls 100 - 2 the complexity of the operation objects based on information regarding the location where the operation objects are visually recognized. Therefore, it is possible to eliminate a concern that the visibility of the operation object is deteriorated by displaying the operation objects in a complexity corresponding to the display location of the operation objects. Accordingly, it is possible to improve the operability while maintaining the visibility of the operating objects.

(Fourth Modification Example)

In a fourth modification example of the embodiment, the display location of the operation objects may be controlled on the basis of information security. In particular, the information processing apparatus causes 100 - 2 the projection imaging device 10 to project the operation objects at a location corresponding to a degree of information security with respect to operations of the devices to be operated. For example, the projection controller sets 103 specify a location corresponding to an information security level of the contents of operations by the operation objects as the display location of the operation objects. Further, the modification example will be described with reference to FIG 31 described in detail. 31 Fig. 12 is a diagram showing an example of the display control for the operation objects in the information processing system 1 according to the fourth modification example of the embodiment.

When an operation operation display instruction operation is performed, the projection control unit detects 103 Information specifying an information security level of contents of operations by the operation objects (hereinafter referred to as security information). For example, when an operator's instruction operation for the operation object display is performed by the recognition unit 101 is detected, the projection control unit detects 103 Security information for credentials like in 31 illustrated by the Bedienungsobj ekte entered.

Next sets the projection controller 103 determines a display location of the operation objects based on the security information, and causes the projection imaging device 10 to project the operating objects to the specified location. In a case where the security level indicated by the detected security information is equal to or greater than a predetermined level, the projection controller sets 103 For example, the palm of the user as the display location of the operating objects fixed. Then, the projection controller causes 103 the projection imaging device 10 , an operating object 42D to project onto the palm of the user like in 31 illustrated.

According to the fourth modification example of the embodiment, the operation objects are displayed so that the operation objects in the location are visually recognized according to a degree of information security related to operations of the devices to be operated, as described above. Therefore, it is possible to prevent others from knowing the contents of the operations using the operation objects. Accordingly, it is possible to protect the privacy of the user or to secure the information security.

(Fifth Modification Example)

In a fifth modification example of the embodiment, the above-mentioned predetermined movement may be another movement as an instruction operation for the operation object display. In particular, the information processing apparatus causes 100 - 2 the projection imaging device 10 to project the operation objects to a location corresponding to a predetermined movement directed to the devices to be operated on the basis of the predetermined movement. When a first movement directed towards devices to be operated is from the recognition unit 101 is detected, sets the projection controller 103 For example, operating objects for the devices to be operated as projection targets fixed. If then a second movement from the detection unit 101 is detected causes the projection control unit 103 the projection imaging device 10 to project the operation objects to a location corresponding to the second movement. Further, the modification example will be described with reference to FIG 32 described in detail. 32 Fig. 12 is a diagram showing an example of the display control for the operation objects in the information processing system 1 according to the fifth modification example of the embodiment.

When the first movement is detected, the projection controller lays down 103 specify the operation objects for the devices to be operated specified from the first movement as projection targets. For example, if a hand grasping movement is as in the above diagram in FIG 32 illustrated by the recognition unit 101 is detected, specifies the device selection unit 102 the display device 20 in a direction in which the hand is directed, is available. Then put the projection controller 103 the operating objects for the specified display device 20 as projection goals.

When the second movement is detected, the projection controller sets 103 Next, fixes the location corresponding to the second movement as the projection location of the operation objects, and causes the projection imaging apparatus 10 to project the operating objects to the specified location. For example, if a movement of the opening of the closed hand as in the lower diagram in 32 illustrated by the recognition unit 101 is detected, sets the projection controller 103 the palm of the hand as the projection location of the operating objects. Then becomes an operating object 42D projected onto the palm, as in 32 illustrated.

According to the fifth modification example of the embodiment, the operation objects associated with a target of a predetermined movement related to the display of the operation objects as described above include an operation object for a device to be operated as a target of the predetermined movement as described above. Therefore, it is possible to select the device to be operated which the user wants to operate in the operation object display instruction operation. Accordingly, it is possible for the user to intuitively select the device to be operated and to improve the operability.

(Sixth Modification Example)

In a sixth modification example of the embodiment, the operation objects displayed in accordance with the above-mentioned predetermined movement as the operation-object-display instruction operation may be operation-object operation objects existing in the same real space as the real space in which a target of the given movement is present. Specifically, when the operation control display instruction is recognized, the device selection unit selects 102 to-use devices that are present in the real space in which an object is present, which is the target of the instruction operation for the operation object display. The projection control unit 103 sets operation objects of the selected devices to be operated as projection targets. Then, the operation objects set as the projection targets are projected to the location according to the projection operation for the projection. Further, the modification example will be described with reference to FIG 33 to 35 described in detail. 33 to 35 are diagrams illustrating examples of the display control for the operation objects in the information processing system 1 according to the sixth modification example of the embodiment.

When the operation object display instruction operation is recognized, the detection unit recognizes 101 an object as the target of the instruction operation for the display. For example, when a tap operation is recognized as an instruction operation for the operation object display, the recognition unit recognizes 101 the table 60 which is located in a living room as in 33 illustrates in which the tapping operation has been performed, as a tapping target.

Next, when an object is recognized as a target of the instruction operation for the display, the device selection unit selects 102 to-use devices that are present in real space such as that of the object. For example, when a tap operation and a tap target from the detection unit 101 be recognized, specifies the device selection unit 102 the living room, in which the recognized as a tipping table 60 exists as a device selection area. Then the device selection unit selects 102 a lighting device, an air conditioning device, and a display device provided in the living room.

Next sets the projection controller 103 Set operation objects for the devices to be operated as projection targets and set the object as the destination of the instruction operation for the display as the projection location of the operation objects. Then, the designated operation objects are projected to the designated projection location. For example, the projection controller causes 103 that the operating objects 46 to 48 for the lighting device, the air-conditioning device, and the display device that are selected by the device selection unit 102 were selected, each on the table 60 on which the tap operation was performed, be projected as in 33 illustrated.

When an operation recognized on the projected operation object changes the projection control unit 103 Next, the display of the operation objects according to the recognized operation. If, for example, a touch control, the operating object 48 was performed by the recognition unit 101 is detected changes the projection controller 103 the projected operating objects into the operating object 48 related operating objects 49 as in 34 illustrated. The operating object 49 For example, an operation object for performing a detailed operation of the operation devices to be operated 48 be. In the example in 34 is the operating object 49 a list of reusable content.

When an operation recognized on operation objects after a change, the projection control unit changes 103 as well as the display of the operating objects according to the detected operation. If, for example, a tactile control that is attached to a part of the operating object 49 was performed by the recognition unit 101 is detected changes the projection controller 103 the projected operating objects into an on the part of the operating object 49 related operating object 50 as in 35 illustrated. The operating object 50 may be an operation object for performing detailed operations on the part of the operation object 49 be. In the example in 35 is the operating object 50 a set of controls that allows you to reproduce, pause, rewind, and fast forward, and adjust the volume. In addition, the operating object 50 Contain information related to the content.

Further, the projection controller causes 103 the projection imaging device 10 in that, for a projected first operating object, the user is to project a suggestion for operating the devices to be operated, which relates to a projected first operating object, as a second operating object. In particular, the device selection unit selects 104 a device to be operated capable of operating with respect to the device to be operated, as a first operation object, which is projected by an operation by the user. Then, the projection controller causes 103 the projection imaging device 10 to project, together with the first operating object, a second operating object that proposes to the user an operation of a device to be operated by the device selection unit 102 was selected.

For example, when a content selection operation by the operation object 49 as in 34 is detected, selects the device control unit 104 a lighting device and a sound output device capable of functioning in conjunction with a display device displaying a film with the selected content. The projection control unit 103 causes the selected lighting device and sound output device to function in conjunction with the display device, or generates an operation object 51 to suggest to the user how the lighting device and the sound output device are made to function. In detail, ( 1 ) Adjust the brightness and color rendering of the lighting according to the display, ( 2 ) Stopping sound output other than the sound related to the display, ( 3 ) Setting the tone according to the time zones (for example, suppressing strong bass in the middle of the night) and the like. Then, the projection controller causes 103 the projection imaging device 10 , the operating object 51 together with the operating object 50 to project.

In this case, it is possible to provide the user with an operation result exceeding the expectations by additionally proposing operations to an operation which the user intends. Therefore, it is possible to further improve the usability.

In addition, operation items for a device to be operated that has been selected on the basis of information about an environment in the same real space as that of the destination of the instruction operation for display (hereinafter also referred to as space environment information) can be displayed. The environment in the real space includes a temperature, a humidity, a brightness, a smell, a noise volume and the like in the real space. For example, when an instruction operation for the operation object display is performed on a table, the device selection unit selects 102 at least one air conditioning device in a case where a display device, the air conditioning device and a lighting device are provided in the same room as the table, and when the room temperature is equal to or greater than a threshold value. Then, the projection controller causes 103 the projection imaging device 10 To project operating objects for the selected air conditioning device.

In addition, operation objects for a device to be operated based on information on a shape of a person present in the same real space as that of the destination of the instruction operation for the display (hereinafter also referred to as person shape information) may be used instead of or in addition to the above space environment information is displayed. The shape of the person includes the state in the body such as a pulse or a body temperature, and states, actions and the like outside the body such as a posture. For example, when the user performs an instruction operation for the operation object display at a table, and in a case where there is a person other than the user in the same room as that of the table, the recognition unit detects 101 Body temperature information of the person. When the body temperature stated by the person's detected body temperature information is lower is as a threshold, selects the device selection unit 102 at least one air conditioning device. Then, the projection controller causes 103 the projection imaging device 10 To project operating objects for the selected air conditioning device.

In addition, operation items for a device to be operated selected on the basis of time information about a time in which the instruction operation for the display has been performed may be displayed in place of or in addition to the aforementioned space environment information or person shape information. The time information includes information indicating a time, a time zone, a date, a day, a season, or schedule information or the like. When the user performs instruction operation for the operation object display on a table, the recognition unit detects 101 Time information. When a time zone indicated by the detected time information is an evening time zone, the device selection unit selects 102 next, at least one lighting device off. Then, the projection controller causes 103 the projection imaging device 10 To project operating objects for the selected lighting device.

Further, the processing in the modification example will be described with reference to FIG 36 described. 36 Fig. 10 is a flowchart showing an example of processing by the information processing system 1 conceptually illustrated in the sixth modification example of the embodiment. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

The information processing system 1 captures body shape information (step S411 ) and determines a complexity of operation objects based on the acquired body shape information (step S412 ).

Next, the information processing system 1 determines if a given movement has been detected (step S413 ). In particular, the device selection unit provides 103 determines whether an instruction operation for the operation object display from the recognition unit 101 was detected.

When it is determined that the predetermined movement has been detected, the information processing system sets 1 determine a display location of the operation objects on the basis of the predetermined movement (step S414 ). Specifically, when it is determined that the instruction operation for the operation object display has been recognized, the projection control unit sets 103 a location corresponding to the instruction operation for the display as the projection location of the operation objects.

Next, the information processing system 1 determines whether space environment information or person shape information has been acquired for a room in which a target of the given movement exists (step S415 ). In particular, the device selection unit provides 102 determines whether information about an environment in the room where the target of the instruction operation for the display exists or information about a form of a person present in the room has been acquired. It should be noted that the room environment information is from a measuring device such as a sensor used in the information processing system 1 is provided separately, or the personal form information from the recognition unit 101 can be detected.

When it is determined that the room environment information or the personal shape information has been detected for the room in which the target of the preset operation exists, the information processing system selects 1 devices to be operated based on the room environment information or person shape information. In particular, the device selection unit selects 102 an apparatus to be operated from the devices to be operated, which is provided in the room in which the object of the above-mentioned instruction operation for the display exists on the basis of the acquired room environment information or the personal form information.

If it is determined that both the room environment information and the personal shape information for the room in which the target of the given movement exists have not been detected, the information processing system detects 1 also time information (step S417 ). Specifically, when it is determined that both the room environment information and the personal shape information have not been detected, the device selection unit detects 102 Time information about a time when the above instruction operation for the display was performed.

Next, the information processing system chooses 1 an apparatus to be operated based on the time information (step S418 ). In particular, the device selection unit selects 102 an operating device to be operated from the devices to be operated, which is provided in the space where the object of the above-mentioned instruction operation for the display exists on the basis of the detected time information.

Then the information processing system shows 1 Operating objects for the device to be operated at the specified location in the specified form (step S419 ). In particular, the projection control unit causes 103 the projection imaging device 10 to project the operating objects for the device to be operated in the specified form at the specified projection location.

Although the example in which a single device to be operated has been selected has been described above, it should be noted that a set of devices to be operated may be selected. For example, a set of devices to be operated related to air conditioning (e.g., a set of an air conditioning device and a fan device) may be a set of devices to be operated relating to the playback of movies (e.g., a set of a display device and a sound output device ) or the like.

In addition, service items may be displayed for a set of devices to be operated. For example, an operator object may be projected for each set of devices to be serviced relating to the air conditioning or any set of device-to-use related to the playback of movies. When operating an operation object for the set of devices to be operated, it is to be noted that at least a part of the devices to be operated in the set is controlled in accordance with the operation.

According to the sixth modification example of the embodiment, the operation objects associated with the target of the predetermined movement related to the display of the operation objects as described above include operation objects for the devices to be operated which are in the same real space as that of the target of the predetermined movement as described above. Therefore, the user can use the information processing system 1 cause to display operation objects for the device to be operated that the user wants to operate without selecting the device to be operated one by one. Accordingly, it is possible to reduce the time and effort required until the device to be operated is operated by reducing the operation procedure.

Moreover, the device to be operated which exists in the same real space as that of the target of the above-mentioned predetermined movement on the displayed operation object is selected on the basis of information about the environment in the same real space as that of the target of the given movement , Therefore, for example, it is possible to display operation objects for a device to be operated for maintaining or improving the environment of the room by selecting the device to be operated in accordance with the environment of the room in which the user is present. Accordingly, it is possible to improve the usability by displaying appropriate operation objects according to the state of the room.

In addition, the device to be operated is selected on the basis of information about the shape of the person who is present in the same real space as that of the target of the given movement. Therefore, for example, it is possible to display operation objects for a device to be operated for maintaining or improving a pleasant feeling felt by the person present in the room by the device to be operated in accordance with the shape of the user or another person present in the room in which the user is present is selected. Therefore, it is possible to improve the usability by displaying appropriate operation items according to the condition of the person.

In addition, the device to be operated is selected on the basis of time information. A state of a room in which a person feels comfortable differs generally in time. Therefore, it is possible to appropriately display operation objects that the user wants to operate in a state of a room that changes with time by selecting the operation device related to the operation object on the basis of time information becomes. Accordingly, it is possible to improve the usability by displaying appropriate operation items corresponding to a time.

(Seventh Modification Example)

In a seventh modification example of the embodiment, operation items may be automatically displayed. In particular, the information processing apparatus causes 100 - 2 the projection imaging device 10 to project an operation object for notification to the operator (hereinafter referred to as message operation object) in response to the reception of the message. For example, when the message is received to the user, the projection controller sets 103 a projection location of the message-related message operation object for the device to be operated. Then, the projection controller causes 103 the projection imaging device 10 , the message control object to the designated projection location to project. Further, the modification example will be described with reference to FIG 37 described in detail. 37 Fig. 12 is a diagram showing an example of the display control for the message operation object in the information processing system 1 according to the seventh modification example of the embodiment.

The device control unit 104 Determines if the message was received from the device to be served to the user. For example, the device control unit 104 determines whether call information is from a telephone call application such as an intercom, a telephone, or a smartphone, or information such as an e-mail or a message through the communication unit 105 were recorded.

When it is determined that the message has been received from the device to be operated on the user, the projection controller sets 103 determine the projection location of the message operation object based on the location of the user as the notification destination. For example, if it is determined that call information has been acquired, the projection controller sets 103 the projection location of the call information-related message operation object for the device to be operated (hereinafter also referred to simply as the message information object related to the call information) on the basis of the location of the user who is judged to be destination of the call information.

Next, the projection controller causes 103 the projection imaging device 10 to project the message control object to the specified projection location. In a case where, for example, the operating objects 46 and 48 projected in the vicinity of the user as in the upper graph in 37 illustrates causes the projection controller 103 the projection imaging device 10 , a message service object 52 to project that message message object 52 the operating objects 46 and 48 is adjacent. It should be noted that the projection location of the message operation object may be set according to the user or an attribute of the user. For example, the message service object may be projected on the side of a dominant hand of the user within a range adjacent to the service items.

When an operation of the projected message operation object is detected, the projection control unit changes 103 Further, the form of the message service object. For example, when a touch operation on the message operation object 52 from the detection unit 101 is detected, the projection control unit switches 103 the message service object 52 into a message service object 53 around. The message service object 53 For example, after switching, it may be a message service object for detailed operation of the device to be operated with the message service object 52 in relationship. Specifically, the message service object has 53 three subobjects and one display screen as in the lower diagram in FIG 37 illustrated. The subobjects may be operator objects for selecting whether to answer a call from an intercom or a telephone or the like. In addition, an image in which a person appears who is the originator of the call can be displayed on the display screen. It should be noted that the picture in which the person who originated the call also appears on the message operation object 52 before switching is displayed as in the upper graph in 37 illustrated.

An example of the processing according to the seventh modification example has been described above. Next, an example in which a plurality of users are judged to be destination of the call information will be described. In particular, the message service object is displayed at a location where the message service object is visually recognized by each of the plurality of users. For example, when the call information is received, the projection controller specifies 103 Users who are rated as destinations of call information. In the case where the plurality of users is specified, the projection control unit causes 103 the projection imaging device 10 to project the call information related message service object to a location where the message service object is visually recognized by each of the plurality of specified users. Further, an example of the display control for the message operation object for a plurality of users will be described with reference to FIG 38 described. 38 Fig. 12 is a diagram showing an example of the display control for the message operation object for the plurality of users in the information processing system 1 according to the seventh modification example of the embodiment.

When the message is received from the device to be operated on to the user, the projection controller specifies 103 the user as the message target. For example, when call information is received from an intercom or the like, the projection controller specifies 103 the user U3 to U5 as in 38 illustrates on the basis of a result of recognition of the user by the recognition unit 101 , It should be noted that Users can be specified in any individual rooms as long as the users are in a room within a given area (for example, in a building). For example, a user will also be U3 specified in a room (for example, in a room) different from that of the user U4 to U5 differs but is in the same building as in 38 illustrated.

In a case where the plurality of users is specified, the projection controller sets 103 Next, the respective projection locations of the message operation objects are determined on the basis of the plurality of positions of the respective specified users. For example, the projection controller sets 103 each places fixed to the respective position of the specified user U3 to U5 are adjacent, as projection locations of the message operation objects fixed.

Then, the projection controller causes 103 the projection imaging device 10 to project the message operation objects to the designated projection locations, respectively. For example, the respective message operation objects become 52A to 52C projected to places that correspond to the respective users U3 to U5 are adjacent, as in 38 illustrated.

Although the example in which the message operation objects are respectively displayed to each of the plurality of users has been described above, it should be noted that the plurality of users can be displayed a common message object. In particular, in a case where a plurality of users are in the same room, a common notification service object may be displayed to the plurality of users. For example, a message service object may be for the users U4 and U5 that are in the same room can be projected as in 38 illustrated. In this case, the message service object may be projected to a location where the message service object is visually owned by both users U4 and U5 is recognized, ie a place in the field of view of both the user U4 and U5 ,

Above, the example was described in which the message service object of a plurality of users is displayed. Next, an example will be described in which a message operation object is projected so that the message operation object is visually recognized only by a part of a plurality of users. In particular, the message service object is displayed in a location where the message service object is visually recognized only by certain operators. For example, when call information is received, the projection controller specifies 103 Users who are rated as destinations of call information. In a case where a single user is specified, the projection control unit causes 103 the projection imaging device 10 to project the call information related message service object to the specified location where the message service object is visually recognized only by the specified users. Further, an example of the display control for the message operation object in which the message operation object is visually recognized only by a specific user will be referred to with reference to FIG 39 described. 39 Fig. 12 is a diagram showing an example of the display control for the message operation object in which the message operation object is visually recognized only by a specific user in the information processing system 1 according to the seventh modification example of the embodiment.

When a message is received from the device to be operated on to the user, the projection controller specifies 103 a user as a message destination. When call information is received from a telephone call application or the like, the projection control unit detects 103 Destination information (for example, a telephone number) from the call information. Then specify the projection controller 103 a user associated with the captured phone number. Specifically, a user is like in 39 illustrates based on a result of the face recognition processing based on face information provided by one of the recognition unit 101 have been detected, and the face information specified from a telephone number is specified.

In a case where a single user is specified (hereinafter also referred to as a specific user) and other persons are in the room in which the particular user is located, the projection control unit sets 103 Next, a projection location of the message operation object based on each of the positions of the specific user and the other persons. For example, the projection control unit detects 103 Information about the field of view of the particular user U8 passing through one of the detection unit 101 obtained recognition, and other persons U6 and U7 as the specific user U8 , Then put the projection controller 103 a place in the field of vision of the particular user U8 and not in view of other people U6 and U7 is fixed as the projection location of the message operation object based on visual field information. It should be noted that it is at the projection of the Message handler can be about body parts such as the back of other persons.

Then, the projection controller causes 103 the projection imaging device 10 to project the message control object to the specified projection location. For example, the message service object becomes 52 projected onto a place that is in the blind spot of other people U6 and U7 located as in 39 This example demonstrates where the message service object 52 visually by the particular user U8 is recognized.

It should be noted that in the example described above, in which the single user is specified, certain users may be a plurality of users whose destination information is specified. In this case, notification operation objects may be displayed one by one for each of the plurality of specific users, or a common notification service object may be displayed for the plurality of specific users.

The above has described the case in which message operation objects are projected so that the notification operation objects are visually recognized only by a part of a plurality of users. Next, an example will be described in which message operation objects are displayed only in a case where no persons other than specific users are present. Specifically, the notification operation objects are displayed in a case where no persons other than certain operation subjects are in a room in which the designated operation subjects are present. For example, when call information is received, the projection controller specifies 103 Users who are rated as destinations of call information. In a case where persons other than the specific user are in the room in which the particular users are present, the projection control unit causes 103 the projection imaging device 10 to wait with the display of the message operation objects until the other persons move out of the room. When other people move out of the room, the projection control unit causes 103 the projection imaging device 10 to project the call information related message service objects to locations adjacent to the particular users. Further, another example of display control for the notification operation objects in which the notification operation objects are visually recognized only by specific users becomes the information processing system 1 according to the seventh modification example of the embodiment with reference to FIG 40 illustrated. 40 Fig. 12 is a diagram showing another example of display control for a message operation object that is visually recognized only by a specific user in the information processing system 1 according to the seventh modification example of the embodiment.

When a message is received from a device to be operated to a user, the projection controller specifies 103 the user as the message target. For example, a user becomes U10 as in 40 illustrates based on a result of the face recognition processing based on face information provided by one of the recognition unit 101 have been obtained, and the face information specified from message-related destination information is specified.

Next, the projection controller waits 103 in a case where other people are in the room in which the particular user is, the other people are moving out of the room. For example, the projection control unit detects 103 the presence of the other person U9 as the specific user U10 from information obtained by the detection unit 101 obtained recognition were obtained. Then turn the projection controller 103 Determine if this other person U9 has moved out of the room in which the particular user is U10 located.

When this other person moves out of the room, the projection control unit causes 103 the projection imaging device 10 to project the message operation object to a location adjacent to the particular user. For example, if this other person U9 moved out of the room in which the particular user U10 located as in 40 illustrated, causes the Proj ektionssteuereinheit 103 the projection imaging device 10 , the message service object related to the call information 52 to project to a location where the message operator object 52 visually of the particular user U10 is recognized.

According to the seventh modification example of the embodiment, the operation objects include the notification operation object related to the notification to the operation subject, and the notification operation object is displayed in accordance with the reception of the notification as described above. Therefore, it is possible to notify the user of the message by automatically displaying the operation objects for the devices to be operated in response to the message in response to the reception. Accordingly, it is possible to reduce the time required for checking or responding to the message.

In addition, the above-mentioned notification to the operator includes a message to a plurality of operators, and the above-mentioned message operator is displayed in a location where the message operator is visually recognized by each of the plurality of operators. Therefore, it is possible to make the notification notice to each of a plurality of users, and it is possible for each of the plurality of users to perform operations of the devices to be operated in response to the notification, respectively. Accordingly, it is possible for another user to handle the message even in a case where a particular user who has noticed the message can not serve the message operation object.

In addition, the above notification to the operator includes a message to a specific operator, and the above-mentioned message operator is displayed in a location where the message operator is visually recognized only by the particular operator. Therefore, in a case where the content of the message has private information of the particular user, it is possible to prevent persons other than the specific user from learning private information. Accordingly, it is possible to balance between privacy protection for the particular user and responsiveness to the notification.

In addition, the above-mentioned notification operation object is displayed in a case where no persons other than the specific operation subject are present in a room in which the above-mentioned specific operation subject is present. Therefore, it is possible to reliably prevent private information of the particular user from being known to other people.

(Eighth Modification Example)

In an eighth modification example of the embodiment, operation objects may be displayed at a location corresponding to a body state of the operation subject instead of the location corresponding to the predetermined movement. In particular, when a body condition of the user of the recognition unit 101 is detected, sets the projection controller 103 from the user's body or the user's environment, a projection location corresponding to the body condition of the user. Further, the modification example will be described with reference to FIG 41 described in detail. 41 Fig. 12 is a diagram showing an example of the display control for operation objects in the information processing system 1 according to the second modification example of the embodiment.

When a predetermined movement, ie, a command operation for the operation object display is detected, the recognition unit recognizes 101 the body shape of the user. For example, in a case where the above-mentioned instruction operation of the user is recognized for the display, the recognition unit recognizes 101 a state of a body part used in the operation by the user (hereinafter also referred to as an operation part). The operating part is, inter alia, a finger, a hand, an arm, a leg and the like.

Next sets the projection controller 103 a location corresponding to the recognized body state of the user as the projection location. In particular, sets the projection control unit 103 the projection location of the operation objects on the basis of the state of the operation part of the user, that of the recognition unit 101 is recognized. For example, in a case where the user holds an object in the left hand as in FIG 41 illustrates, represents the projection controller 103 determine that the left hand is in an unavailable state while the right hand is in a condition available for operation. Then put the projection controller 103 the projection location of the operation objects with reference to the right hand, which is in a state available for operation. In the example in 41 becomes an area that the right hand can reach as a projection region 40 established. Then, the operation objects become the projection region 40 projected.

In addition, the operation objects may be displayed at a location corresponding to a posture of the operation subject. Especially if the attitude of the user of the detection unit 101 is detected, sets the projection controller 103 from the user's body or the user's environment, the projection location according to the user's posture. For example, in a case where the user is standing sideways with the left side of his body positioned as above, a predetermined region will be on the right side of the user, in other words on the side to which the user's face is directed , set as the projection location of the operation objects.

According to the eighth modification example of the embodiment, the operation objects are projected to the location corresponding to the body state of the operation subject as described above. Therefore, it is possible to project the operation objects to a place where the user has the Operating objects according to the body state of the user can operate easily. Accordingly, it is possible to improve the operability in more use cases.

In addition, the operation objects are displayed at the location according to the posture of the operation subject. Therefore, it is possible to display the operation objects in a place where the operation objects are easily recognized by the user. Accordingly, it is possible to improve the visibility of the operation objects and thus improve the operability.

(Ninth Modification Example)

In a ninth modification example, the operation objects in a location may be displayed in accordance with information with which the operation subject is specified. In particular, when a command operation for the operation object display is recognized, the projection control unit sets 103 the display location of the operation objects based on the user specification information. Further, the modification example will be described with reference to FIG 42 described in detail. 42 Fig. 12 is a diagram showing an example of the display control for operation objects in the information processing system 1 according to the ninth modification example of the embodiment. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

When the operation object display instruction operation is detected, the projection control unit detects 103 the user specification information.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation objects to a projection location set based on the user specification information. In particular, the projection control unit detects 103 Projection location information for the operation objects set for the user to the acquired user specification information, and causes the projection imaging device 10 to project the operation objects to the projection location specified by the acquired projection location information. For example, in a case where the right side of the user is set as the projection location for the user as in FIG 42 Fig. 11 illustrates a predetermined region on a wall on the right side of the user as a projection region 40 set for the operating objects as in 42 illustrated. Then, the operation objects become the designated projection region 40 projected.

In addition, the operation objects in a location may be displayed in accordance with information specifying attributes of the operation subject. In particular, when a command operation for the operation object display is recognized, the projection control unit sets 103 the display location of the operation objects based on the user attribute information. More specifically, the projection controller detects 103 Information about the dominant hand from attributes of the user attribute information acquired when the operation object display instruction operation is detected, and causes the projection imaging device 10 to project the operation objects on the side of the dominant hand indicating the information about the dominant hand. In a case where, for example, the dominant hand of the user is as in 42 Illustrates the right hand is a predetermined region on a wall on the right side of the user as a projection region 40 the operating objects are set as in 42 illustrated. Then, the operation objects become the designated projection region 40 projected.

According to the ninth modification example of the embodiment, the operation objects at the location corresponding to the information with which the operation subject is specified are displayed as described above. Therefore, it is possible to display the operation items in a location suitable for the individual users. Accordingly, it is possible to further improve the operability or usability for the individual users.

In addition, the operation objects are displayed at the location corresponding to the information with which attributes of the operation subject are specified. Therefore, it is possible to improve the operability or usability as compared with a case where the display location is uniformly set by displaying the operation objects at the location according to the characteristics of the user. In addition, it is possible to alleviate a concern that the information security deteriorates by controlling the display location of the operation object without information with which the individual users are specified.

(Tenth Modification Example)

In a tenth modification example of the embodiment, an operation object can be operated indirectly. In particular, the operating object can be operated on the basis of an operation performed on a body part of the user. The recognition unit recognizes even more precisely 101 the operation performed on the body part of the user when the operation object is displayed. Then that captures Projection control unit 103 an operation performed on the operation object in accordance with the recognized operation performed on the body part of the user, and updates the display of the operation object according to the detected operation. In addition, the display control unit controls 104 the devices to be operated according to the detected operation. Further, the modification example will be described with reference to FIG 43 described in detail. 43 Fig. 12 is a diagram illustrating an example of operation of an operation object in the information processing system 1 according to the tenth modification example of the embodiment.

The recognition unit 101 Attempts to detect an operation performed on a body part of the user when the operation items are displayed. If, for example, the operating object 49 is projected as in 43 The discovery engine tries to detect 101 , an operation of the operating object 49 detected by the user on the user's own body part. The operation performed on the body part includes an operation of touching the fingers of one hand with fingers of the other hand and the like.

When the operation performed on the body part of the user is detected, the projection control unit controls 103 the display of the operating object according to the detected operation. For example, when an operation of touching the thumb of the left hand with right-hand fingers is detected, the selected content becomes with the operation object 49 changed.

When an operation performed on the user's body part is detected, the projection control unit controls 104 also the device to be operated with respect to the displayed operating object according to the detected operation. For example, when an operation of touching a middle finger of the left hand with right-hand fingers is detected, a movie becomes one with the operation object 49 selected content from the display device.

It should be noted that operating functions may each be associated with each of the fingers. For example, a power on operation may be associated with the thumb, the selection of a higher channel may be associated with an index finger, and the selection of a lower channel may be associated with the middle finger.

According to the tenth modification example of the embodiment, the operation object is operated on the basis of the operation performed on the body part of the user as described above. Therefore, the user can operate the operation object without touching the operation object. Accordingly, the user can operate the operation object without moving or changing the posture, and it is possible to improve the usability. In addition, the user can intuitively operate the operation object. In particular, in a case where the operation is performed with a somatic sensation without visually recognizing the operation target, the line of sight becomes free. In this case, the user can operate the operation object, in other words, the device to be operated while, for example, continuously watching a movie.

Although the example in which the operation performed on the displayed operation object is an operation using a body part of the user has been described above, the device to be operated can be directly operated by operation by the user's body part without displaying the operation object.

Third Embodiment of the Present Disclosure (Relocation of Operating Objects)

Above, the second embodiment of the present invention has been described. Next, a third embodiment of the present disclosure will be described. In the third embodiment of the present disclosure, mainly, a function of the operation object displacement in the information processing system becomes 1 described.

<System configuration>

First, a functional configuration of the information processing system 1 described according to the embodiment. It should be noted that only the information processing device 100 - 3 similar to the second embodiment will be described. Also, the description of functions that are substantially the same as functions in the first or second embodiment will be described among the functions of the information processing apparatus 100 - 3 omitted.

(Detection Unit)

The recognition unit 101 recognizes a given operation performed by an operator on an operation object. In particular, the recognition unit recognizes 101 an operation of gripping the operation object and an operation of releasing the operation object. For example, the recognition unit attempts 101 an operation of grasping an operation object displayed on the hand, and recognize a subsequent operation of opening the gripping hand.

(Projection control unit)

The projection control unit 103 serves as a reference control unit and controls a reference of a place where the operation objects are displayed so that the operation objects are visually recognized (hereinafter also referred to as a reference of the display location) based on the predetermined operation performed on the operation objects. In particular, when there is an object in a real space as a reference of the display location and a predetermined operation performed on the displayed operation objects is detected, the projection control unit changes 103 the object as a reference of the display location. Then, the operation object is displayed on the basis of the changed reference of the display location. In this way, a shift of the operating objects is realized. Further, a shift of the operation objects will be described with reference to FIG 44 described in detail. 44 Fig. 12 is a diagram showing an example of a displacement control of the operation objects in the information processing system 1 according to the third embodiment of the present disclosure.

First, a first operation is detected under predetermined operations for operating objects. If the user in a case where the operating object 42D is projected on a palm of the user as in the upper graph in 44 illustrates, for example, the first operation of grasping a control object 42D performs as in the middle diagram in 44 illustrates, the first operation of the detection unit 101 recognized.

When the first operation is detected, the projection controller causes 103 the projection imaging device 10 to change the shape of the operating object. When the above-mentioned first operation is detected, the projection control unit causes 103 the projection imaging device 10 for example, the projection of the operating object 42D temporarily stop. For example, if the state in which the hand engages, continues as in the middle graph in 44 illustrates the projection of the control object 42D stopped. It should be noted that the display of the control object 42D can be reduced in size, rather than the display of the operating object 42D to stop. In this case, the user can perform a subsequent second operation without the operation object 42D to lose sight of.

Next, a second operation is detected among the predetermined operations for the operation object. For example, if the user performs the second operation of opening the gripping hand as in the lower diagram in FIG 44 illustrates, in other words, the second operation of releasing the grasped operating object 42D towards a table, becomes the second operation of the detection unit 101 recognized.

When the second operation is detected, the projection controller changes 103 the reference of the display location for the operation object selected by the above first operation based on the location corresponding to the second operation. In particular, the projection control unit changes 103 the reference of the display location for the operating object, by the first operation, ie gripping, on an object as the target of the second operation, ie, let go. For example, when the second operation of letting go is detected, the projection control unit changes 103 the reference of the display location of the operating object 42D on which the first operation of the gripping has been performed, from the user's palm to the table, which is a target of the second operation.

When the reference of the display location is changed, the projection control unit causes 103 the projection imager 10 to project the operation object on the basis of the reference of the display location after the change. In particular, the projection control unit controls 103 the shape of the operating object when the reference of the display location is changed. More precisely sets the projection control unit 103 the shape of the operation object after the change based on information for reference of the display location after the change. The display location reference information after the change includes information that specifies the form of the display location reference after the change. For example, if the reference of the display location is changed to the table, the projection controller sets 103 the shape of the operating object 42D according to a region available for projection on the table. Specifically, the projection control unit sets 103 a complexity of the operating object 42D according to the width of the region available for projection. In the example of the lower diagram in 44 becomes an operating object 45 projected with extended display content and advanced operation functions compared to those of the operation object 42D because the region available for projection on the table is wider than the area available for projection on the palm, which is the reference of the display location before the change.

In addition, the information regarding the reference of the display location after the change includes information specifying attributes of the reference of the display location after the change. For example, if the reference of the display location is changed to the table, the projection controller sets 103 the shape of the operating object 42D according to the attributes of the table. Specifically, in a case where the kind of table is a dining table, the display content and the operation functions of the operating object 42D extended.

In addition, the shape of the operation object after the change can be similarly changed on the basis of the display location reference information before the change.

The displacement control for the operating objects has been described above. Furthermore, the projection control unit 103 cause the operating object to follow the reference of the display location. In particular, the project control unit changes 103 the display location of the Bedienungsobj ect according to the displacement of the reference of the display location. The following ability of the operation objects with respect to the reference of the display location will be described with reference to FIG 45 described in detail. 45 Fig. 12 is a diagram showing an example of a followability of the operation objects relative to the reference of the display location in the information processing system 1 illustrated according to the embodiment.

When the above-mentioned second operation is detected, the projection control unit changes 103 the reference of the display location to the object representing the target of the second operation. For example, if the second operation is detected as being performed in a direction to the table as in the upper diagram in FIG 45 illustrates, the projection controller changes 103 the reference of the display location on the table. As a result, the operating object 45 projected onto the table.

Then the projection controller changes 103 the display location of the control object corresponding to the displacement of the object that forms the reference of the display location. For example, if the table is moved as in the lower graph in 45 illustrates, the projection controller changes 103 the projection location of the control object 45 according to the relocation of the table. As a result, it is possible to prompt the user to visually recognize that the operating object 45 is moved together with the relocation of the table as in the lower graph in 45 illustrated, ie as followed by the operating object 45 the relocation of the table.

It should be noted that the object may be a person as a reference of the display location. In particular, the projection control unit changes 103 the display location of the control object according to the displacement of the person forming the reference of the display location. The case where the reference of the display location is a person will be explained with reference to FIG 46 described in detail. 46 Fig. 12 is a diagram illustrating an example of a followability of the operation object in a case where the reference of the display location in the information processing system 1 according to the embodiment is a person.

First, the projection control unit causes 103 the projection imaging device 10 to project the operation object in a location according to attributes or a form of the person forming the reference of the display location. In a case where the dominant hand of the person forming the reference of the display location is, for example, the left hand becomes the operation object 45 projected to the place where the operating object 45 can be easily operated with the dominant hand of the person as in the upper graph in 46 illustrated. In addition, the operating object 45 in a case where the body of the person forming the reference of the display location is turned to the table, projected on the table, which is estimated to fall into the field of vision of the person as in the upper diagram in FIG 46 illustrated.

Then the projection controller changes 103 the display location of the control object according to the displacement of the person forming the reference of the display location. For example, if the person is relocated as in the lower graph in 46 illustrates, the projection controller changes 103 the projection location of the control object 45 according to the displacement of the person. As a result, it is possible to prompt the user to visually recognize that the operating object 45 is shifted together with the displacement of the person as in the lower diagram in 46 illustrated, ie as followed by the operating object 45 the relocation of the person.

<Processing by the system>

Next, the processing by the information processing system 1 described according to the embodiment.

(Overall processing)

First, the overall processing by the information processing system 1 according to the embodiment with reference to 47 described. 47 is a flowchart that provides an example of overall processing the information processing system 1 conceptually illustrated.

The information processing system 1 indicates an operation object based on a reference of a display location (step S501 ). In particular, when an instruction operation for the operation object display is performed, the projection control unit causes 103 the projection imaging device 10 to project the operating object based on an initial reference of the display location.

Next, the information processing system 1 determines whether the operation performed on the operating object has been detected (step S502 ). In particular, the device selection unit provides 103 determines whether the operation performed on the operating object by the recognition unit 101 was detected.

When it is determined that the operation performed on the operation object is detected, the information processing system stops 1 determines whether the detected operation is a given operation (step S503 ). In particular, sets the projection control unit 103 Determine whether the detection unit 101 detected operation is the default operation. It should be noted that the projection control unit 103 In a case where the predetermined operation is a set of a plurality of operations (for example, the above-mentioned first and second operation sets), it determines whether the recognized operation is an operation (for example, the first operation) under predetermined operations is performed first.

When it is determined that the recognized operation is a predetermined operation, the information processing system controls 1 the reference of the display location of the operating object on the basis of the predetermined operation (step S504 ). In particular, the projection control unit changes 103 the reference of the display location of the operation object based on the recognized predetermined operation. It should be noted that details will be described later.

When it is determined that the detected operation is not a predetermined operation, the information processing system controls 1 the device to be operated on the basis of the recognized operation (step S505 ). In particular, when an operation other than the predetermined operation is recognized, the device control unit controls 104 the device to be operated for the operating object on the basis of the detected operation.

It also provides the information processing system 1 determines whether the distance between the user and the operation object is equal to or greater than a threshold for a predetermined time, while no operation performed on the operation object in step S502 is detected (step S506 ). In particular, the projection controller represents 103 Determines whether the period in which the by the recognition unit 101 If the detected distance between the user and the operation object is equal to or greater than the threshold value, it lasts for a time equal to or greater than the predetermined time.

When it is determined that the distance between the user and the operation object is equal to or greater than the threshold for the predetermined time, the information processing system sets 1 determines whether the line of sight has deviated from the operating object for a given time (step S507 ). Specifically, when it is determined that the period in which the distance between the user and the operation object is equal to or greater than the threshold continues over the time equal to or greater than the predetermined time, the projection controller sets 103 and determining whether a period in which the line of sight of the user deviates from the operation object is equal to or greater than the predetermined time.

When it is determined that the line of sight has deviated from the operation object for the predetermined time, the information processing system stops 1 the display of the operating object (step S508 ). Specifically, when it is determined that the period in which the line of sight of the user deviates from the operation object is equal to or greater than the predetermined time, the projection control unit causes 103 the projection imaging device 10 to end the projection of the control object. It should be noted that a message such as an alarm may be provided to the user before the projection of the control object is terminated. The message may be a projection of an alarm-related image, a sound output, or the like.

(Processing of controlling a reference of the display location)

Next, the processing of controlling a reference of a display location in the information processing system becomes 1 with reference to 48 described. 48 Fig. 10 is a flowchart showing an example of the processing of controlling the reference of the display location in the information processing system 1 conceptually illustrated according to the embodiment.

The information processing system 1 sets a reference of a display location of operation objects based on the predetermined operation (step S511 ). In particular, sets the projection control unit 103 a destination of a recognized predetermined operation as a reference of the display location. For example, a destination where the above-mentioned second operation has been performed is set as a reference of the display location.

Next, the information processing system 1 Determines if the reference of the display location has been changed (step S512 ). In particular, the projection controller represents 103 Determines whether the reference of the display location after the decision differs from the reference of the display location before the decision.

When it is determined that the reference of the display location has been changed, the information processing system stops 1 determines whether attribute information or shape information of a destination has been acquired after the change of the reference of the display location (step S513 ). In particular, in a case where the references of the display locations differ from each other before and after the decision, the projection controller sets 103 determines whether the attribute information or the shape information of the destination has been acquired after the change of the reference of the display location. It should be noted that it can be determined whether such information can be captured.

It also provides the information processing system 1 determines whether attribute information or shape information of a change source of the reference of the display location has been detected (step S514 ). In particular, the information processing system provides 103 determines whether attribute information or shape information of the change source of the reference of the display target has been detected. It should be noted that it can be determined whether such information can be captured.

When it is determined that the attribute information or shape information of the destination after the change or the change source of the reference of the display location has been acquired, the information processing system sets 1 determines the shape of the operation objects on the basis of the acquired attribute information or shape information (step S515 ). In particular, sets the projection control unit 103 a complexity of the operation objects based on the acquired attribute information or shape information of the destination after the change or on the basis of the change source of the reference of the display location.

It should be noted that the shape of the operation objects may be set based on both the information of the destination after the change and the information of the source of change of the reference of the display location.

<Brief description of the third embodiment>

According to the third modification example of the present disclosure, the information processing system controls 1 ie the information processing device 100 - 3 , the display of the operation objects for the device to be operated, and controls the reference of the place where the operation objects are displayed so that the operation objects are visually recognized based on the predetermined operation performed by the operation subject of the device to be operated on the operation objects be performed as described above.

Conventionally, a projection location itself of a virtual object is shifted according to a predetermined movement of the user. For example, when the predetermined movement of the user is stopped, the displacement of the virtual object is also stopped, and the projection location of the virtual object is not affected by the displacement or the like of an object existing at the projection site. However, real objects are displaced along with the displacement of the object on which the real object is placed. Since behaviors of the conventional virtual object are different from those of a real object in this way, there is a concern that the operation of the user performed on the virtual object becomes unreliable.

Meanwhile, according to the embodiment, it is possible to control the display location of the operation object not only according to a direct operation of the user on the operation object but also according to an indirect operation, where the reference conditions of the display location are irrelevant to the actions of the user, and the like Reference of the display location of the operating object is controlled. Therefore, the user can arrange or relocate the operation object as if the user were dealing with a real object. Accordingly, it is possible to operate the displacement location of the operation object and to have the feeling of moving a real object.

In addition, the above reference includes objects in a real space, and the location where the above-mentioned operation object is displayed is changed according to a displacement of the objects. Therefore, it is possible to prompt the user for visual recognition as if the operation object has been moved between objects in real space. Furthermore, it is possible for the user to intuitively understand which of the objects one Reference of the display location of the operation object is, in other words, which of the objects is associated with the operation object by the display location of the operation object following the displacement of the objects. Therefore, the user can easily check if the operation object has been moved to an intended object.

In addition, the above-mentioned object includes the above-mentioned operation subject, and the above-mentioned operation object is displayed so that the operation object is visually recognized at a location corresponding to the attributes or the shape of the operation subject. Therefore, the user can handle the operation object as if it belongs to him. Therefore, it is possible to improve the usability. Further, it is possible to display the operation object so that the operation object follows the location suitable for the operation by the user. Accordingly, it is possible to maintain the operability while shifting the operation object.

In addition, the information processing apparatus controls 100 - 3 the shape of the above operation object when the above-mentioned reference is controlled. There is a likelihood that a display environment or display conditions of the operation object will change before and after the change of the reference of the display location, that is, before and after the displacement of the operation object. Thus, it is possible to display the operation object in the form suitable for the reference of the display location after the change by controlling the shape of the operation object when the reference of the display location is controlled.

In addition, the shape of the above-mentioned operation object is controlled on the basis of the above-mentioned information about the reference after the control. Therefore, it is possible to increase a likelihood that the shape after the change is suitable for the displacement target by changing the shape according to the displacement target of the operation object.

In addition, the above reference information after the control includes information specifying attributes or a form of the above reference after the control. Therefore, it is possible to more reliably increase the likelihood that the shape after the change is suitable for the displacement target by displaying the operation object in the form according to characteristics of the displacement target. In addition, it is possible to more reliably increase the likelihood that the shape after the change is suitable for the displacement target irrespective of a change in the conditions of the displacement target by displaying the operation object in the form according to conditions of the displacement target that may change becomes.

In addition, the shape of the above-mentioned operation object includes a complexity of the operation object. Therefore, it is possible to maintain or improve the visibility or operability of the operation object by controlling the complexity of the display or operations according to the displacement of the operation object.

In addition, the above-mentioned predetermined operation includes the set of the first operation and the second operation of the above-mentioned operation object and the information processing device 100 - 3 changes the reference of the operation object selected by the first operation based on a location corresponding to the second operation. Therefore, the user can only relocate the desired operation object by expressly selecting the operation object as the destination of the relocation. Accordingly, it is not necessary for an operation object whose displacement the user does not intend, and it is possible to improve the operability of a displacement operation. In addition, the user can smoothly perform the shifting operation for the operation object by shifting the operation object through a series of first and second operations. Accordingly, it is possible to suppress the complexity of the displacement operation for the operation object.

<Modification Examples>

The third embodiment of the present invention has been described above. It should be noted that the embodiment is not limited to the above-mentioned example. Hereinafter, the first to sixth modification examples of the embodiment will be described.

(First Modification Example)

In a first modification example of the embodiment, the predetermined operation may be another operation. In particular, the predetermined operation involves contact between the destinations of the references of the display location. If from the detection unit 101 For example, it is detected that an object has been brought into contact with another object as a reference of the display location, changes the projection control unit 103 the reference of the display location to this other object. Further, the modification example will be described with reference to FIG 49 described in detail. 49 is a chart that is an example of one predetermined operation in the information processing system 1 according to the first modification example of the embodiment.

The projection control unit 103 causes the projection imaging device 10 to project an operating object onto the object as a reference of the display location. For example, becomes an operating object 42D whose reference of the display location is a user U11 or a hand or palm of the user U11 is on the palm of the user U11 projected as in the upper diagram in 49 described.

If it is detected that the object has been brought into contact with another object as a reference of the display location, the projection control unit changes 103 Next, the reference of the display location to this other object. For example, if it is detected that the user U11 with the hand on which the operating object 42D is projected with another user U12 Hands shake in as in the middle graph 49 illustrates, the projection controller changes 103 this other user U12 or the user's hand or palm U12 as a reference of the display location of the control object 42D ,

Then, the projection controller causes 103 the projection imaging device 10 to project the operation object based on the reference of the display location after the change. As in the lower diagram in 49 illustrates, becomes the operating object 42D on the palm of the other user mentioned above U12 projected that forms the reference of the display location after the change. It should be noted that the operating object 42D not on the above user U11 which forms the reference of the display location before the change. In this way, it is possible to prompt the user for visual recognition, as if the operating object had been displaced by shaking hands with the user.

In the above, the example in which the reference of the display location is changed by the contact between the targets of the references of the display location has been described. However, the default operation may be another operation. In particular, the predetermined operation may be an operation to cause the operation object to rotate. For example, if the recognition unit 101 the operation of rotating the operation object is detected changes the projection control unit 103 the reference of the display location to an object at a rotation target. Further, another mode of the modification example will be described with reference to FIG 50 described in detail. 50 is a diagram that is another example of the default operation in the information processing system 1 according to the first modification example of the embodiment.

The projection control unit 103 causes the projection imaging device 10 to project the operation object based on the reference of the display location. For example, the operating object becomes 45 whose reference of the display location is a user U13 is projected onto a table, leaving the operating object 45 to the user 13 points as in the upper diagram in 50 described.

When an operation of rotating the operation object is detected, the projection control unit changes 103 Next, the reference of the display location to the reference of the display location for a user who is present in a direction related to the rotation. If, for example, by the user U13 performed operation, the operating object 45 to cause it to turn so that the operating object 45 to the direction of a user U14 is recognized, as in the upper graph in 50 illustrates, the projection controller changes 103 the reference of the display location of the operating object 45 by the user U13 on the user U14 ,

Then, the projection controller causes 103 the projection imaging device 10 to project the operation object based on the reference of the display location after the change. For example, the operating object becomes 45 causes it to rotate so that the operating object 45 to the user U14 points as in the lower diagram in 50 illustrated. In this way, it is possible to prompt the user for visual recognition as if an owner of the operation object had been replaced by the user's rotation operation.

Above, the example was described in which the reference of the display location was changed by the operation of rotating the operation object. However, the default operation may be another operation. In particular, the predetermined operation may be an operation of flicking the operating object to the destination after the change of the reference of the display location. For example, when the operation of flicking the operation object from the detection unit 101 is detected changes the projection controller 103 the reference of the display location to the object estimated from the snippet control. Further, another mode of the modification example will be described with reference to FIG 51 to 53 described in detail. 51 to 53 are diagrams, the other examples of the given operation in the information processing system 1 according to the first modification example of the embodiment.

First, a first example of the control of the reference of the display location based on the operation of flicking the operation object will be described with reference to FIG 51 described.

The projection control unit 103 causes the projection imaging device 10 to project the operation object based on the reference of the display location. For example, the operating object becomes 45 whose reference of the display location is a user U15 is projected into an area the user's hand U15 can reach, as in 51 illustrated.

When the operation of flicking the operation object is detected, the projection control unit changes 103 the reference of the display location to an object present in the split direction, based on the strength of the operation. If, for example, the operating object 45 by the user U15 is shifted to an extent that exceeds a predetermined range with respect to the projection location before the operation, as in 51 illustrates, estimates the projection controller 103 the direction in which the operating object 45 was snapped. Then the projection controller changes 103 the reference of the display location to a user U16 who is present in the esteemed direction. It should be noted that in a case where the operating object 45 within the specified range, the reference of the display location is not changed.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation object based on the reference of the display location after the change. For example, the operating object becomes 45 so projected that the operating object is shifted in the Schnipprichtung until the operating object 45 reaches the area the user's hand U16 reached, which forms the reference of the display location after the change. It should be noted that the operating object 45 can be temporarily deleted and then close to the user U16 can be projected instead of being projected so that the operating object 45 is relocated.

Next, a second example of the reference of the display location based on the operation of flicking the operation object will be explained with reference to FIG 52 described.

The projection control unit 103 causes the projection imaging device 10 to project the operation object based on the reference of the display location. For example, the operating object becomes 45 whose reference of the display location is a user U17 is projected into an area the user's hand U17 reached, as in the upper graph and in the lower graph in 52 illustrated.

When the operation of flicking the operation object is detected, the projection control unit changes 103 Next, the reference of the display location to an object existing in the clip direction based on a shape of an operation body performing the operation. For example, if one of a user U17 performed operation of the flipping of a control object 45 with a finger like in the upper diagram in 52 is detected, the projection location of the operating object is shifted according to the operation. In this case, the reference of the display location is not changed. If, however, one from the user U17 performed operation of the flipping of the operating object 45 with five fingers as in the lower diagram in 52 is detected, estimates the projection controller 103 the direction in which the operating object 45 was snapped with the operation. Then the projection controller changes 103 the reference of the display location to a user U18 who is present in the esteemed direction.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation object based on the reference of the display location after the change. Because the details are essentially the same as those in the example in 51 , their description is omitted.

Next, a third example of the control of the reference of the display location based on the operation of flicking the operation object will be described with reference to FIG 53 described.

The projection control unit 103 causes the projection imaging device 10 to project the operation object based on the reference of the display location. For example, the operating object becomes 45 whose reference of the display location is a user U19 is projected into an area the user's hand U19 reached, as in the upper graph and in the lower graph in 53 illustrated.

When the operation of flicking the operation object is detected, the projection control unit changes 103 Next, the reference of the display location to an object existing in the clip direction based on a part of the operation object touched in the operation. For example, when an operation of flicking the operation object 45 through a user U19 is detected in a state in which an information display part of the operation object 45 is touched as in the upper graph in 53 illustrates, the projection location of the control object 45 shifted according to the operation. In this case, the reference of the display location is not changed. If, however, an operation of flicking the operating object 45 by the user U19 is detected in a state in which an upper end of the operating object 45 is touched as in the lower graph in 53 illustrates, estimates the projection controller 103 the direction in which the operating object 45 was snapped with the operation. Then the projection controller changes 103 the reference of the display location to a user U20 who is present in the esteemed direction.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation object based on the reference of the display location after the change. Because the details are essentially the same as those in the example in 51 , their description is omitted.

According to the first modification example of the embodiment, the predetermined operation includes contact between the destinations of the references of the display locations as described above. Therefore, it is possible to prompt the user for visual recognition as if the operation object had been moved between the objects. Accordingly, the user can intuitively relay and receive the operation object.

In addition, the predetermined operation includes the operation of rotating the operation object. Therefore, the user can change an operator of the operation object by changing the orientation of the operation object. Accordingly, the user can easily change the operator of the operation object.

In addition, the predetermined operation includes the operation of flicking the operation object to the destination after changing the reference of the display location. Therefore, the user can change the operator of the operation object to a desired user even in a case where the displacement target of the operation object is not present in a nearby place. Accordingly, it is possible to reduce the load of the user for the given operation.

Second Modification Example

In a second modification example of the embodiment, the reference of the display location may be a position in a real space. In particular, the projection control unit causes 103 the projection imaging device 10 to project the operating object based on the position in the real space that forms the reference of the display location. Further, the projection control unit controls 103 the type of reference of the display location corresponding to a shape of the operation subject when the predetermined operation is performed. The modification example will be described with reference to FIG 54 described in detail. 54 Fig. 12 is a diagram showing an example of a change in the reference of the display location in the information processing system 1 according to the second modification example of the embodiment.

When the specified operation is detected, the detection unit recognizes 101 the form of the operator who performs the predetermined operation. For example, if an operation of opening the left hand toward the table as in the upper graph in 54 illustrates (the above-mentioned second operation) is detected recognizes the detection unit 101 in that the user performing the operation grips the left wrist with the right hand.

The projection control unit 103 selects the type of reference of the display location according to a form of the operator who performs the predetermined operation. In particular, the projection control unit selects 103 the type of reference of the display location from an object, a person, a position and the like according to the shape of the user performing the predetermined operation performed by the recognition unit 101 was detected. For example, when it is detected that the user performing the predetermined operation grips the wrist of the hand with which the operation is performed with the other hand, the projection control unit selects 103 the position in real space as the target of the reference of the display location.

Then, the projection controller causes 103 the projection imaging device 10 to project the operation object based on the reference of the display location after the change. For example, the operating object becomes 45 projected on a table located at the position which is the reference of the display location after the change. Since the reference of the display location is the position at which the table is arranged and not the table, it should be noted that the display location of the operating object 45 is not relocated, even if the table as in the lower graph in 54 illustrated, and the operating object 45 is projected onto the ground.

According to the second modification example of the embodiment, the reference of FIG Display location The position in real space and the display location of the operation object are displayed based on the position in real space as described above. Therefore, even if an object is located at the place where the operation object is projected, it is possible to continue the projection of the operation object to the same place regardless of the displacement of the object. Accordingly, it is possible to suppress the displacement despite the user's intention.

In addition, the information processing apparatus controls 100 - 3 the type of reference of the display location corresponding to the shape of the operator when the predetermined operation is performed. Therefore, the user can determine the kind of the destination after changing the reference of the display location, that is, a destination to which the operation object is associated. Accordingly, the user can handle the operation object as intended by the user, and it is possible to improve the usability.

(Third Modification Example)

In a third modification example of the embodiment, a shape of an operation object may be controlled on the basis of information on an objective unit of operation of the operation object when the reference of the display location is controlled. In particular, the shape of the operation object may be controlled on the basis of information on a state of the operation object (hereinafter also referred to as state information). For example, in a case where a display content of the operation object is different from a display content of an initial state when the reference of the display location of the operation object is changed, the projection control unit decides 103 to reset the display content of the operation object to the display content of the initial state.

In addition, the shape of the operation object may be controlled on the basis of information about a destination to be operated via the operation object (hereinafter also referred to as operation target information). For example, the projection controller sets 103 Display contents of the operation object according to the kind of the content that the device to be operated, which is operated by the operation object reproduces when the reference of the display location of the operation object is changed.

Further, the modification example will be described with reference to FIG 55 described in detail. 55 Fig. 10 is a flowchart showing an example of the processing of controlling the reference of the display location in the information processing system 1 according to the third modification example of the embodiment conceptually illustrated. It is to be noted that the description of the processing which is substantially the same as the above-mentioned processing is omitted.

The information processing system 1 sets a reference of a display location of the operation object based on a predetermined operation (step S521 ) and determines if the display location reference has changed (step S522 ).

When it is determined that the reference of the display location has been changed, the information processing system stops 1 determines whether status information of the operating object has been detected (step S523 and sets a shape of the operation object based on the state information (step S524 ). In particular, when information is detected that specifies the complexity of the operating object, the projection controller sets 103 determines whether the complexity of the display content or the operation functions of the operation object differs from a complexity of the initial state based on the information. If it is determined that the complexity is different from that of the initial state, the projection controller decides 103 to reset the complexity of the operation object to the complexity of the initial state.

It also provides the information processing system 1 determines whether operation target information of the operation object has been detected (step S525 and sets a shape of the operation object based on the operation target information (step S524 ). In particular, when information is detected from which the device to be operated with the operation object or content handled by the device to be operated is specified, the projection controller sets 103 the type of device to use or content specified from the information. Then put the projection controller 103 the shape of the operating object according to the type of device or content to be operated.

According to the third modification example of the embodiment, the shape of the operation object is controlled on the basis of information on the objective unit of operation with respect to the operation object as described above. For example, in a case where the change of the reference of the display location corresponds to a change of the operators and the operation object is directly transferred to the operator after the change, there may be a case where the operator's operation content before the change is known to the operator after the change is. In a case where In particular content for adults is served by the operating object, it is not convenient to relocate the operating object in the same form to a child. Meanwhile, according to the modification example, it is possible to cause the shape of the operation object to change according to the shape of the operation object or the content or the like operated by the operation object when the operation object is displaced. In this way, it is possible to suppress that the operation content becomes known to the operator as a displacement target.

(Fourth Modification Example)

In a fourth modification example of the embodiment, a destination after changing a reference of a display location may be explicitly presented to the user. In particular, the projection control unit controls 103 the display of a display object from which a destination is specified after changing the reference of the display location. For example, the projection controller estimates 103 An object as a candidate for the destination after the change of the reference of the display location and causes the projection imaging device 10 to project the display object from which the estimated object is captured by the user. Further, the modification example will be described with reference to FIG 56 described in detail. 56 Fig. 12 is a diagram illustrating an example in which the object after changing the reference of the display location of the operation object in the information processing system 1 according to the fourth modification example of the embodiment is expressly shown.

The projection control unit 103 causes the projection imaging device 10 to project the operation object based on the reference of the display location. For example, the operating object becomes 45 whose reference of the display location is a user U21 is projected into an area the user's hand U21 can reach, as in 56 illustrated.

When a start of operation of changing the reference of the display location of the operation object is detected, the projection control unit estimates 103 next, an object as a candidate for the destination after changing the reference of the display location. For example, when an operation of touching the operation object 45 is detected with a finger, searches the projection controller 103 for an object that can form a reference of a display location that exists in the environment of the operation object. As a result, the users will U22 and U23 discovered and the users U22 and U23 are estimated as candidates for the goal after changing the reference of the display location.

Next, the projection controller causes 103 the projection imaging device 10 to project the display object expressing the estimated candidates for the destination after the change. For example, the display objects become 54A and 54B each projected as arrows, by the operating object 45 on the users U22 and U23 are judged as candidates for the destination after the change of the reference of the display location, as in 56 illustrated.

Then the projection controller changes 103 the reference of the display location corresponding to an operation of changing the reference of the display location of the operating object. As a result, the operation object is visually recognized by the user as if the operation object has been displaced.

According to the fourth modification example of the embodiment, the information processing apparatus controls 100 - 3 the display of the operation object specifying the destination after the change of the reference of the display location, as described above. Therefore, the user can previously acquire an object to which the operation object can be moved. Accordingly, it is possible to eliminate a concern that the displacement of the operation object to the intended object fails. In addition, it is possible to alleviate a concern that the user fails the operation by displaying the operation direction instruction display object such as an arrow.

(Fifth Modification Example)

In a fifth modification example of the embodiment, a plurality of operation objects may be mixed. Specifically, in a case where another operation object exists to a destination after changing a reference of a display location, the projection control unit mixes 103 the operation object whose reference of the display location is changed with this other operation object as the destination after changing the reference of the display location. For example, an operation object whose reference of the display location is changed (hereinafter also referred to as an operation object as a change source) and an operation object as a destination after the change of the display location (hereinafter also referred to as an operation object as a destination after the change) can be integrated. Further, the modification example will be described with reference to FIG 57 described in detail. 57 FIG. 13 is a diagram illustrating an example of mixing operation items in the information processing system 1 according to the fifth modification example of the embodiment.

The projection control unit 103 causes the projection imaging device 10 to project the operation objects based on the reference of the display location. For example, the operating object becomes 45 whose reference of the display location is a user U24 is projected into an area the user's hand U24 achieved as in 57 illustrated. It also becomes an operating object 49 whose reference of the display location is a user U24 is projected into an area the user's hand U25 reached, as in the upper graph in 57 illustrated.

When an operation of changing the reference of the display location of the operation items is detected, the projection control unit changes 103 Next, the reference of the display location to an object at the destination after changing the change operation. Then, the operation objects are projected to shift the operation objects to the object at the destination after the change. For example, when an operation of shifting the operation object 45 to the user U25 is recognized by the user U24 is performed as in the upper graph in 57 illustrates, the projection controller changes 103 the reference of the display location of the operating object 45 on the user U25 , Then the operating object 45 so projected that the operating object 45 to the user U25 is relocated.

When the operation object reaches the operation object with respect to the destination after changing the reference of the display location, the operation control unit integrates 103 then operating objects. Then an operator object is projected, which is obtained by the integration. If, for example, the operating object 45 to the operating object 49 for the user U25 is displaced causes the projection control unit 103 that part of the operating object 45 and a part of the operating object 49 be integrated with each other. Then, an operation object gained by the integration becomes 55 for the user U25 projected as in the lower diagram in 57 illustrated.

It is to be noted that the mixing of the operation objects may be a merge of an operation object of a change source and an operation object of a destination after the change. For example, an operation object to a recording reservation for a recording apparatus such as a hard disk recording apparatus can be displayed by fusing a control object for a display device and a control object for a timer device.

In addition, an operation object can be divided into a plurality of operation objects. For example, when a divisional operation performed by the user on a displayed operation object is detected, the projection control unit sets 103 a plurality of operating objects with respect to the operating object. Then, the set plurality of operation items is displayed instead of the original operation item.

According to the fifth modification example of the embodiment, the information processing apparatus mixes 100 - 3 in the case where another operation object exists with respect to the target after the change of the reference of the display location, the operation object whose reference of the display location is changed, and this other operation object as a goal after the change of the reference of the display location as described above. Therefore, the user can intuitively structure the operation objects. In addition, the user can intuitively change the displayed operation object into the desired operation object by displaying the operation object by being able to perform an operation on the operation object mixed by mixing the operation objects. Accordingly, it is possible to improve the usability.

(Sixth Modification Example)

In a sixth modification example of the embodiment, the information processing apparatus 100 - 2 further perform attributes of an operation object based on a predetermined operation. In particular, when a predetermined operation of the detection unit 101 is detected controls the projection controller 103 Attributes of the operating object on which the predetermined operation was performed, according to a target of the specified operation. For example, attributes of the operation object include an operation subject that operates the operation object. Further, the modification example will be described with reference to FIG 49 and 58 described in detail. 58 Fig. 12 is a diagram showing an example of the display control for an operation object in the information processing system 1 according to the sixth modification example of the embodiment.

When a predetermined operation performed on a person is detected, the projection control unit changes 103 a reference of a display location and also changes attributes of the control object. As above with reference to 49 described, for example, when the user shakes hands is detected, the reference of the display location is changed to a user who forms the counterpart of the handshake. This also includes an owner of the operating object, which forms one of the attributes of the operating object, also changed to a user who forms the counterpart of the handshake.

When a predetermined operation is performed on an object other than a person, the projection controller changes 103 Next, only the reference of the display location. For example, when a predetermined operation performed by a user forming the above-mentioned counterpart of handshaking is recognized, as in the upper diagram in FIG 58 illustrates, the projection controller changes 103 the reference of the display location on the table. Then the operating object 45 projected onto the table. In this case, the owner of the operating object is not changed. It should be noted that other attributes of the operating object can be changed.

Next, the projection controller controls 103 a form of the operation object based on the attributes of the operation object. In particular, the projection control unit shifts 103 in response to the displacement of the operator operating the operation object, the operation object based on the reference of the display location. As in the lower diagram in 58 illustrates, the operating object projected on the table becomes 45 so projected that the operating object 45 on the table according to the displacement of the user who is the owner of the operating object 45 is, is relocated. Since the reference of the display location is the table, the operating object becomes 45 in this example, not projected outside the table, even if the user moves away from the table. It should be noted that in the case where the reference of the display location is the user, the operation object shifts out of the table according to the displacement of the user.

Although the example in which the display location of the operation object is controlled on the basis of the attributes of the operation object is described above, it should be noted that the complexity of the operation object can be controlled. For example, an operation object having a display content or operation functions may be projected according to an owner of the operation object.

According to the sixth modification example of the embodiment, the information processing apparatus controls 100 - 3 Further, the attributes of the operation object based on the predetermined operation as described above. Therefore, it is possible to finely control the projection of the operation object by controlling not only the reference of the display location but also the attributes of the operation object based on the predetermined operation. Accordingly, it is possible to address a variety of user needs.

In addition, the above-mentioned attributes of the operation object include the operation subject operating the operation object. Therefore, it is possible to finely control the projection of the operation object according to the user of the operation object. Accordingly, it is possible to project the operation object according to the individual users.

In addition, the information processing apparatus controls 100 - 3 the shape of the operating object on the basis of the above-mentioned attributes of the operating object. Therefore, it is possible to properly set the visibility or operability of the operation object by displaying the operation object in the form according to the attributes of the operation object. In a case where, in particular, an attribute of the operation object is a user, it is possible to further improve the usability since the operation object is projected in the form suitable for the individual users.

(Seventh Modification Example)

In a seventh modification example of the embodiment, an operation object can be reproduced. In particular, the projection control unit outputs 103 the operating object based on a predetermined operation again. For example, when the predetermined operation is detected, the projection control unit causes 103 the projection imaging device 10 to project a new operation object obtained by reproducing an operation object to the predetermined operation. The modification example will be described with reference to FIG 59 described in detail. 59 Fig. 12 is a diagram illustrating an example of the display control for an operation object in the information processing system 1 according to the seventh modification example of the embodiment.

The projection control unit 103 causes the projection imaging device 10 to project an operation object for a device to be operated based on a reference of a display location. For example, becomes an operating object 56 for a smartphone 70 as in the left diagram in 59 Illustrated on a table projected a user's hand U26 reached. The operating object 56 may be a list of music files, for example. Furthermore, a different operating object 57A newly projected as in the left diagram in 59 illustrated by the user U26 a selection operation on operation object 56 is carried out. The operating object 57A may be a music file, for example.

When a playback operation performed on the projected operation object is detected, the projection control unit causes 103 the projection imaging device 10 to project the new operation object based on the projected operation object. For example, when an operation of dividing the operation object 57A in two as in the right diagram in 59 is detected, causes the projection controller 103 the projection imaging device 10 , an operating object 57B to project, which is essentially the same as the operating object 57A ,

Further, the reference of the display location of the new operation object obtained by the reproduction can be controlled on the basis of a predetermined operation. For example, when an operation of handing over the operation object 57B reproduced by the user U26 to another user U27 is recognized as in the right graph in 59 illustrates, the projection controller changes 103 the reference of the display location of the operating object 57B on the user U27 ,

It should be noted that the operating object can be moved via communication. For example, in a case where the reference of the display location of the operation object has been changed to an object existing in another building, the projection control unit may 103 cause a projection device in this other building via the communication to project information about the operating object. Further, after the change of the reference of the display location, the destination may be a virtual object. For example, the user may U27 as in the right diagram in 59 illustrated to be present in a building that differs from a building in which the user U26 is present, and it can be a movie according to the user U27 be projected. When an operation of handing over the operating object 57B by the user U26 to the user U27 being projected as a movie, becomes the operating object 57B for the user U27 projected via the communication as an actual object.

Above, the example in which the operation object is played is described. Further, the synchronization of the operation object with respect to the reproduction can be controlled. In particular, the projection control unit controls 103 the synchronization between the reproduced operation object and the operation object as a reproduction source on the basis of the form of the operation subject of the predetermined operation. The control of the synchronization of the operation object with respect to the reproduction will be described with reference to FIG 60 described in detail. 60 Fig. 12 is a diagram showing an example of the display control for an operation object with respect to the reproduction in the information processing system 1 according to a seventh modification example of the embodiment.

The projection control unit 103 causes the projection imaging device 10 to project the operation object based on the reference of the display location. As in the upper diagram and in the lower diagram in 60 illustrates becomes an operating object 45A projected in an area representing the hand of the user U28 can reach.

When a playback operation for the projected operation object is recognized, the detection unit recognizes 101 the form of the playback operation. For example, when an operation of flicking the operation object 45A by a user U28 is detected as in the upper graph and in the lower graph in 60 illustrates the number of fingers of the user U28 detected with which the operation was performed. In the upper diagram in 60 a finger is recognized and in the lower graph in 60 two fingers are detected.

The projection control unit 103 Determines whether the reproduced operation object has been synchronized with the operation object as the playback source based on the form of the reproducing operation. For example, the projection controller represents 103 determines if the synchronization was performed according to the number of fingers in the detected playback operation. Non-synchronization of the control object with respect to the reproduction is shown in the upper graph in FIG 60 in which a finger is detected while synchronizing the operation object with respect to the playback in the lower chart in FIG 60 is determined by recognizing two fingers. The content of the synchronization includes the synchronization of the complexity such as, for example, the display content or the operating functions of the operating object.

Then, the projection controller causes 103 the projection imaging device 10 to project the reproduced operation object and the control of the operation object is accordingly performed if the synchronization has been performed. As in the upper diagram in 60 illustrates becomes an operating object 45B which is essentially the same as the operating object 45A on one user U29 projected as the reference of the display location. Because the operating objects 45A and 45B not be synchronized with each other, become the operating object 45A and 45B independently operated. Meanwhile, becomes an operating object 58 which is substantially the same as a part of the operating object 45A on the user U29 projected as the reference of the display location as in the lower diagram in 60 illustrated. Because the operating objects 45A and 58 For example, the display of the operating object changes as well 58 when the display of the operating object 45A changes. It should be noted that the reproduction of the operation object may be a reproduction of only a part of the operation object as a reproduction source, as in the lower diagram in FIG 60 illustrated.

Although the example has been described above in which it is controlled whether the operation object has been synchronized with respect to the reproduction, the synchronization of other modes can be controlled. For example, a synchronized part of the control object can be controlled for playback. Specifically, a part of the operation object to be synchronized can be controlled as to whether the part forms part or all of the operation object, or the like. In addition, for example, a time can be set in which the synchronization or non-synchronization is active.

In addition, a management user for the operation object can be set on the reproduction. For example, a display object indicating the management user may be displayed along with an operation object for the management user.

According to the seventh modification example of the embodiment, the operation object is reproduced on the basis of the above-mentioned predetermined operation as described above. Therefore, a plurality of persons can control the same device to be operated. Therefore, it is not necessary to relocate the operation object to a user who wants to operate the operation object, and it is possible to improve the usability.

Specifically, the synchronization between the reproduced operation object and the operation object as a reproduction source is controlled on the basis of the form of the operation subject of the above-mentioned predetermined operation. Therefore, the user can select the synchronization of the operation object according to the situation. For example, in a case of an operation object such as a TV program table, it is desirable to share information such as which program is selected by all users, and it is thus considered that the operation object should be synchronized. However, since it is considered that the individual users perform various operations on an operation object such as a controller for a video game, it is considered that the operation object can not be synchronized. In this way it is possible to improve the usability by allowing a situation-appropriate selection of the synchronization.

<Hardware Configuration According to an Embodiment of the Present Disclosure>

Above was the information processing device 100 according to each embodiment of the present disclosure. The above-mentioned processing by the information processing apparatus 100 is through a collaboration between software and hardware of the information processing device 100 realized, which will be described below.

61 FIG. 10 is an explanatory diagram showing a hardware configuration of the information processing apparatus. FIG 100 in accordance with an embodiment of the present disclosure. As in 61 illustrates, includes the information processing device 100 a processor 131 , a store 132 , a bridge 133 , a bus 134 , an interface 135 , an input device 136 , an output device 137 , a storage device 138 , a drive 139 , a connection port 140 and a communication device 141 ,

(Processor)

The processor 131 works as arithmetic processing device and realizes the functions of the recognition unit 101 , the device selection unit 102 , the projection control unit 103 and the device control unit 104 in the information processing apparatus 100 in cooperation with various programs. The processor 131 causes the operation of a variety of logical functions of the information processing apparatus 100 by using a control circuit to execute the programs stored in the memory 132 or another storage medium are stored. For example, the processor 131 a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP) or a system on a chip (SoC) system.

(Storage)

The memory 132 stores programs, arithmetic parameters, or the like, from the processor 131 be used. For example, the memory includes 132 a random access memory (RAM) and temporarily stores programs that are used by the processor 131 be executed, or parameters and the like, which are changed in an appropriate manner. In addition, the memory includes 132 a read-only memory (ROM) and the RAM and the ROM realize the functions of the memory unit. It should be noted that an external storage device is connected via the connection port 140 , the communication device 141 or the like as part of the memory 132 can be used.

It should be noted that the processor 131 and the memory 132 by an internal bus including a CPU bus or the like.

(Bridge and bus)

The bridge 133 establishes a connection between the buses. In particular, the bridge connects 133 the internal bus with which the processor 131 and the memory 132 connected and the bus 134 with which the interface 135 connected is.

(Input device)

The input device 136 is used by the user to the information processing device 100 to operate or information in the information processing device 100 enter. For example, the input device includes 136 an input means for the user inputting information; an input control circuit which generates an input signal based on the input by the user and the input signal to the processor 131 or the like outputs. It should be noted that the input means may be a mouse, a keyboard, a touchpad, a switch, a lever, a microphone, or the like. The user of the information processing apparatus 100 may enter various types of data or instructions for processing operations of the information processing apparatus 100 deploy by holding the input device 136 served.

(Output device)

The output device 137 serves to inform the user and realizes the functions of the input and output unit. The output device 137 may be a display device or a sound output device. For example, the output device 137 a device such as a liquid crystal (LCD) display device, an organic light emitting diode (OLED) device, a projector, a speaker or a headphone, or a module that provides outputs for such device.

It should be noted that the input device 136 or the dispenser 137 may include an input and output device. For example, the input and output device may be a touch screen.

(Storage device)

The storage device 138 is a device for storing data. The storage device 138 For example, a storage medium, a recording device that records data in a recording medium, a reading device that reads out data from a recording device, an eraser that erases data recorded in a recording medium, and the like. The storage device 138 saves programs that the CPU 131 performs, and different types of data.

(Drive)

The drive 139 is a recording medium reader / writer and is in the information processing apparatus 100 contained or connected externally to this. The drive 139 reads information stored on a mounted removable storage medium such as a magnetic disk, an optical disk, a magnetic-optical disk or a semiconductor memory, and supplies the information to the memory 132 out. In addition, the drive can 139 also write information to the removable media.

(Connection terminal)

The connection port 140 is a connection for direct connection of devices with the information processing device 100 , For example, the connection port 140 a Universal Serial Bus (UBS) port, an IEEE1394 port, a Small Computer System Interface (SCSI) port, or the like. Alternatively, the connection port 140 an RS 232C Terminal, an optical audio terminal, a high-definition multimedia interface (HDMI, high-definition multimedia interface) (registered trademark) terminal or the like. Data can be transferred between the information processing device 100 and the external device by connecting the external device to the connection port 140 is connected.

(Communication device)

The communication device 141 mediates the communication between the information processing device 100 and an external device and realizes the functions of the communication unit 105 , In particular, the communication device performs 141 the communication according to a wireless communication concept or a wired communication concept. For example, the communication device performs 141 wireless communication according to a mobile communication concept, such as Wideband Code Division Multiple Access (WCDMA, Broadband Code Division Multiple Access) (registered trademark), Long Term Evolution (LTE, Long-term Evolution), or LTE-A. It should be noted that the communication device 141 wireless communication according to any wireless communication concept, such as a short-range wireless communication concept such as Bluetooth (registered trademark), Near Field Communication (NFC), a wireless USB or a TransferJet (registered trademark) or a wireless local area network (LAN) such as Wi-Fi (registered trademark). In addition, the communication device 141 perform wired communication such as communication via a signal line or a wired LAN communication.

It should be noted that the information processing device 100 where appropriate, no part of the above with reference to 61 described configuration or has any additional configuration. In addition, a one-chip information processing module on which a whole or part of the above with reference to 61 described configuration is provided.

<Conclusion>

As described above, according to the first embodiment of the present disclosure, candidates for the user-to-use device selected by the first device selection based on the body shape of a user are presented to the user, and the user can select a device to be operated from the candidates. Therefore, it is not necessary for the user to move to the device to be operated first. In addition, it is possible to suppress mis-selection of the device to be operated and prevent re-selection of the device to be operated by selecting an operation target from the presented candidates for the device to be operated. Also, it is possible to operate the device to be operated without a special device such as a remote control and to suppress the effort for searching for the remote control or the like by operating the device to be operated based on the selection from the selection objects. Therefore, it is possible to reduce a load of the user for selecting the device to be operated that the user wants to operate.

Moreover, according to the second embodiment of the present disclosure, it is possible to display the operation object in a complexity corresponding to the states of the user estimated from the body shape of the user. Therefore, it is possible to improve the possibilities that operation items that are required for an operation that the user desires are displayed in each state of the user. Accordingly, it is possible to suppress fluctuations in accordance with the user's conditions regarding satisfaction with the operation items.

In addition, according to the third embodiment of the present disclosure, it is possible to have the display location of the operation object not only according to a direct operation of the operation object by the user but also according to an indirect operation, the reference conditions of the display location being irrelevant to the actions of the user, and the like by controlling the reference of the display location of the operating object. Therefore, the user can arrange or relocate the operation object as if the user were dealing with a real object. Accordingly, it is possible to operate the displacement location of the operation object and to have the feeling of moving a real object.

Although the preferred embodiment (s) of the present disclosure have been described with reference to the accompanying drawings, the present disclosure is not limited to the above examples. A person skilled in the art can find various changes and modifications within the scope of the appended claims, and it is to be understood that these, of course, fall within the technical scope of the present disclosure.

For example, although the selection objects and the operation objects are projected in the above embodiments, the present technology is not limited to such an example. In particular, the selection objects and the operation objects can be visually recognized by the user by placing them on an image of the outside world. For example, an image of the selection objects is placed on the image of the outside world by the user having a display device (for example, carries a HUD (Head Up Display)) which transmits light of the image to the outside world and displays an image to the selection objects on a display unit of the display device, or projects an image light to the image from the display device onto the user's eye. It should be noted that the above display device may be a HMD (Head Mount Display) on which the image of the outside world and the image are displayed. In this case, the user can perceive the selection objects without projecting the selection objects in a real space. Therefore, it is possible to configure the information processing system 1 simplify and reduce the cost and burden of introducing the information processing system 1 to reduce. In addition, since the selection objects are not visually recognized by others not wearing the above-mentioned display device, it is possible to prevent the other people's field of view from being disturbed. In addition, the same applies to the operating objects.

Although the example in which devices such as a display device, an air conditioning device, a fan device, a recording device, a lighting device or a sound output device are controlled as devices to be operated has been described in the above embodiments, other devices may be controlled. For example, the devices may be an electric blanket, a microwave, a washing machine, a refrigerator, or a bathroom fixture.

Furthermore, the effects described in this specification are only illustrative or exemplary effects and not limiting. That is, with or instead of the above effects, the technology according to the present disclosure can achieve other effects that will be apparent to those skilled in the art from the description of this specification.

Further, the steps illustrated in the flowcharts of the above embodiment include not only processes that are chronologically performed according to the described order, but also processes that are not necessarily performed chronologically, but also performed in parallel or individually. Further, it is needless to say that even in the steps that are chronologically processed, the order may be appropriately changed according to the circumstances.

In addition, a computer program for causing a computer system to have functions similar to those of the above-described respective functional configurations of the information processing apparatus 100 equivalent are produced. In addition, a storage medium in which the computer program is stored can also be provided. Here, the computer system includes a single computer such as the hardware used in the information processing device 100 or a variety of computers that perform the series of processes.

1. (1) Informationsverarbeitungsvorrichtung, beinhaltend:
   • eine Anzeigesteuereinheit, die die Anzeige von Auswahlobjekten zu zu bedienenden Vorrichtungen steuert, die durch die erste Vorrichtungsauswahl basierend auf Informationen ausgewählt wurden, aus denen eine Form eines Körpers eines Bedienungssubjekts, das auf die zu bedienenden Vorrichtungen gerichtet ist, geschätzt wird; und
   • eine Vorrichtungssteuereinheit, die die zu bedienenden Vorrichtungen steuert, die durch die zweite Vorrichtungsauswahl basierend auf Informationen zu einer Auswahlbedienung, die durch das Bedienungssubjekt an den Auswahlobjekten durchgeführt wurde, ausgewählt wurden.
2. (2) Informationsverarbeitungsvorrichtung nach (1), bei der die Auswahlobjekte Objekte beinhalten, die die zu bedienenden Vorrichtungen angeben, die durch die oben genannte erste Vorrichtungsauswahl ausgewählt wurden.
3. (3) Informationsverarbeitungsvorrichtung nach (1) oder (2), bei der die Auswahlobjekte so angezeigt werden, dass die Auswahlobjekte visuell in einer Form erkannt werden, die auf Prioritätsinformationen basiert.
4. (4) Informationsverarbeitungsvorrichtung nach (3), bei der die Prioritätsinformationen Informationen beinhalten, die in der ersten Vorrichtungsauswahl auf der Basis von Informationen festgelegt werden, aus denen die Form des Körpers geschätzt wird.
5. (5) Informationsverarbeitungsvorrichtung nach (3) oder (4), bei der die Prioritätsinformationen Informationen beinhalten, die auf der Basis biologischer Informationen des Bedienungssubjekts oder von Informationen über eine Umgebung des Bedienungssubjekts festgelegt werden.
6. (6) Informationsverarbeitungsvorrichtung nach einem von (3) bis (5), bei der die Prioritätsinformationen Informationen beinhalten, die auf der Basis von Informationen zu Bedienungen der zu bedienenden Vorrichtungen in der Vergangenheit festgelegt werden.
7. (7) Informationsverarbeitungsvorrichtung nach einen von (1) bis (6), bei der die Anzeigesteuereinheit die Anzeige von Bedienungsobjekten für die zu bedienenden Vorrichtungen steuert, die durch die zweite Vorrichtungsauswahl ausgewählt wurden.
8. (8) Informationsverarbeitungsvorrichtung nach einem von (1) bis (7), bei der die Auswahlobjekte auf einem Körpers des Bedienungssubjekts oder in der der Umgebung des Bedienungssubjekts angezeigt werden, wobei der Körper oder die Umgebung eine Region aufweisen, in der die Auswahlobjekte so angezeigt werden können, dass die Auswahlobjekte vom Bedienungssubjekt visuell erkannt werden.
9. (9) Informationsverarbeitungsvorrichtung nach einem von (1) bis (7), bei der die Auswahlobjekte an einem Ort entsprechend einer vom Bedienungssubjekt durchgeführten Entscheidungsbedienung der ersten Vorrichtungsauswahl angezeigt werden.
10. (10) Informationsverarbeitungsvorrichtung nach (9), bei der der Ort entsprechend der Entscheidungsbedienung einen Teil des Körpers des Bedienungssubjekts oder die Umgebung des Bedienungssubjekts beinhaltet, die durch die Entscheidungsbedienung bestimmt werden.
11. (11) Informationsverarbeitungsvorrichtung nach (9) oder (10), bei der der Ort entsprechend der Entscheidungsbedienung eine Anzeigeeinheit beinhaltet, die durch die Entscheidungsbedienung bestimmt wird.
12. (12) Informationsverarbeitungsvorrichtung nach einem von (1) bis (11), ferner beinhaltend:
    • eine Meldungssteuereinheit, die eine Meldung der zu bedienenden Vorrichtungen steuert, die durch die erste Vorrichtungsauswahl ausgewählt wurden, wenn die oben genannten Auswahlobjekte angezeigt werden.
13. (13) Informationsverarbeitungsvorrichtung nach (12), bei der die Meldung eine Anzeigeausgabe beinhaltet, die eine Verknüpfung zwischen den ausgewählten zu bedienenden Vorrichtungen und den Auswahlobjekten angibt.
14. (14) Informationsverarbeitungsvorrichtung nach (12) oder (13), bei der die Meldung eine Tonausgabe aus einer Region beinhaltet, in der sich die zu bedienenden Vorrichtungen befinden.
15. (15) Informationsverarbeitungsvorrichtung nach einem von (1) bis (14), bei der die Form des Körpers eine Form der visuellen Erkennung des Bedienungssubjekts beinhaltet, und die zu bedienenden Vorrichtungen, von denen festgestellt wird, dass sie mindestens teilweise in das Blickfeld des Bedienungssubjekts fallen, durch die erste Vorrichtungsauswahl ausgewählt werden.
16. (16) Informationsverarbeitungsvorrichtung nach einem von (1) bis (15), bei der die Form des Körpers eine Haltung des Bedienungssubjekts beinhaltet und die zu bedienenden Vorrichtungen, von denen festgestellt wird, dass sie in eine Region fallen, die aus der Haltung des Bedienungssubjekts festlegt wird, durch die erste Vorrichtungsauswahl ausgewählt werden.
17. (17) Informationsverarbeitungsvorrichtung nach einem von (1) bis (16), bei der die Form des Körpers eine Bewegung des Bedienungssubjekts beinhaltet und die zu bedienenden Vorrichtungen, von denen festgestellt wird, dass sie in eine Region fallen, die aus der Bewegung des Bedienungssubjekts festlegt wird, durch die erste Vorrichtungsauswahl ausgewählt werden.
18. (18) Informationsverarbeitungsvorrichtung nach einem von (1) bis (17), bei der die Form des Körpers eine Spracherzeugung des Bedienungssubjekts beinhaltet und die zu bedienenden Vorrichtungen, von denen festgestellt wird, dass sie in eine Region fallen, die aus der Spracherzeugung des Bedienungssubjekts festlegt wird, durch die erste Vorrichtungsauswahl ausgewählt werden.
19. (19) Informationsverarbeitungsverfahren beinhaltend unter Verwendung eines Prozessors:
    • Steuern einer Anzeige von Auswahlobjekten zu zu bedienenden Vorrichtungen, die durch die erste Vorrichtungsauswahl basierend auf Informationen ausgewählt wurden, aus denen eine Form eines Körpers eines Bedienungssubjekts, das auf die zu bedienenden Vorrichtungen gerichtet ist, geschätzt wird; und
    • Steuern der zu bedienenden Vorrichtungen, die durch die zweite Vorrichtungsauswahl basierend auf Informationen zu einer Auswahlbedienung, die durch das Bedienungssubjekt an den Auswahlobjekten durchgeführt wurde, ausgewählt wurden.
20. (20) Programm, das dazu dient, ein Computersystem zur Realisierung des Folgenden zu veranlassen:
    • eine Anzeigesteuerfunktion, die die Anzeige von Auswahlobjekten zu zu bedienenden Vorrichtungen steuert, die durch die erste Vorrichtungsauswahl basierend auf Informationen ausgewählt wurden, aus denen eine Form eines Körpers eines Bedienungssubjekts, das auf die zu bedienenden Vorrichtungen gerichtet ist, geschätzt wird; und
    • eine Vorrichtungssteuerfunktion, die zu bedienende Vorrichtungen steuert, die durch die zweite Vorrichtungsauswahl basierend auf Informationen zu einer Auswahlbedienung, die durch das Bedienungssubjekt an den Auswahlobjekten durchgeführt wurde, ausgewählt wurden.

In addition, the present technology may also be configured as follows.

1. (1) Information processing apparatus including:
   • a display control unit that controls the display of selection objects to-be-operated devices selected by the first device selection based on information from which a shape of a body of an operation subject directed to the devices to be operated is estimated; and
   • a device control unit that controls the devices to be operated selected by the second device selection based on information on a selection operation performed by the operator on the selection objects.
2. (2) Information processing apparatus according to ( 1 ) in which the selection objects include objects indicating the devices to be operated selected by the above-mentioned first device selection.
3. (3) Information processing apparatus according to ( 1 ) or ( 2 ), in which the selection objects are displayed so that the selection objects are visually recognized in a form based on priority information.
4. (4) Information processing apparatus according to ( 3 ), wherein the priority information includes information set in the first device selection based on information from which the shape of the body is estimated.
5. (5) Information processing apparatus according to ( 3 ) or ( 4 ), wherein the priority information includes information determined based on biological information of the operator or information about an environment of the operator.
6. (6) Information processing apparatus according to one of ( 3 ) to ( 5 ), where the priority information includes information based on information about operations of the serving devices in the past.
7. (7) Information processing apparatus according to one of ( 1 ) to ( 6 ) in which the display control unit controls the display of operation items for the devices to be operated selected by the second device selection.
8. (8) Information processing apparatus according to one of ( 1 ) to ( 7 ), wherein the selection objects are displayed on a body of the operation subject or the environment of the operation subject, the body or the environment having a region in which the selection objects can be displayed so that the selection objects are visually recognized by the operation subject.
9. (9) Information processing apparatus according to one of ( 1 ) to ( 7 ) in which the selection objects are displayed at a location corresponding to a decision operation of the first device selection made by the operator.
10. (10) Information processing apparatus according to ( 9 ) in which the location according to the decision operation includes a part of the body of the operation subject or the environment of the operation subject determined by the decision operation.
11. (11) Information processing apparatus according to ( 9 ) or ( 10 ) in which the location according to the decision operation includes a display unit determined by the decision operation.
12. (12) Information processing apparatus according to one of ( 1 ) to ( 11 ), further comprising:
    • a message control unit that controls a message of the devices to be operated selected by the first device selection when the above-mentioned selection objects are displayed.
13. (13) Information processing apparatus according to ( 12 ) in which the message includes a display output indicating a link between the selected devices to be operated and the selection objects.
14. (14) Information processing apparatus according to ( 12 ) or ( 13 ), where the message includes a sound output from a region in which the devices to be operated are located.
15. (15) Information processing apparatus according to one of ( 1 ) to ( 14 ), in which the shape of the body includes a form of visual recognition of the operator, and the devices to be operated, which are determined to at least partially fall within the field of view of the operator, are selected by the first device selection.
16. (16) Information processing apparatus according to one of ( 1 ) to ( 15 ), wherein the shape of the body includes an attitude of the operator, and the devices to be operated, which are determined to fall in a region determined from the attitude of the operator, are selected by the first device selection.
17. (17) Information processing apparatus according to one of ( 1 ) to ( 16 ), in which the shape of the body includes a movement of the operator, and the devices to be operated, which are determined to fall in a region determined from the movement of the operator, are selected by the first device selection.
18. (18) Information processing apparatus according to any one of ( 1 ) to ( 17 ), wherein the shape of the body includes a speech production of the operator, and the devices to be operated which are determined to fall in a region determined from the speech production of the operator are selected by the first device selection.
19. (19) Information processing method using a processor:
    • Controlling a display of selection objects to-be-accessed devices selected by the first device selection based on information that constitutes a shape of a body of an operator subjecting to addressed to serving devices is estimated; and
    • Controlling the devices to be operated selected by the second device selection based on information on a selection operation made by the operator on the selection objects.
20. (20) Program for causing a computer system to do the following:
    • a display control function that controls the display of selection objects to-be-operated devices selected by the first device selection based on information from which a shape of a body of an operation subject directed to the devices to be operated is estimated; and
    • a device control function that controls devices to be operated selected by the second device selection based on information on a selection operation performed by the operator on the selection objects.

• (21) Informationsverarbeitungsvorrichtung, beinhaltend:
  • eine Erfassungseinheit, die Informationen zu einer Körperform eines Bedienungssubjekts gewinnt; und
  • eine Anzeigesteuereinheit, die die Kompliziertheit eines Bedienungsobjekts für eine zu bedienende Vorrichtung, das visuell erkannt wird, als wäre das Bedienungsobjekt in einem realen Raum vorhanden, auf der Basis von Informationen zur Körperform steuert.
• (22) Informationsverarbeitungsvorrichtung nach (21), bei der die Form des Körpers eine Haltung des Bedienungssubjekts beinhaltet und das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell in einer Kompliziertheit entsprechend Informationen zur Haltung des Bedienungssubjekts erkannt wird.
• (23) Informationsverarbeitungsvorrichtung nach (21) oder (22), bei der Informationen zur Körperform biologische Informationen des Bedienungssubjekts beinhalten und das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell in einer Kompliziertheit entsprechend den biologischen Informationen des Bedienungssubjekts erkannt wird.
• (24) Informationsverarbeitungsvorrichtung nach einem von (21) bis (23), bei der die Körperform eine Aktion des Bedienungsobjekts beinhaltet und das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell in einer Kompliziertheit entsprechend Informationen zu der Aktion des Bedienungssubjekts erkannt wird.
• (25) Informationsverarbeitungsvorrichtung nach einem von (21) bis (24), bei der die Anzeigesteuereinheit ferner die Kompliziertheit des Bedienungsobjekts auf der Basis von Informationen zu einem Ort, an dem das Bedienungsobjekt visuell erkannt wird, Informationen, aus denen das Bedienungssubjekt spezifiziert wird, oder Informationen, aus denen Attribute des Bedienungssubjekts spezifiziert werden, steuert.
• (26) Informationsverarbeitungsvorrichtung nach einem von (21) bis (25), bei der das Bedienungsobjekt auf einem Körper des Bedienungssubjekts oder in der Umgebung des Bedienungssubjekts so angezeigt wird, dass das Bedienungsobjekt visuell erkannt wird.
• (27) Informationsverarbeitungsvorrichtung nach einem von (21) bis (26), bei der das Bedienungsobjekt an einem Ort entsprechend einem Grad der Informationssicherheit in Bezug auf eine Bedienung der zu bedienenden Vorrichtung so angezeigt wird, dass das Bedienungsobjekt visuell erkannt wird.
• (28) Informationsverarbeitungsvorrichtung nach einem von (21) bis (27), bei der das Bedienungsobjekt an einem Ort entsprechend Informationen, aus denen das Bedienungssubjekt spezifiziert wird, oder Informationen, aus denen ein Attribut des Bedienungssubjekts spezifiziert wird, angezeigt wird.
• (29) Informationsverarbeitungsvorrichtung nach einem von (21) bis (28), bei der das Bedienungsobjekt an einem Ort entsprechend der Körperform des Bedienungssubjekts angezeigt wird.
• (30) Informationsverarbeitungsvorrichtung nach (29), bei der die Körperform des Bedienungssubjekts eine vorgegebene Bewegung des Bedienungssubjekts beinhaltet und das Bedienungsobjekt an einem Ort entsprechend der vorgegebenen Bewegung angezeigt wird.
• (31) Informationsverarbeitungsvorrichtung nach (30), bei der das Bedienungsobjekt das Bedienungsobjekt beinhaltet, das mit einem Ziel der vorgegebenen Bewegung assoziiert ist.
• (32) Informationsverarbeitungsvorrichtung nach (31), bei der das Bedienungsobjekt, das mit dem Ziel der vorgegebenen Bewegung assoziiert ist, das Bedienungsobjekt für die zu bedienende Vorrichtung beinhaltet, die das Ziel der vorgegebenen Bewegung ist.
• (33) Informationsverarbeitungsvorrichtung nach (31) oder (32), bei der das Bedienungsobjekt, das mit dem Ziel der vorgegebenen Bewegung assoziiert ist, das Bedienungsobjekt für die zu bedienende Vorrichtung beinhaltet, die in einem realen Raum, der derselbe ist wie ein Raum des Ziels der vorgegebenen Bewegung, vorhanden ist.
• (34) Informationsverarbeitungsvorrichtung nach (33), bei dem die zu bedienende Vorrichtung, die im selben realen Raum wie einem Raum des Ziels der vorgegebenen Bewegung auf dem angezeigten Bedienungsobjekt vorhanden ist, auf der Basis von Informationen zu einer Umgebung im selben realen Raum wie dem Raum des Ziels der vorgegebenen Bewegung, Informationen zu einer Form einer Person, die im selben realen Raum wie dem Raum des Ziels der vorgegebenen Bewegung anwesend ist, oder Zeitinformationen ausgewählt wird.
• (35) Informationsverarbeitungsvorrichtung nach einem von (21) bis (34), bei der das Bedienungsobjekt ein Meldungsbedienungsobjekt für eine Meldung an das Bedienungssubjekt beinhaltet und das Meldungsbedienungsobjekt entsprechend dem Empfang der Meldung angezeigt wird.
• (36) Informationsverarbeitungsvorrichtung nach (35), bei der die Meldung an das Bedienungssubjekt eine Meldung an eine Vielzahl der Bedienungssubjekte beinhaltet und das Meldungsbedienungsobjekt an einem Ort angezeigt wird, an dem das Meldungsbedienungsobjekt visuell von jedem der Vielzahl von Bedienungssubjekten erkannt wird.
• (37) Informationsverarbeitungsvorrichtung nach (35) oder (36), bei der die Meldung an das Bedienungssubjekt eine Meldung an ein bestimmtes Bedienungssubjekt beinhaltet und das Meldungsbedienungsobjekt an einem Ort angezeigt wird, an dem das Meldungsbedienungsobjekt visuell nur von dem bestimmten Bedienungssubjekt erkannt wird.
• (38) Informationsverarbeitungsvorrichtung nach (37), bei dem das Meldungsbedienungsobjekt in einem Fall angezeigt wird, in dem keine anderen Personen als das bestimmte Bedienungssubjekt in einem Raum anwesend sind, in dem das bestimmte Bedienungssubjekt anwesend ist.
• (39) Informationsverarbeitungsverfahren beinhaltend unter Verwendung eines Prozessors:
  • Gewinnen von Informationen zu einer Körperform eines Bedienungssubjekts; und
  • Steuern der Kompliziertheit eines Bedienungsobjekts für eine zu bedienende Vorrichtung, das visuell erkannt wird, als wäre das Bedienungsobjekt in einem realen Raum vorhanden, auf der Basis von Informationen zu der Körperform.
• (40) Programm zur Veranlassung eines Computersystems, Folgendes zu verwirklichen:
  • eine Erfassungsfunktion des Erfassens von Informationen zu einer Körperform eines Bedienungssubjekts; und
  • eine Anzeigesteuerfunktion des Steuerns der Kompliziertheit eines Bedienungsobjekts für eine zu bedienende Vorrichtung, das visuell erkannt wird, als wäre das Bedienungsobjekt in einem realen Raum vorhanden, auf der Basis von Informationen zur Körperform.

In addition, the present technology may also be configured as follows.

• (21) Information processing apparatus including:
  • a detection unit that obtains information about a body shape of an operation subject; and
  • a display control unit that controls the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object existed in a real space based on body shape information.
• (22) Information processing apparatus according to ( 21 ), in which the shape of the body includes a posture of the operation subject, and the operation object is displayed so that the operation object is visually recognized in a complexity corresponding to information for posture of the operation subject.
• (23) Information processing apparatus according to ( 21 ) or ( 22 ) in which body shape information includes biological information of the operator and the operation object is displayed so that the operation object is visually recognized in a complexity corresponding to the biological information of the operator.
• (24) Information processing apparatus according to one of ( 21 ) to ( 23 ), wherein the body shape includes an action of the operation object and the operation object is displayed so that the operation object is visually recognized in a complexity corresponding to information of the operation of the operation subject.
• (25) Information processing apparatus according to one of ( 21 ) to ( 24 ), wherein the display control unit further controls the complexity of the operation object based on information about a location where the operation object is visually recognized, information specifying the operation subject, or information specifying attributes of the operation subject.
• (26) Information processing apparatus according to one of ( 21 ) to ( 25 ), in which the operation object is displayed on a body of the operation subject or in the environment of the operation subject so that the operation object is visually recognized.
• (27) Information processing apparatus according to one of ( 21 ) to ( 26 ) in which the operation object is displayed at a location corresponding to a degree of information security with respect to an operation of the device to be operated so that the operation object is visually recognized.
• (28) Information processing apparatus according to one of ( 21 ) to ( 27) wherein the operation object is displayed in a location corresponding to information from which the operation subject is specified, or information specifying an attribute of the operation subject.
• (29) Information processing apparatus according to one of ( 21 ) to ( 28 ) in which the operation object is displayed in a location corresponding to the body shape of the operation subject.
• (30) Information processing apparatus according to ( 29 ) in which the body shape of the operation subject includes a predetermined movement of the operation subject, and the operation object is displayed in a location corresponding to the predetermined movement.
• (31) Information processing apparatus according to ( 30 ), wherein the operation object includes the operation object associated with a target of the given movement.
• (32) Information processing apparatus according to ( 31 ), in which the operation object associated with the target of the given movement includes the operation object for the device to be operated which is the target of the predetermined movement.
• (33) Information processing apparatus according to ( 31 ) or ( 32 ) in which the operation object associated with the target of the predetermined movement includes the operation object for the device to be operated, which exists in a real space which is the same as a space of the target of the given movement.
• (34) Information processing apparatus according to ( 33 ) in which the device to be operated, which exists in the same real space as a space of the target of the predetermined movement on the displayed operation object, based on information about an environment in the same real space as the space of the target of the given movement, information on a form of a person who is present in the same real space as the space of the target of the given movement, or time information is selected.
• (35) Information processing apparatus according to one of ( 21 ) to ( 34 ), wherein the operation object includes a notification operation object for a notification to the operation subject, and the notification operation object is displayed according to the reception of the notification.
• (36) Information processing apparatus according to ( 35 ), wherein the notification to the operator includes a message to a plurality of the operators, and the message operator is displayed in a location where the message operator is visually recognized by each of the plurality of operators.
• (37) Information processing apparatus according to ( 35 ) or ( 36 ), wherein the notification to the operator includes a message to a specific operator and the message operator is displayed in a location where the message operator is visually recognized only by the particular operator.
• (38) Information processing apparatus according to ( 37 ) in which the notification operation object is displayed in a case where no persons other than the specific operation subject are present in a room in which the specific operation subject is present.
• (39) Information processing method using a processor:
  • Obtaining information about a body shape of an operator; and
  • Controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object were present in a real space based on information about the body shape.
• (40) Program for causing a computer system to realize:
  • a detection function of acquiring information about a body shape of an operation subject; and
  • a display control function of controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object were present in a real space based on body shape information.

• (41) Informationsverarbeitungsvorrichtung, beinhaltend:
  • eine Anzeigesteuereinheit, die die Anzeige eines Bedienungsobjekts für eine zu bedienende Vorrichtung steuert, und
  • eine Referenzsteuereinheit, die eine Referenz eines Ortes, an dem das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell erkannt werden kann, auf der Basis einer vorgegebenen Bedienung, die durch ein Bedienungssubjekt der zu bedienenden Vorrichtung am Bedienungsobjekt durchgeführt wird, steuert.
• (42) Informationsverarbeitungsvorrichtung nach (41), bei der die Referenz des Ortes ein Objekt in einem realen Raum beinhaltet und der Ort, an dem das Bedienungsobjekt angezeigt wird, entsprechend der Verlagerung des Objekts geändert wird.
• (43) Informationsverarbeitungsvorrichtung nach (42), bei der das Objekt das Bedienungssubjekt beinhaltet und das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell an einem Ort entsprechend einem Attribut oder einer Form des Bedienungssubjekts erkannt wird.
• (44) Informationsverarbeitungsvorrichtung nach einem von (41) bis (43), bei der die Referenz des Ortes eine Position in einem realen Raum beinhaltet und das Bedienungsobjekt auf der Basis der Position im realen Raum angezeigt wird.
• (45) Informationsverarbeitungsvorrichtung nach einem von (41) bis (44), bei der die Anzeigesteuereinheit ferner eine Art der Referenz des Ortes entsprechend einer Form des Bedienungssubjekts steuert, wenn die vorgegebene Bedienung durchgeführt wird.
• (46) Informationsverarbeitungsvorrichtung nach einem von (41) bis (45), bei der die Anzeigesteuereinheit die Form des Bedienungsobjekts steuert, wenn die Referenz des Ortes gesteuert wird.
• (47) Informationsverarbeitungsvorrichtung nach (46), bei der die Form des Bedienungsobjekts auf der Basis von Informationen zu einem Zweck einer am Bedienungsobjekt durchgeführten Bedienung gesteuert wird.
• (48) Informationsverarbeitungsvorrichtung nach (46) oder (47), bei der die Form des Bedienungsobjekts auf der Basis von Informationen zur Referenz des Ortes vor der Steuerung oder zur Referenz des Ortes nach der Steuerung gesteuert wird.
• (49) Informationsverarbeitungsvorrichtung nach (48), bei der die Informationen zur Referenz des Ortes vor der Steuerung oder zur Referenz des Ortes nach der Steuerung Informationen beinhalten, aus denen ein Attribut oder eine Form der Referenz des Ortes vor der Steuerung oder der Referenz des Ortes nach der Steuerung spezifiziert wird.
• (50) Informationsverarbeitungsvorrichtung nach einem von (46) bis (49), bei der die Form des Bedienungsobjekts die Kompliziertheit des Bedienungsobjekts oder die Folgefähigkeit als Reaktion auf eine Verlagerung der Referenz des Ortes beinhaltet.
• (51) Informationsverarbeitungsvorrichtung nach einem von (41) bis (50), bei der die Anzeigesteuereinheit die Anzeige eines Bedienungsobjekts steuert, aus dem ein Ziel der Änderung der Referenz des Ortes spezifiziert wird.
• (52) Informationsverarbeitungsvorrichtung nach einem von (41) bis (51), bei der in einem Fall, in dem ein anderes Bedienungsobjekt an einem Ziel der Änderung der Referenz des Ortes vorhanden ist, die Anzeigesteuereinheit das Bedienungsobjekt, dessen Referenz des Ortes geändert werden soll, und das andere Bedienungsobjekt am Ziel der Änderung der Referenz des Ortes mischt.
• (53) Informationsverarbeitungsvorrichtung nach einem von (41) bis (52), bei der die Referenzsteuereinheit ferner ein Attribut des Bedienungsobjekts auf der Basis der vorgegebenen Bedienung steuert.
• (54) Informationsverarbeitungsvorrichtung nach (53), bei dem das Attribut des Bedienungsobjekts das Bedienungssubjekt zur Bedienung des Bedienungsobjekts beinhaltet.
• (55) Informationsverarbeitungsvorrichtung nach (53) oder (54), bei der die Anzeigesteuereinheit eine Form des Bedienungsobjekts auf der Basis des Attributs des Bedienungsobjekts steuert.
• (56) Informationsverarbeitungsvorrichtung nach einem von (41) bis (55), bei der das Bedienungsobjekt auf der Basis der vorgegebenen Bedienung kopiert wird.
• (57) Informationsverarbeitungsvorrichtung nach (56), bei der eine Synchronisierung zwischen einer Kopie des Bedienungsobjekts und dem Bedienungsobjekt, das eine Kopierquelle darstellt, auf der Basis einer Form des Bedienungssubjekts bei der vorgegebenen Bedienung gesteuert wird.
• (58) Informationsverarbeitungsvorrichtung nach einem von (41) bis (57), bei der die vorgegebene Bedienung einen am Bedienungsobjekt durchgeführten Satz aus einer ersten Bedienung und einer zweiten Bedienung beinhaltet und die Referenzsteuereinheit die Referenz des Bedienungsobjekts, das durch die erste Bedienung ausgewählt wurde, auf der Basis eines Ortes entsprechend der zweiten Bedienung ändert.
• (59) Informationsverarbeitungsverfahren beinhaltend unter Verwendung eines Prozessors:
  • Steuern der Anzeige eines Bedienungsobjekts für eine zu bedienende Vorrichtung und
  • Steuern einer Referenz eines Ortes, an dem das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell erkannt werden kann, auf der Basis einer vorgegebenen Bedienung, die durch ein Bedienungssubjekt der zu bedienenden Vorrichtung am Bedienungsobjekt durchgeführt wird.
• (60) Programm zur Veranlassung eines Computersystems, Folgendes zu verwirklichen:
  • eine Anzeigesteuerfunktion des Steuerns der Anzeige eines Bedienungsobjekts für eine zu bedienende Vorrichtung; und
  • eine Referenzsteuerfunktion des Steuerns einer Referenz eines Ortes, an dem das Bedienungsobjekt so angezeigt wird, dass das Bedienungsobjekt visuell erkannt werden kann, auf der Basis einer vorgegebenen Bedienung, die durch ein Bedienungssubjekt der zu bedienenden Vorrichtung am Bedienungsobjekt durchgeführt wird.

In addition, the present technology may also be configured as follows.

• (41) An information processing apparatus including:
  • a display control unit that controls the display of an operation object for a device to be operated, and
  • a reference control unit that controls a reference of a location where the operation object is displayed so that the operation object can be visually recognized based on a predetermined operation performed by an operation subject of the device to be operated on the operation object.
• (42) Information processing apparatus according to ( 41 ), where the reference of the location includes an object in a real space and the location where the operation object is displayed is changed according to the displacement of the object.
• (43) Information processing apparatus according to ( 42 ) in which the object includes the operation subject and the operation object is displayed so that the operation object is visually recognized in a location corresponding to an attribute or a shape of the operation subject.
• (44) Information processing apparatus according to one of ( 41 ) to ( 43 ), where the reference of the location includes a position in a real space and the operation object is displayed based on the position in the real space.
• (45) Information processing apparatus according to one of ( 41 ) to ( 44 ), wherein the display control unit further controls a kind of reference of the location according to a shape of the operation subject when the predetermined operation is performed.
• (46) Information processing apparatus according to one of ( 41 ) to ( 45 ) in which the display control unit controls the shape of the operation object when the reference of the location is controlled.
• (47) Information processing apparatus according to ( 46 ) in which the shape of the operation object is controlled on the basis of information for a purpose of operation performed on the operation object.
• (48) Information processing apparatus according to ( 46 ) or ( 47 ), in which the shape of the operating object is controlled on the basis of information for reference of the place before the control or for reference of the place after the control.
• (49) Information processing apparatus according to ( 48 ), where the pre-control location reference information or the post-control location reference information includes information specifying an attribute or a form of the reference of the location before the control or the reference of the location after the control.
• (50) Information processing apparatus according to one of ( 46 ) to ( 49 ), in which the shape of the operation object includes the complexity of the operation object or the following ability in response to a displacement of the reference of the location.
• (51) Information processing apparatus according to one of ( 41 ) to ( 50 ) in which the display control unit controls display of an operation object from which a destination of change of the reference of the location is specified.
• (52) Information processing apparatus according to one of ( 41 ) to ( 51 ) in which, in a case where another operation object is present at a destination of change of the reference of the location, the display control unit changes the operation object whose reference of the location is to be changed, and the other operation object aims to change the reference of the location mixed.
• (53) Information processing apparatus according to one of ( 41 ) to ( 52 ), wherein the reference control unit further controls an attribute of the operation object on the basis of the predetermined operation.
• (54) Information processing apparatus according to ( 53 ), wherein the attribute of the operation object includes the operation subject for operating the operation object.
• (55) Information processing apparatus according to ( 53 ) or ( 54 ), wherein the display control unit controls a shape of the operation object based on the attribute of the operation object.
• (56) Information processing apparatus according to one of ( 41 ) to ( 55 ) in which the operation object is copied based on the predetermined operation.
• (57) Information processing apparatus according to ( 56 ) in which synchronization between a copy of the operation object and the operation object, which is a copy source, is controlled on the basis of a shape of the operation subject in the predetermined operation.
• (58) Information processing apparatus according to one of ( 41 ) to ( 57 ), wherein the predetermined operation includes a set of first operation and a second operation performed on the operation object, and the reference control unit changes the reference of the operation object selected by the first operation on the basis of a location corresponding to the second operation.
• (59) Information processing method using a processor:
  • Controlling the display of an operation object for a device to be operated, and
  • Controlling a reference of a place where the operation object is displayed so that the operation object can be visually recognized based on a predetermined operation performed by an operation subject of the device to be operated on the operation object.
• (60) Program for creating a computer system to realize:
  • a display control function of controlling the display of an operation object for a device to be operated; and
  • a reference control function of controlling a reference of a location where the operation object is displayed so that the operation object can be visually recognized based on a predetermined operation performed by an operation subject of the device to be operated on the operation object.

LIST OF REFERENCE NUMBERS

10

Projection imaging device

20

display device

21

Air conditioning device

22

fan device

100

Information processing apparatus

101

recognizer

102

Device selection unit

103

Projection control unit

104

Device control unit

105

communication unit

106

storage unit

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• JP 2011188023 A [0004]

Claims (20)

An information processing apparatus, comprising: a detection unit that obtains information about a body shape of an operation subject; and a display control unit that controls the complexity of an operation object for a device to be operated that is visually recognized as if the operation object exists in a real space based on body shape information. Information processing device according to Claim 1 wherein the body shape includes a posture of the operation object, and the operation object is displayed so that the operation object is visually recognized in a complexity corresponding to information for posture of the operation subject. Information processing device according to Claim 1 wherein the body shape information includes biological information of the operator and the operator object is displayed so that the operator object is visually recognized in a complexity corresponding to the biological information of the operator. Information processing device according to Claim 1 wherein the body shape includes an action of the operation object, and the operation object is displayed so that the operation object is visually recognized in a complexity corresponding to information of the operation of the operation subject. Information processing device according to Claim 1 wherein the display control unit further controls the complexity of the operation object based on information about a location where the operation object is visually recognized, information specifying the operation subject, or information specifying attributes of the operation subject. Information processing device according to Claim 1 wherein the operation object is displayed on a body of the operation subject or in the environment of the operation subject so that the operation object is visually recognized. Information processing device according to Claim 1 wherein the operation object is displayed at a location corresponding to a degree of information security with respect to an operation of the device to be operated so that the operation object is visually recognized. Information processing device according to Claim 1 wherein the operation object is displayed in a location corresponding to information from which the operation subject is specified or information specifying an attribute of the operation subject. Information processing device according to Claim 1 wherein the operation object is displayed at a location corresponding to the body shape of the operation subject. Information processing device according to Claim 9 wherein the body shape of the operation subject includes a predetermined movement of the operation subject and the operation object is displayed in a location corresponding to the predetermined movement. Information processing device according to Claim 10 wherein the operation object includes the operation object associated with a target of the predetermined movement. Information processing device according to Claim 11 wherein the operation object associated with the target of the predetermined movement includes the operation object for the device to be operated which is the target of the predetermined movement. Information processing device according to Claim 11 wherein the operation object associated with the target of the predetermined movement includes the operation object for the device to be operated existing in a real space which is the same as a space of the target of the predetermined movement. Information processing device according to Claim 13 wherein the device to be operated, which exists in the same real space as a space of the target of the predetermined movement on the displayed operation object, on a basis of information on an environment in the same real space as the space of the destination of the given movement, information about a form of a person who is present in the same real space as the space of the target of the given movement, or time information is selected. Information processing device according to Claim 1 wherein the operator object includes a message operator object for a message to the operator, and the message service object is displayed according to the receipt of the message. Information processing device according to Claim 15 wherein the message to the operator includes a message to a plurality of the operator and the message operator is displayed in a location where the message operator is visually recognized by each of the plurality of operators. Information processing device according to Claim 15 wherein the notification to the operator includes a message to a particular operator and the message operator is displayed in a location where the message operator is visually recognized only by the particular operator. Information processing device according to Claim 17 wherein the notification operation object is displayed in a case where no persons other than the specific operation subject are present in a room in which the specific operation subject is present. An information processing method comprising using a processor: Obtaining information about a body shape of an operator; and Controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object were present in a real space on the basis of information about the body shape. Program for causing a computer system to accomplish the following: a detection function of acquiring information about a body shape of an operation subject; and a display control function of controlling the complexity of an operation object for a device to be operated, which is visually recognized as if the operation object exists in a real space on the basis of information about the body shape.

Images