JP6823391B2 - Programs, computer-readable media, terminals, estimators and estimation methods - Google Patents

Description

The present invention relates to programs, computer-readable media, terminal devices, estimation devices and estimation methods.

Conventionally, in a device such as a personal computer, whether or not a detection target such as a human body exists in the space within the field of view of the sensor is estimated by using the output from the infrared sensor when the operation is performed (for example,). See Patent Document 1).
[Patent Document 1] Japanese Unexamined Patent Publication No. 2012-078959

However, in the conventional detection method, there is a possibility that the detection target is estimated to exist even though it does not exist in the space in the visual field, and improvement in estimation accuracy is desired.

In the first aspect of the present invention, the computer is provided in the terminal device and an estimation unit that estimates the spatial state using the first output value received from the first sensor, which is an infrared sensor provided in the terminal device. The second output value received from the second sensor is used to detect at least one of the movement and the posture of the terminal device, and according to at least one of the movement and the posture of the terminal device detected by the detection unit. Provided are a program that functions as a control unit that controls estimation of a spatial state by an estimation unit, a computer-readable medium provided with the program, and an estimation device.

In the second aspect of the present invention, an estimation unit that estimates a spatial state using a first output value received from a first sensor, which is an infrared sensor provided in the terminal device, and a second unit provided in the terminal device. 2 A detection unit that detects at least one of the movement and orientation of the terminal device using the second output value received from the sensor, and an estimation unit according to at least one of the movement and orientation of the terminal device detected by the detection unit. Provided is an estimation device including a control unit that controls estimation of a spatial state by the above, and a terminal device including the estimation device.

In the third aspect of the present invention, an estimation step for estimating a spatial state using a first output value received from a first sensor, which is an infrared sensor provided in the terminal device, and a third aspect provided in the terminal device. 2 The estimation unit according to the detection step of detecting at least one of the movement and the posture of the terminal device using the second output value received from the sensor and at least one of the movement and the posture of the terminal device detected by the detection unit. Provided is an estimation method including a control step for controlling the estimation of the spatial state by.

The above outline of the invention does not list all the features of the present invention. Sub-combinations of these feature groups can also be inventions.

It is an external view which shows the terminal apparatus which concerns on this embodiment. It is an external view which shows the terminal apparatus which concerns on this embodiment. It is an external view which shows the terminal apparatus which concerns on this embodiment. It is an external view which shows the terminal apparatus which concerns on this embodiment. It is a block diagram which shows the terminal apparatus which concerns on this embodiment. It is a figure for demonstrating the movement which can be detected by the terminal apparatus which concerns on this embodiment. It is a flowchart which shows the operation of the terminal apparatus which concerns on this embodiment. It is a flowchart which shows the operation of the initial posture correspondence control. It is a flowchart which shows the operation of the posture correspondence control. It is a flowchart which shows the operation of the estimation process by the 1st estimation algorithm. It is a flowchart which shows the operation of the estimation process by the 3rd estimation algorithm. It is a flowchart which shows the operation of the estimation process by the 2nd estimation algorithm. It is a flowchart which shows the operation of the automatic login process. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the orientation of the display surface corresponding to each estimation algorithm. It is a figure for demonstrating the setting of the threshold value by the orientation of a display surface. It is a figure for demonstrating the setting of the threshold value by the orientation of a display surface. It is a figure for demonstrating the setting of the threshold value according to the angle of the 1st housing and the 2nd housing. It is a figure for demonstrating the setting of the threshold value according to the angle of the 1st housing and the 2nd housing. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure for demonstrating the setting of the threshold value by the posture of the mounted terminal apparatus. It is a figure which shows the operation example of a terminal device. It is a block diagram which shows the computer which concerns on this embodiment.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the inventions claimed in the claims. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.

[1. Appearance of terminal device]
1A to 1D are external views showing a terminal device 1 according to the present embodiment. The terminal device 1 includes a first housing 11 having a display surface 111. The terminal device 1 may further include a second housing 12 having a keyboard 120. For example, the terminal device 1 may be a notebook computer.

The display surface 111 is a display screen on the display 110 (for example, a liquid crystal display). The display surface 111 may function as a so-called touch panel by being laminated with the touch sensor 112 that detects the contact of the operator's finger or the like.

The first housing 11 and the second housing 12 may each have a rectangular plate shape. The first housing 11 and the second housing 12 may have the same size.

The second housing 12 is provided with a variable angle with respect to the first housing 11. As a result, the second housing 12 may be opened and closed with the display surface 111 and the keyboard 120 inside in cooperation with the first housing 11. In addition to / instead of this, the second housing 12 may be opened and closed with the display surface 111 and the keyboard 120 on the outside in cooperation with the first housing 11.

The second housing 12 may be detachable from the first housing 11. When the second housing 12 is detached from the first housing 11, it can be attached to the first housing 11 in a direction different from that before attachment / detachment and / or at a position different from that before attachment / detachment. May be good. As a result, the second housing 12 may be opened and closed with at least one of the display surface 111 and the keyboard 120 on the outside in cooperation with the first housing 11.

For example, the terminal device 1 may be used in any of the laptop type shown in FIG. 1A, the tent type shown in FIG. 1B, the tablet type shown in FIG. 1C, and the stand type shown in FIG. 1D. In the terminal device 1 shown in FIGS. 1B to 1D, the keyboard 120 of the second housing 12 may face the side of the first housing 11 or may face the opposite side. Further, in the tablet-type terminal device 1 of FIG. 1C, the second housing 12 is mounted on the back side of the first housing 11, but it may be detached.

[2. Functional configuration of terminal device]
FIG. 2 is a block diagram showing a terminal device according to the present embodiment.

The terminal device 1 performs an operation of transitioning a state depending on whether or not an operator is present in the vicinity. For example, the terminal device 1 may automatically switch between a login state and a log-off state. In addition to / instead of this, the terminal device 1 may automatically switch between the normal power state and the power saving state depending on whether or not an operator is present in the vicinity. The terminal device 1 includes an infrared sensor 21, one or more acceleration sensors 22, and an estimation device 10. The terminal device 1 may further include one or more geomagnetic sensors 23, a positional relationship sensor 24, a user interface 30, an imaging unit 40, an imaging control unit 50, and an authentication unit 60.

[2-1. Infrared sensor]
The infrared sensor 21 is an example of the first sensor. The infrared sensor 21 is a sensor that outputs a signal based on the infrared rays received from the field of view. For example, the infrared sensor 21 may be a thermal infrared sensor that utilizes a temperature change generated by absorbing infrared energy, or a change in conductivity or electromotive force generated by electrons excited by incident light energy. It may be a quantum type infrared sensor that utilizes.

Examples of the thermal infrared sensor include a pyroelectric element using the pyroelectric effect, a thermocouple and a thermopile using the thermoelectric effect, and a bolometer using the effect of changing the electric resistance with temperature. Examples of the quantum infrared sensor include a phototube using the external photoelectric effect, a photoconducting sensor using the internal photoelectric effect, and a photovoltaic sensor. Examples of the photoconducting type sensor and the photovoltaic type sensor include a photodiode using a material containing mercury cadmium telluride (HgCdTe, etc.), a material containing indium and antimony, and an arsenide (InSb, InAsSb, InAs, etc.) as an element material. Phototransistors can be mentioned.

A quantum infrared sensor is preferable between the thermal infrared sensor and the quantum infrared sensor. This is because the quantum infrared sensor has an excellent response speed. Further, the infrared sensor 21 is preferably operable without cooling, and is preferably small and thin so that it can be mounted on various thin devices. As an example, the infrared sensor 21 may be an infrared sensor "IR1011" (trade name) manufactured by Asahi Kasei Electronics Co., Ltd., or "AK9720", "AK9750", "AK9752" in which an IC for signal processing is added to the sensor. "(Product name) may be used.

The infrared sensor 21 may be provided in the first housing 11. For example, the infrared sensor 21 is included in the field of view (for example, an observable range) when the operator is operating the terminal device 1 or when the operator is looking at the display surface 111. May be installed in. For example, the infrared sensor 21 may be installed so that an exposed portion (for example, a face portion) of the body surface is included in the visual field in order to receive infrared rays from the body surface of the operator. As an example, the infrared sensor 21 may be provided at the upper end of the first housing 11 (as an example, the farthest end from the second housing 12 when the terminal device 1 is used in a laptop type). It may be provided on the bezel portion of the display 110 at the upper end portion, preferably. Further, the infrared sensor 21 may be directed so that the space in the vicinity of the keyboard 120 is in the field of view when the terminal device 1 is used in the laptop type.

The infrared sensor 21 may supply an infrared output value based on the received infrared rays to the estimation device 10. The infrared output value is an example of the first output value. For example, the infrared sensor 21 may output an infrared output value to the estimation unit 302 described later in the estimation device 10. The infrared sensor 21 may output the signal value itself generated by the sensor portion as an infrared output value, or the infrared output is a value obtained by subjecting the signal value to signal processing such as A / D conversion, amplification, and buffer. It may be output as a value.

[2-2. Acceleration sensor]
The one or more acceleration sensors 22 is an example of the second sensor, and measures the acceleration as a physical quantity for detecting at least one of the movement and the posture of the terminal device 1.

Here, the movement of the terminal device 1 may be, for example, the movement of the terminal device 1 by a person or a means of transportation (for example, a car or a train), or may be a change in posture. The posture of the terminal device 1 may be at least one of the orientation and the outer shape of the terminal device 1, for example, the orientation of the first housing 11 and the second housing 12, and the orientation of the first housing 11 and the second housing. It may be a relative positional relationship of twelve. The posture of the terminal device 1 detected in the present embodiment will be described in detail later with reference to FIG.

For example, the acceleration sensor 22 may be an acceleration sensor that detects acceleration in at least one direction, and may preferably be a three-dimensional acceleration sensor that detects acceleration in three directions. The acceleration sensor 22 may be provided in the first housing 11. The acceleration sensor 22 may supply an acceleration output value based on the measured acceleration to the detection unit 301. The acceleration output value is an example of the second output value.

[2-3. Geomagnetic sensor]
The one or more geomagnetic sensors 23 are sensors that measure the geomagnetism, and are so-called electronic compasses. The geomagnetic sensor 23 may be provided in the first housing 11 or in the second housing 12. The geomagnetic sensor 23 may supply the measured geomagnetic output value based on the geomagnetism to the detection unit 301.

[2-4. Positional sensor]
The positional relationship sensor 24 measures a physical quantity for detecting the positional relationship between the first housing 11 and the second housing 12. For example, the positional relationship sensor 24 may be an angle sensor that detects the angle formed by the first housing 11 and the second housing 12. Further, the positional relationship sensor 24 may be an acceleration sensor provided in the second housing 12. By using the acceleration sensor of the second housing 12 together with the acceleration sensor 22 of the first housing 11, the angle formed by the first housing and the second housing 12 can be detected. The positional relationship sensor 24 may supply a signal value based on the detected physical quantity to the detection unit 301.

[2-5. Estimator]
The estimation device 10 estimates the spatial state. The estimation device 10 includes a detection unit 301, an estimation unit 302, and a control unit 303. These configurations may be configured using analog circuits, digital circuits, and / or combinations of processors or microcontrollers that operate programmatically.

[2-5-1. Detection unit]
The detection unit 301 detects at least one of the movement and the posture of the terminal device 1 by using the acceleration output value received from the acceleration sensor 22. The detection unit 301 may further use the output value received from the geomagnetic sensor 23 and / or the positional relationship sensor 24 to detect at least one of the movement and the posture of the terminal device 1.

[2-5-2. Estimator]
The estimation unit 302 estimates the spatial state using the infrared output value received from the infrared sensor 21. For example, the estimation unit 302 may estimate as a spatial state whether or not a detection target such as a person exists in the space within the field of view of the infrared sensor 21. The estimation unit 302 may output the estimation result to the user interface 30 or the like outside the estimation device 10. The operation of the estimation unit 302 will be described in detail later.

[2-5-3. Control unit]
The control unit 303 controls the estimation of the spatial state by the estimation unit 302 according to at least one of the movement and the posture of the terminal device 1 detected by the detection unit 301. For example, the control unit 303 may disable the estimation of the spatial state by the estimation unit 302 or the output of the estimation result in response to the detection unit 301 detecting that the terminal device 1 is moving. In addition to / instead, the control unit 303 may change at least one of the estimation algorithm and the estimation parameter by the estimation unit 302 according to the orientation of the display surface 111 detected by the detection unit 301. The estimated parameter may be a threshold used for comparison with the infrared output value, but may be a coefficient multiplied by the infrared output value prior to comparison with the threshold. The operation of the control unit 303 will be described in detail later.

[2-6. User interface]
The user interface 30 exchanges information between the terminal device 1 and the operator. For example, the user interface 30 has a display 110, a touch sensor 112, a keyboard 120, and the like, and information may be exchanged between the operating system of the terminal device 1 and the operator. Further, the user interface 30 may perform a login process (and / or a login screen display process) and a logout process of the terminal device 1 according to the user's operation. Further, the user interface 30 may automatically perform login processing and logout processing according to the estimation result by the estimation device 10.

[2-7. Imaging unit]
The imaging unit 40 images the space in the field of view of the infrared sensor 21. For example, the imaging unit 40 may be able to image a space including the face portion of the operator when performing face recognition of the operator who operates the terminal device 1. The field of view of the imaging unit 40 may overlap with the field of view of the infrared sensor 21 and may be the same. As an example, the imaging unit 40 may be a CCD camera. The image pickup unit 40 may supply the data of the captured image to the authentication unit 60 or the like.

[2-8. Imaging control unit]
The image pickup control unit 50 activates the image pickup unit 40. For example, the imaging control unit 50 may activate the imaging unit 40 when the estimation unit 302 estimates the existence of a person as a detection target.

[2-9. Authentication Department]
When the image pickup unit 40 is activated by the image pickup control unit 50, the authentication unit 60 uses the image pickup unit 40 to authenticate the face of the operator. For example, the authentication unit 60 may detect the face portion in the captured image supplied from the imaging unit 40. Further, the authentication unit 60 may perform face authentication by collating the detected face with the face image of one or a plurality of operators registered in the terminal device 1. When the face authentication is successful, the authentication unit 60 may perform the login process by outputting a signal notifying this to the user interface 30. As a result, when a person comes within the field of view of the infrared sensor 21, face recognition and login processing are automatically performed.

According to the above terminal device 1, the estimation of the spatial state using the infrared output value is controlled according to at least one of the movement and the posture of the terminal device 1, so that the movement and the posture of the terminal device 1 are known. The estimation of can be different from the control without it. Therefore, the accuracy of estimation can be improved.

Further, since the estimation of the spatial state or the output of the estimation result is disabled according to the movement of the terminal device 1, the existence of a person is known in the situation where the existence of the person using the terminal device 1 is known. Can be prevented from being estimated. Therefore, it is possible to reduce computational resources and prevent erroneous estimation that a person does not exist.

Further, since at least one of the estimation algorithm and the estimation parameter is changed according to the orientation of the display surface 111, the estimation accuracy is compared with the case where the estimation is performed with the same estimation algorithm and estimation parameter regardless of the orientation of the display surface 111. Can be improved.

[3. Movement that can be detected by the terminal device]
FIG. 3 is a diagram for explaining a movement that can be detected by the terminal device 1 according to the present embodiment.

In the terminal device 1 according to the present embodiment, the acceleration sensor 22 may be able to detect the rotation of the first housing 11 around the three axes. For example, the acceleration sensor 22 may be able to detect rotation around a pitch axis (see axis X in the figure) parallel to the length direction of the first housing 11. Further, the acceleration sensor 22 may be able to detect rotation around the yaw axis (see axis Y in the drawing) parallel to the width direction of the first housing 11. Further, the acceleration sensor 22 may be able to detect rotation around the roll axis (see axis X in the drawing) parallel to the thickness direction of the first housing 11. As an example, the pitch axis, yaw axis and roll axis may pass through the center of the first housing 11. In addition to / instead of the acceleration sensor 22, the terminal device 1 may use another sensor as the second sensor. The other sensor may be, for example, an angular velocity sensor, as long as it can measure a physical quantity for detecting at least one of the movement and the posture of the terminal device 1 independently or in cooperation with another sensor.

Further, in the terminal device 1, the geomagnetic sensor 23 may be able to detect the direction in which the terminal device 1 is facing. By using such a geomagnetic sensor 23 together with the acceleration sensor 22, it is possible to detect movements with low detection accuracy only with the acceleration sensor 22, for example, movements of the terminal device 1 in which acceleration acts in the horizontal direction with high accuracy. it can. In addition to / instead of the geomagnetic sensor 23, the terminal device 1 may include a gyro sensor for measuring angular velocity or angular acceleration. By using the gyro sensor together with the acceleration sensor 22, it is possible to detect the movement of the terminal device 1 such that the acceleration acts in the horizontal direction with high accuracy.

Further, in the terminal device 1, the positional relationship sensor 24 may be able to detect the relative positional relationship between the first housing 11 and the second housing 12. For example, the angle formed by the first housing 11 and the second housing 12 may be detectable.

[4. Operation of terminal device]
FIG. 4 is a flowchart showing the operation of the terminal device 1 according to the present embodiment. By executing the processes of S1 to S9, the terminal device 1 estimates the state of the target space, for example, whether or not the detection target exists.

First, in S1, the terminal device 1 performs the activation process in response to the activation operation by the operator, and is in the activation state. For example, the terminal device 1 may enable the infrared sensor 21, the acceleration sensor 22, the geomagnetic sensor 23, and the positional relationship sensor 24.

Next, in S3, the detection unit 301 determines whether or not the movement of the terminal device 1 (for example, at least one of the movement maintaining the posture and the posture change) is detected by the detection unit 301. For example, the detection unit 301 may detect the movement of the terminal device 1 by using the acceleration output value received from the acceleration sensor 22. As an example, the detection unit 301 may detect the motion by comparing the acceleration output value from the acceleration sensor 22 with the threshold value. Further, the detection unit 301 may detect the motion by further comparing the output value from at least one of the geomagnetic sensor 23 and the positional relationship sensor 24 with the threshold value. In the process of S3, it is sufficient to determine whether or not the movement of the terminal device 1 is detected, and the posture itself does not have to be detected.

When it is determined that the movement of the terminal device 1 is detected in S3 (S3; Yes), the control unit 303 repeats the process of S3. As a result, the estimation performed by the estimation unit 302 in S8 and S9 and later described later is disabled. Note that the estimation is disabled may mean that the output of the estimation result is disabled.

If the movement of the terminal device 1 is not detected in S3 (S3; No), then in S5, the control unit 303 uses the acceleration output value received from the acceleration sensor 22, which is the terminal device 1 at the present time. Detects the posture. The detection unit 301 may further use the geomagnetic output value received from the geomagnetic sensor 23 to detect the posture of the terminal device 1.

For example, the detection unit 301 may detect whether or not the terminal device 1 is in a closed state with the display 110 inside, or may detect whether or not the terminal device 1 is in a closed state. In addition, the detection unit 301 may detect whether the terminal device 1 is in any form of laptop type, tent type, tablet type, or stand type. Further, the detection unit 301 may detect whether the height of the upper end of the display surface 111 (the end on the side where the infrared sensor 21 is provided) is higher or lower than the height of the lower end.

Further, the detection unit 301 may detect whether the orientation of the display surface 111 is diagonally upward or diagonally downward. The direction of the display surface 111 is obliquely upward, for example, the display surface 111 may be oriented upward in a state of being tilted with respect to the direction of gravity, and the display surface 111 is perpendicular to the direction of gravity. And / or parallel cases may be included. The direction of the display surface 111 diagonally downward may mean, for example, that the display surface 111 is tilted downward with respect to the direction of gravity, and the display surface 111 is perpendicular to the direction of gravity. And / or parallel cases may be included. As an example, the detection unit 301 may detect the angle of the first housing 11 around the pitch axis. The orientation of the display surface 111 and the angle of the first housing 11 will be described in detail later with reference to FIGS. 11A to 11G.

Next, in S7, the control unit 303 determines whether or not the movement of the terminal device 1 has been detected after the activation process in S1. When it is determined in S7 that the movement of the terminal device 1 has never been detected (S7; No), then in S8, the terminal device 1 is controlled according to the initial posture of the terminal device 1. After performing the response control, the process is transferred to S3. The details of the initial posture correspondence control process will be described later with reference to FIG.

Further, when it is determined in S7 that the movement of the terminal device 1 has been detected after the activation process of S1 (S7; Yes), then in S9, the terminal device 1 is different from the initial posture. After performing posture correspondence control, which is control according to the posture of the terminal device 1, the process shifts to S3. As described above, when the movement of the terminal device 1 is detected, the estimation is performed after these are completed. The details of the posture correspondence control process will be described later with reference to FIG.

[4-1. Initial posture correspondence control processing]
FIG. 5 is a flowchart showing the operation of the initial posture correspondence control process. In this initial posture correspondence control process and the posture correspondence control process described later, the infrared output value from the infrared sensor 21 provided in the terminal device 1 is used to describe a spatial state, for example, a person as a detection target. Existence / absence is presumed. Further, the estimation of the spatial state is controlled according to at least one of the movement and the posture of the terminal device 1. For example, at least the estimation algorithm and estimation parameters by the estimation unit 302 are determined according to the orientation of the display surface 111 when the movement of the terminal device 1 is completed (as an example, the orientation of the display surface 111 detected by the process of S5 described above). One is changed.

In this process, first, in S31, the control unit 303 determines the orientation of the display surface 111. For example, the control unit 303 may determine whether the orientation of the display surface 111 is diagonally upward or diagonally downward.

When it is determined in S31 that the direction is diagonally upward (S31; diagonally upward), the control unit 303 then selects the second estimation algorithm as the estimation algorithm by the estimation unit 302 in S35. Here, the second estimation algorithm is an estimation algorithm on the premise that the detection target exists or does not exist or is in an unknown state.

Next, in S36, the terminal device 1 performs estimation by the second estimation algorithm and ends the attitude correspondence control. The details of the estimation process by the second estimation algorithm will be described later with reference to FIG.

When it is determined in S31 that the orientation of the display surface 111 is diagonally downward (S31; diagonally downward), then in S34, the control unit 303 determines whether or not the upper end of the display surface 111 is lower than the lower end. To do. When it is determined in S34 that the upper end of the display surface 111 is lower than the lower end (S34; Yes), the control unit 303 shifts the process to S35.

When it is determined in S34 that the upper end of the display surface 111 is higher than the lower end (S34; No), then in S37, the control unit 303 determines whether or not the terminal device 1 is in the closing state. .. When it is determined in S37 that the terminal device 1 is not in the closing state (S37; No), the control unit 303 shifts the process to S35.

When it is determined in S37 that the terminal device 1 is in a closed state (S37; Yes), then in S38, the control unit 303 is different from the second estimation algorithm as the estimation algorithm by the estimation unit 302. Select a third estimation algorithm. Here, the third estimation algorithm is an estimation algorithm on the premise that the detection target does not exist.

Next, in S39, the terminal device 1 performs estimation by the third estimation algorithm and ends the attitude correspondence control. As a result, when the terminal device 1 is in the closed state immediately after the start-up, the estimation is performed on the premise that the detection target does not exist. The estimation process by the third estimation algorithm will be described in detail later with reference to FIG.

[4-2. Posture correspondence control processing]
FIG. 6 is a flowchart showing the operation of the posture correspondence control process.

In this process, first, in S51, the control unit 303 determines the orientation of the display surface 111. For example, the control unit 303 may determine whether the orientation of the display surface 111 is diagonally upward or diagonally downward.

When it is determined in S51 that the direction is diagonally upward (S51; diagonally upward), then in S52, the control unit 303 uses the second estimation algorithm and the third estimation algorithm as the estimation algorithm by the estimation unit 302. Selects a different first estimation algorithm. Here, the first estimation algorithm is an estimation algorithm on the premise that a detection target exists.

Next, in S53, the terminal device 1 performs estimation by the first estimation algorithm and ends the attitude correspondence control. The estimation process by the first estimation algorithm will be described in detail later with reference to FIG. 7.

When it is determined in S51 that the orientation of the display surface 111 is diagonally downward (S51; diagonally downward), then in S54, the control unit 303 determines whether or not the upper end of the display surface 111 is lower than the lower end. To do.

When it is determined in S54 that the upper end of the display surface 111 is lower than the lower end (S54; Yes), then in S55, the control unit 303 selects the second estimation algorithm as the estimation algorithm by the estimation unit 302. ..

Next, in S56, the terminal device 1 performs estimation by the second estimation algorithm and ends the attitude correspondence control. The details of the estimation process by the second estimation algorithm will be described later with reference to FIG.

When it is determined in S54 that the upper end of the display surface 111 is higher than the lower end (S54; No), then in S57, the control unit 303 determines whether or not the terminal device 1 is in the closing state. .. When it is determined in S57 that the terminal device 1 is not in the closing state (S57; No), the control unit 303 shifts the process to S55.

When it is determined in S57 that the terminal device 1 is in a closed state (S57; Yes), then in S58, the control unit 303 selects a third estimation algorithm as the estimation algorithm by the estimation unit 302. ..

Next, in S59, the terminal device 1 performs estimation by the third estimation algorithm and ends the attitude correspondence control. The estimation process by the third estimation algorithm will be described in detail later with reference to FIG.

[4-2. Estimation processing by the first estimation algorithm]
FIG. 7 is a flowchart showing the operation of the estimation process by the first estimation algorithm. The first estimation algorithm estimates on the premise that the detection target exists when the movement of the terminal device 1 is completed.

First, in S101, the control unit 303 performs a threshold value setting process for setting the first threshold value. For example, since it is estimated that the detection target exists at the present time, the control unit 303 may set the first threshold value so that it can be determined that the detection target has changed from the existence to the absence. As an example, the control unit 303 may set the first threshold value lower than the current infrared output value. The control unit 303 may select any one of a plurality of candidates for the first threshold value stored in the estimation device 10 in advance and set it as the first threshold value. The control unit 303 may use a fixed value stored in the estimation device 10 in advance as the first threshold value regardless of the current infrared output value.

Next, in S103, the control unit 303 determines whether or not the movement of the terminal device 1 is detected, as in S3 described above. When it is determined in S103 that the movement of the terminal device 1 is detected (S103; Yes), the control unit 303 ends the estimation process by the first estimation algorithm. As a result, the estimation performed by the estimation unit 302 in S105 and later described later is disabled.

If it is determined in S103 that the movement of the terminal device 1 is not detected (S103; No), then in S105, the estimation unit 302 determines whether or not the infrared output value has fallen below the first threshold value.

When it is determined in S105 that the infrared output value is not lower than the first threshold value (S105; No), that is, when it is estimated that the detection target still exists, the control unit 303 sets the process to S101. Migrate. As a result, the first threshold value is updated according to the current infrared output value.

If it is determined in S105 that the infrared output value is below the first threshold value (S105; Yes), then in S107, the estimation unit 302 estimates that the detection target does not exist.

Next, in S109, the user interface 30 shifts the terminal device 1 to the logout state. However, if the terminal device 1 is not in the logged-in state, the process of S109 may not be performed.

Next, in S110, the control unit 303 performs a threshold value setting process for setting the seventh threshold value. For example, since it is estimated that the detection target does not exist at the present time, the control unit 303 may set the seventh threshold value so that it can be determined that the detection target has changed from the absence to the existence. As an example, the control unit 303 may set the seventh threshold value higher than the current infrared output value. The control unit 303 may set the same value as the first threshold value as the seventh threshold value, or may set a value higher or lower than the first threshold value. The control unit 303 may select any one of a plurality of candidates for the seventh threshold value stored in the estimation device 10 in advance and set it as the seventh threshold value. The control unit 303 may use a fixed value stored in the estimation device 10 in advance as the seventh threshold value regardless of the current infrared output value.

Next, in S111, the control unit 303 determines whether or not the movement of the terminal device 1 is detected, as in S3 described above. When it is determined that the movement of the terminal device 1 is detected in S111 (S111; Yes), the control unit 303 ends the estimation process by the first estimation algorithm. As a result, the estimation performed by the estimation unit 302 in S113 and later described later is disabled.

If it is determined in S111 that the movement of the terminal device 1 is not detected (S111; No), then in S113, the estimation unit 302 determines whether or not the infrared output value exceeds the seventh threshold value. When it is determined in S113 that the infrared output value does not exceed the seventh threshold value (S113; No), that is, when it is estimated that the detection target remains absent, the control unit 303 performs processing in S110. To migrate to.

When it is determined in S113 that the infrared output value exceeds the seventh threshold value (S113; Yes), the estimation unit 302 next estimates in S115 that a detection target exists. Next, in S117, the terminal device 1 performs the automatic login process in the same manner as in S1 described above, and then shifts the process to S101. If the terminal device 1 is not in the logout state, the process of S117 may not be performed.

[4-3. Estimation processing by the third estimation algorithm]
FIG. 8 is a flowchart showing the operation of the estimation process by the third estimation algorithm. The third estimation algorithm estimates on the premise that the detection target does not exist when the movement of the terminal device 1 is completed.

First, in S121, the control unit 303 performs the seventh threshold value setting process. For example, since it is estimated that the detection target does not exist at the present time, the control unit 303 may set the seventh threshold value so that it can be determined that the detection target has changed from the absence to the existence. As an example, the control unit 303 may set the seventh threshold value higher than the current infrared output value. The control unit 303 may select any one of a plurality of candidates for the seventh threshold value stored in the estimation device 10 in advance and set it as the seventh threshold value. The control unit 303 may use a fixed value stored in the estimation device 10 in advance as the seventh threshold value regardless of the current infrared output value. The seventh threshold used in the third estimation algorithm may be different from the seventh threshold used in the first estimation algorithm.

Next, in S123, the control unit 303 determines whether or not the movement of the terminal device 1 is detected, as in S3 described above. When it is determined in S123 that the movement of the terminal device 1 is detected (S123; Yes), the control unit 303 ends the estimation process by the third estimation algorithm. As a result, the estimation performed by the estimation unit 302 in S125 and later described later is disabled.

When it is determined in S123 that the movement of the terminal device 1 is not detected (S123; No), then in S125, the estimation unit 302 determines whether or not the infrared output value exceeds the seventh threshold value. When it is determined in S125 that the infrared output value does not exceed the seventh threshold value (S125; No), that is, when it is estimated that the detection target remains absent, the control unit 303 performs processing in S121. To migrate to. As a result, the seventh threshold value is updated according to the current infrared output value.

If it is determined in S125 that the infrared output value exceeds the seventh threshold value (S125; Yes), then in S127, the estimation unit 302 estimates that the detection target exists.

Next, in S129, the terminal device 1 performs an automatic login process in the same manner as in S1 described above. If the terminal device 1 is not in the logout state, the process of S129 may not be performed.

Next, in S130, the control unit 303 performs a threshold value setting process for setting the first threshold value. For example, since it is estimated that the detection target exists at the present time, the control unit 303 may set the first threshold value so that it can be determined that the detection target has changed from the existence to the absence. As an example, the control unit 303 may set the first threshold value lower than the current infrared output value. The control unit 303 may set the same value as the seventh threshold value as the first threshold value, or may set a value higher or lower than the seventh threshold value. The control unit 303 may select any one of a plurality of candidates for the first threshold value stored in the estimation device 10 in advance and set it as the first threshold value. The control unit 303 may use a fixed value stored in the estimation device 10 in advance as the first threshold value regardless of the current infrared output value. The first threshold used in the third estimation algorithm may be different from the first threshold used in the first estimation algorithm.

Next, in S131, the control unit 303 determines whether or not the movement of the terminal device 1 has been detected, as in S3 described above. When it is determined that the movement of the terminal device 1 is detected in S131 (S131; Yes), the control unit 303 ends the estimation process by the third estimation algorithm. As a result, the estimation performed by the estimation unit 302 in S133 and later described later is disabled.

If it is determined in S131 that the movement of the terminal device 1 is not detected (S131; No), then in S133, the estimation unit 302 determines whether or not the infrared output value is below the first threshold value. When it is determined in S133 that the infrared output value does not fall below the first threshold value (S133; No), that is, when it is estimated that the detection target still exists, the control unit 303 sets the process to S130. Migrate.

If it is determined in S133 that the infrared output value is below the first threshold value (S133; Yes), then in S135, the estimation unit 302 estimates that the detection target does not exist. Next, in S137, the user interface 30 shifts the terminal device 1 to the logout state, and then shifts the process to S121. However, if the terminal device 1 is not in the logged-in state, the process of S137 may not be performed.

[4-4. Estimate processing by the second estimation algorithm]
FIG. 9 is a flowchart showing the operation of the estimation process by the second estimation algorithm. The second estimation algorithm estimates on the premise that when the movement of the terminal device 1 is completed, it is unknown whether the detection target exists or does not exist.

First, in S151, the control unit 303 performs the threshold value setting process. For example, since it is unknown at this time whether or not the detection target exists, the control unit 303 has a third threshold value for determining that the detection target exists and a fifth threshold value for determining that the detection target does not exist. A threshold value may be set. As an example, the control unit 303 may use fixed values stored in the estimation device 10 in advance as the third threshold value and the fifth threshold value, regardless of the current infrared output value. The third threshold value and the fifth threshold value may be the same value or may be different values. Further, the third threshold value may be the same as or different from any of the candidates for the seventh threshold value used in the first and third estimation algorithms. Further, the fifth threshold value may be the same as or different from any of the candidates for the first threshold value used in the first and third estimation algorithms.

Next, in S153, the control unit 303 determines whether or not the movement of the terminal device 1 has been detected, as in S3 described above. When it is determined in S153 that the movement of the terminal device 1 is detected (S153; Yes), the control unit 303 ends the estimation process by the second estimation algorithm. As a result, the estimation performed by the estimation unit 302 in S155 and later described later is disabled.

If it is determined in S153 that the movement of the terminal device 1 is not detected (S153; No), then in S155, the estimation unit 302 determines whether or not the infrared output value from the infrared sensor 21 exceeds the third threshold value. judge.

When it is determined in S155 that the infrared output value exceeds the third threshold value (S155; Yes), then in S157, the estimation unit 302 estimates that the detection target exists.

Next, in S159, the terminal device 1 performs an automatic login process in the same manner as in S1 described above. If the terminal device 1 is not in the logout state, the process of S159 may not be performed.

Next, in S161, the estimation device 10 finishes the estimation process by the second estimation algorithm after performing the estimation by the first estimation algorithm in the same manner as in S53 described above.

When it is determined in S155 that the infrared output value does not exceed the third threshold value (S155; No), then in S163, the control unit 303 detects the movement of the terminal device 1 as in S3 described above. Determine if it has been done. When it is determined in S163 that the movement of the terminal device 1 is detected (S163; Yes), the control unit 303 ends the estimation process by the second estimation algorithm. As a result, the estimation performed by the estimation unit 302 is performed in S165 and later described later.

When it is determined in S163 that the movement of the terminal device 1 is not detected (S163; No), then in S165, the estimation unit 302 determines whether or not the infrared output value from the infrared sensor 21 has fallen below the fifth threshold value. To judge.

When it is determined in S165 that the infrared output value does not fall below the fifth threshold value (S165; No), the control unit 303 shifts the process to S151. As a result, the third threshold value and the fifth threshold value are updated according to the current infrared output value. Further, when it is determined in S165 that the infrared output value is below the fifth threshold value (S165; Yes), then in S167, the estimation unit 302 estimates that the detection target does not exist.

Next, in S169, the user interface 30 shifts the terminal device 1 to the logout state. However, if the terminal device 1 is not in the logged-in state, the process of S169 may not be performed. Next, in S171, the estimation device 10 finishes the estimation process by the second estimation algorithm after performing the estimation by the third estimation algorithm in the same manner as in S59 described above.

[4-5. Automatic login process]
FIG. 10 is a flowchart showing the operation of the automatic login process. The terminal device 1 automatically performs a login process by executing the processes S21 to S30.

First, in S21, the user interface 30 determines whether or not the terminal device 1 is in the logged-in state. When it is determined that the terminal device 1 is in the login state (S21; Yes), the terminal device 1 ends the automatic login process.

When it is determined that the terminal device 1 is not in the login state (S21; No), the image pickup control unit 50 then activates the image pickup unit 40 in S23. Next, in S25, the imaging unit 40 images the space in the field of view of the infrared sensor 21, and then in S27, the authentication unit 60 is activated to perform face recognition on the face in the captured image supplied from the imaging unit 40.

Next, in S29, the authentication unit 60 determines whether or not the authentication is successful, and if it is determined that the authentication is unsuccessful (S29; No), the terminal device 1 shifts the process to S25. However, in this case, the terminal device 1 may terminate the subroutine of the automatic login process without performing the login process according to S30 described later. When it is determined in S29 that the authentication is successful (S29; Yes), in S30, the user interface 30 shifts the terminal device 1 to the login state and ends the automatic login process.

[5. Relationship between display surface orientation and estimation algorithm]
11A to 11G are diagrams for explaining the orientation of the display surface 111 corresponding to each estimation algorithm. In the posture correspondence control process described above, one of the first to third estimation algorithms is selected according to the orientation of the display surface 111 when the movement is completed.

The orientation of the display surface 111 may be defined by the angle θ formed by the normal line of the display surface 111 and the horizontal reference line L. For example, when the first housing 11 is vertically arranged with the infrared sensor 21 side as the upper side, and is rotated once forward around the pitch axis while keeping the length direction (pitch axis direction) horizontal. In addition, the angle θ changes from 0 ° to 360 ° according to the order of FIGS. 11A to 11F (or FIG. 11G). The angle θ may be defined as the first housing 11 rotates about the yaw axis with the width direction (yaw axis direction) horizontal. Here, one rotation of the first housing 11 in the front direction may mean that the display surface 111 rotates so as to face downward and then upward. In FIGS. 11F and 11G, the positions of the second housing 12 with respect to the first housing 11 are different.

According to such an angle θ, when the direction of the display surface 111 is obliquely upward, the angle θ is in a specific angle range, for example, 180 <θ <270 [°] (see FIG. 11D), 270 <θ ≦ 360. It becomes [°] (see FIGS. 11F and 11G). In this case, the estimation is performed using the first algorithm that assumes that the detection target exists.

As shown in FIG. 11D, when 180 <θ <270 [°] when the movement of the terminal device 1 is completed, the operator can operate the terminal device 1 via the touch sensor 112 or use it for moving image playback. This is because it is considered that the terminal device 1 is deformed, and it is highly probable that the operator is present in the field of view of the infrared sensor 21. Further, as shown in FIGS. 11F and 11G, when 270 <θ≤360 [°] when the movement is completed, it is considered that the terminal device 1 is deformed so that the operator can easily use it. This is because there is a high possibility that the operator is present in the field of view of the infrared sensor 21.

Further, when the orientation of the display surface 111 is between horizontal and downward and the upper end of the display surface 111 is higher than the lower end, the angle θ is in a specific angle range, for example, 0 ≦ θ <90 [°]. (See FIG. 11A).

Of these, for example, when the orientation of the display surface is sideways rather than the reference orientation, for example, when 0 ≦ θ <45 [°], it is unknown whether the detection target exists or does not exist. The estimation is performed using the second algorithm premised on. As shown in FIG. 11A, if 0 ≦ θ <45 [°] when the movement is completed, the operator may have deformed the terminal device 1 so as to leave it unattended, and the operator may prefer it. This is because it is difficult to determine in advance whether or not the operator is present in the field of view of the infrared sensor 21 because the terminal device 1 may have been deformed accordingly.

Further, for example, when the orientation of the display surface is downward from the reference orientation (when the terminal device 1 is about to close), for example, when 45 ≦ θ <90 [°], it means that the detection target does not exist. The estimation is performed using the presupposed third algorithm. As shown in FIG. 11A, when 45 ≦ θ <90 [°] when the movement is completed, it is considered that the terminal device 1 is deformed in order to hide the display surface 111 when the operator leaves the seat. This is because there is a high possibility that the operator does not exist in the field of view of the infrared sensor 21.

Further, when the direction of the display surface 111 is obliquely downward and the upper end of the display surface 111 is lower than the lower end, the angle θ is in a specific angle range, for example, 90 <θ ≦ 180 [°] (see FIG. 11C). It becomes. In this case, estimation is performed using a second algorithm that assumes that the detection target exists or does not exist or is in an unknown state.

As shown in FIG. 11C, if 90 <θ≤180 [°] when the movement is completed, the operator may have deformed the terminal device 1 to leave it unattended, and the terminal device 1 should be used on its back. This is because it is difficult to determine in advance whether or not the operator is present in the field of view of the infrared sensor 21 because it may have been deformed.

As shown in FIG. 11B, when θ = 90 [°] when the movement is completed, that is, when the display surface 111 is downward, as in the case of 90 <θ ≦ 180 [°], Since it is difficult to determine in advance whether or not the operator is present in the field of view of the infrared sensor 21, estimation may be performed using the second algorithm.

Further, as shown in FIG. 11E, when θ = 270 [°] when the movement is completed, that is, when the display surface 111 is upward, the operator may have deformed the terminal device 1 so as to leave it unattended. It may have been transformed for use. Therefore, even in this case, it is difficult to determine in advance whether or not the operator is present in the field of view of the infrared sensor 21, so estimation may be performed using the second algorithm.

[6. Relationship between display surface orientation and threshold value]
12A and 12B are diagrams for explaining the setting of the threshold value depending on the orientation of the display surface 111.

As shown in FIGS. 12A and 12B, when the display surface 111 is oriented diagonally upward, the operator operating the terminal device 1 (for example, the face portion of the operator) is in the field of view of the infrared sensor 21. As a result of the area occupied in the area changing according to the orientation of the display surface 111, the infrared output value changes. For example, the terminal device 1 is operated when the display surface 111 has a more upward second orientation (FIG. 12B) as compared with the case where the display surface 111 has a certain first orientation (FIG. 12A). As a result of reducing the area occupied by the operator in the field of view of the infrared sensor 21, the infrared output value becomes low. Therefore, when the display surface 111 is oriented in a certain direction (for example, FIG. 12A), the threshold value that can be used for comparison with the infrared output value for estimating the presence / absence of the operator is used as it is in the other direction (for example, FIG. 12B). ), The estimation accuracy will decrease.

Further, even when the orientation of the display surface 111 is obliquely downward, the area occupied by the operator operating the terminal device 1 on his back in the field of view of the infrared sensor 21 changes according to the orientation of the display surface 111. Therefore, in at least one threshold setting process of S101, S110, S121, S130, and S151 in the above-mentioned first to third estimation algorithms, the control unit 303 has a threshold value to be compared with the infrared output value, that is, the first threshold value and the first threshold value. Any of the 3rd threshold value, the 5th threshold value, and the 7th threshold value may be changed according to the orientation of the display surface 111.

For example, in the threshold value setting process of S101 in the first estimation algorithm, the control unit 303 has a first orientation when the orientation of the display surface 111 is a second orientation upward from the first orientation (for example, FIG. 12B). At least one of the first threshold value and the seventh threshold value may be made smaller than in the case of orientation (for example, FIG. 12A). As a result, when the orientation of the display surface 111 is the second orientation, the threshold value becomes smaller, and as a result, the estimation accuracy is prevented from being lowered.

[7. Relationship between the angle of the first housing and the second housing and the threshold value]
13A and 13B are diagrams for explaining the setting of the threshold value according to the angle of the first housing 11 and the second housing 12.

As shown in FIGS. 13A and 13B, the distance between the terminal device 1 and the operator is different depending on whether the terminal device 1 is used in the laptop type or the tablet type. If the distance between the terminal device 1 and the operator is different, the area occupied by the operator operating the terminal device 1 (for example, the face of the operator) in the field of view of the infrared sensor 21 is different, resulting in an infrared output value. different. As an example, in the case of the laptop type, the area occupied by the operator operating the terminal device 1 in the field of view of the infrared sensor 21 becomes smaller than in the case of the tablet type, and as a result, the infrared output value becomes lower. ..

Therefore, in at least one threshold setting process of S101, S110, S121, S130, and S151 in the above-mentioned first to third estimation algorithms, the control unit 303 responds to the angle of the second housing 12 with respect to the first housing 11. Therefore, the threshold value to be compared with the infrared output value, that is, any of the first threshold value, the third threshold value, the fifth threshold value, and the seventh threshold value may be changed. For example, the control unit 303 makes the threshold value smaller when the first housing 11 and the second housing 12 form a laptop type than when they form a tablet type. You can do it.

In addition to / instead of this, in at least one of the first to third estimation algorithms, the control unit 303 changes the content of the estimation algorithm according to the angle of the second housing 12 with respect to the first housing 11. You may. For example, the control unit 303 does not estimate by comparing the infrared output value and the threshold value according to the angle of the second housing 12 with respect to the first housing 11, but the fluctuation amount of the infrared output value and the threshold value. Estimates may be made by comparison. As an example, when the angle of the second housing 12 with respect to the first housing 11 is larger or smaller than the reference angle, the control unit 303 estimates by comparing the fluctuation amount of the infrared output value with the threshold value. You can. For example, the control unit 303 may presume that a detection target exists when the fluctuation amount of the infrared output value is larger than the threshold value as an example of the fourth threshold value and / or the eighth threshold value. Further, the control unit 303 may estimate that the detection target does not exist when the fluctuation amount falls below the threshold value as an example of the second threshold value and / or the sixth threshold value for a certain period of time. The amount of fluctuation of the infrared output value may be the amount of fluctuation caused by an external factor, and becomes large due to, for example, the movement of a person in the field of view of the sensor. As the amount of fluctuation, the difference value between the low-pass value obtained by applying the low-pass filter to the infrared output value and the raw data may be used, or the difference value between the moving average value of the infrared output value and the raw data may be used.

[8. Relationship between the posture of the mounted terminal device and the threshold value (1)]
14A to 14D are diagrams for explaining the setting of the threshold value depending on the posture of the mounted terminal device 1.

As shown in FIGS. 14A to 14D, the positional relationship between the display surface 111 and the infrared sensor 21 differs depending on how the first housing 11 is erected. If the positional relationship is different, the area occupied by the face portion of the operator operating the terminal device 1 in the field of view of the infrared sensor 21 is different.

For example, in FIG. 14A, the infrared sensor 21 is located above the display surface 111, in FIGS. 14B and 14C, the infrared sensor 21 is located on the side of the display surface 111, and in FIG. 14D, the infrared sensor 21 is located on the display surface. It is located below 111. As a result, the area occupied by the operator's face in the field of view of the infrared sensor 21, and thus the infrared output value, is the largest in the case of FIG. 14A, followed by the case of FIGS. 14B and 14C, and the largest in the case of FIG. 14D. It becomes smaller.

Therefore, in at least one threshold setting process of S101, S110, S121, S130, and S151 in the above-mentioned first to third estimation algorithms, the control unit 303 uses the display surface 111 and the infrared sensor depending on how the first housing 11 is erected. Depending on the positional relationship with 21, the threshold value to be compared with the infrared output value, that is, any of the first threshold value, the third threshold value, the fifth threshold value, and the seventh threshold value may be changed. For example, when the infrared sensor 21 is above the display surface 111 (FIG. 14A), the control unit 303 is more likely to be sideways (FIG. 14B, 14C) and / or below (FIG. 14D). May also increase the threshold. Further, the control unit 303 may have a larger threshold value when the infrared sensor 21 is on the side of the display surface 111 (FIGS. 14B and 14C) than when it is on the lower side (FIG. 14D).

In addition to / instead of this, in at least one of the first to third estimation algorithms, the control unit 303 depends on the positional relationship between the display surface 111 and the infrared sensor 21 depending on how the first housing 11 is erected. The content of the estimation algorithm may be changed. For example, the control unit 303 changes the infrared output value according to the positional relationship between the display surface 111 and the infrared sensor 21 depending on how the first housing 11 is erected, instead of estimating by comparing the infrared output value with its threshold value. Estimates may be made by comparing the amount with its threshold. As an example, the control unit 303 determines the magnitude of the fluctuation amount and the magnitude of the fluctuation amount in any one or two of the cases where the infrared sensor 21 is above the display surface 111, sideways, and below the display surface 111. Estimates may be made by comparison with the threshold value.

[9. Relationship between the posture of the mounted terminal device and the threshold value (2)]
15A and 15B are diagrams for explaining the setting of the threshold value depending on the posture of the mounted terminal device 1.

As shown in FIGS. 15A and 15B, the positional relationship between the display surface 111 and the infrared sensor 21 differs depending on how the first housing 11 is erected, and if the positional relationship is different, the terminal device 1 is operated. The frequency with which the operator's hand portion enters the field of view of the infrared sensor 21 is different. For example, in FIG. 15A, the infrared sensor 21 is located above the display surface 111, and in FIG. 15B, the infrared sensor 21 is located below the display surface 111. As a result, the frequency with which the operator's hand portion enters the field of view of the infrared sensor 21, and thus the fluctuation amount of the infrared output value, becomes larger in the case of FIG. 15B than in the case of FIG. 15A.

Therefore, in at least one threshold setting process of S101, S110, S121, S130, and S151 in the above-mentioned first to third estimation algorithms, the control unit 303 uses the display surface 111 and the infrared sensor depending on how the first housing 11 is erected. Depending on the positional relationship with 21, the threshold value to be compared with the infrared output value, that is, any of the first threshold value, the third threshold value, the fifth threshold value, and the seventh threshold value may be changed. For example, the control unit 303 may set the threshold value larger when the infrared sensor 21 is below the display surface 111 (FIG. 15B) than when it is above the display surface 111 (FIG. 15A).

In addition to / instead of this, in at least one of the first to third estimation algorithms, the control unit 303 depends on the positional relationship between the display surface 111 and the infrared sensor 21 depending on how the first housing 11 is erected. The content of the estimation algorithm may be changed. For example, the control unit 303 changes the infrared output value according to the positional relationship between the display surface 111 and the infrared sensor 21 depending on how the first housing 11 is erected, instead of estimating by comparing the infrared output value with its threshold value. Estimates may be made by comparing the amount with its threshold. As an example, the control unit 303 estimates by comparing the magnitude of the fluctuation amount of the infrared output value with the threshold value in either the case where the infrared sensor 21 is above the display surface 111 or the case where the infrared sensor 21 is below the display surface 111. You may go.

[10. Relationship between the posture of the mounted terminal device and the threshold value (3)]
16A to 16C are diagrams for explaining the setting of the threshold value depending on the posture of the mounted terminal device 1.

As shown in FIGS. 16A to 16C, the positional relationship between the display surface 111 and the infrared sensor 21 differs depending on how the first housing 11 is erected, and when the erection and the positional relationship are different, the terminal device 1 is placed. The area occupied by the upper surface of the desk or the second housing 12 in the field of view of the infrared sensor 21 is different.

For example, in FIG. 16A, the first housing 11 is erected on the second housing 12 facing the side opposite to the second housing 12, and the terminal device 1 is a stand type. Further, the infrared sensor 21 is located above the display surface 111.

In FIG. 16B, the first housing 11 is leaned against the second housing 12 facing the side opposite to the second housing 12, and the terminal device 1 has a tent shape. Further, the infrared sensor 21 is located below the display surface 111. In this state, the area occupied by the desk in the field of view of the infrared sensor 21 is larger than that in the case of FIG. 16A.

In FIG. 16C, the first housing 11 is erected on the second housing 12 facing the side of the second housing 12, and the terminal device 1 is a laptop type. Further, the infrared sensor 21 is located below the display surface 111. In this state, unlike the cases of FIGS. 16A and 16B, the second housing 12 exists in the field of view of the infrared sensor 21.

Here, when the infrared sensor 21 is located above the bezel of the PC as shown in FIG. 16A, the operator's face is included in the field of view of the infrared sensor 21, whereas the infrared sensor 21 is shown in FIGS. 16B and 16C. When 21 is in the lower part of the bezel, the vicinity of the abdomen of the operator wearing the clothing is included in the field of view of the infrared sensor 21. Further, in FIGS. 16B and 16C, the desk and the keyboard are embedded in the lower half of the field of view, and the temperature of the keyboard is higher than that of the desk. Therefore, the infrared output value when the operator is present in the field of view may be higher in the order of FIG. 16B <FIG. 16C <FIG. 16A. The fluctuation amount of the infrared output when the operator does not exist in the field of view and when the operator exists may be larger in FIG. 16A than in FIGS. 16B and 11C.

Therefore, in at least one threshold setting process of S101, S110, S121, S130, and S151 in the above-mentioned first to third estimation algorithms, the control unit 303 sets the first housing 11 and displays the first housing 11 according to the setting. Depending on the positional relationship between the surface 111 and the infrared sensor 21, the threshold value to be compared with the infrared output value, that is, any of the first threshold value, the third threshold value, the fifth threshold value, and the seventh threshold value may be changed. For example, the control unit 303 may have a larger threshold in the case of FIG. 16A than in the case of FIG. 16B and / or the case of FIG. 16C. Further, in the case of FIG. 16C, the control unit 303 may have a larger threshold value than in the case of FIG. 16B.

In addition to / instead of this, in at least one of the first to third estimation algorithms, the control unit 303 has a positional relationship between the standing method of the first housing 11 and the display surface 111 and the infrared sensor 21 according to the standing method. The content of the estimation algorithm may be changed accordingly. For example, the control unit 303 does not estimate by comparing the infrared output value and the threshold value according to the standing method of the first housing 11 and the positional relationship between the display surface 111 and the infrared sensor 21 according to the standing method. Estimation may be performed by comparing the fluctuation amount of the infrared output value with the threshold value. As an example, in any one or two of FIGS. 16A to 16C, the control unit 303 may perform estimation by comparing the fluctuation amount of the infrared output value with the threshold value thereof.

[11. Operation example of terminal device]
FIG. 17 is a diagram showing an operation example of the terminal device 1. FIG. 17 illustrates the transition of the infrared output value, the acceleration dispersion value, and the geomagnetic output dispersion value when the operator deforms the terminal device 1 during use.

First, as shown in the “A” part in the drawing, when the operator is using the terminal device 1 in a laptop type with the display surface 111 facing diagonally upward, the terminal device 1 is used by the first estimation algorithm. It is repeatedly determined whether or not the motion of (S103) is detected and whether or not the infrared output value is below the first threshold value (S105). In this state, it is estimated by the estimation unit 302 that an operator exists.

Subsequently, as shown in the “B” portion in the figure, when the operator lifts the terminal device 1 to deform it into a tent shape, for example, the acceleration output value from the acceleration sensor 22 exceeds the threshold value, so that the terminal device 1 It is determined that the movement of 1 is detected (S103; Yes). As a result, the estimation by the estimation unit 302 is disabled, and the estimation process (S53) and the attitude correspondence control process (S9) by the first estimation algorithm are completed. Then, unless the movement of the terminal device 1 is completed, it is repeatedly determined whether or not the movement of the terminal device 1 is detected (S3). In this state, the presumption that there is an operator is maintained.

Subsequently, as shown in the "C" to "D" parts in the figure, when the operator deforms the terminal device 1 into a tent shape, for example, the acceleration output value from the acceleration sensor 22 exceeds the threshold value. It is determined that the movement of the terminal device 1 has been detected (S3; Yes). Then, as long as the movement is not completed, it is repeatedly determined whether or not the movement of the terminal device 1 is detected (S3). In this state, the presumption that there is an operator is maintained.

Subsequently, as shown in the "D" to "E" parts in the figure, when the operator places the terminal device 1 on the desk and rotates the terminal device 1 horizontally to adjust the orientation, for example, a geomagnetic sensor. When the geomagnetic output value from 23 exceeds the threshold value, it is determined that the movement (change in orientation) of the terminal device 1 has been detected (S3; Yes). Then, as long as the movement is not completed, it is repeatedly determined whether or not the movement of the terminal device 1 is detected (S3). In this state, the presumption that there is an operator is maintained.

Subsequently, as shown in the “E” portion in the figure, when the operator finishes adjusting the orientation of the terminal device 1, it is determined that the movement of the terminal device 1 is not detected (S3; No). Next, it is determined that the orientation of the display surface 111 is diagonally upward (S51; diagonally upward), the first estimation algorithm assuming the presence of the operator is selected, and the first threshold value is set. (S101), it is repeatedly determined whether or not the movement of the terminal device 1 is detected (S103), and whether or not the infrared output value is below the first threshold value (S105).

[10. Other computer configurations]
FIG. 18 is a block diagram showing a configuration of a computer according to the present embodiment.

FIG. 18 shows an example of the configuration of the computer 1900 according to the present embodiment. The computer 1900 according to this embodiment functions as an estimation device 10, a terminal device 1, or a part thereof.

The computer 1900 according to the present embodiment is connected to the host controller 2082 by the input / output controller 2084 and the CPU peripheral portion having the CPU 2000, the RAM 2020, the graphic controller 2075, and the display device 2080 which are interconnected by the host controller 2082. Input / output unit having communication interface 2030, hard disk drive 2040, and DVD drive 2060, ROM 2010 connected to input / output controller 2084, flash memory drive 2050, infrared sensor 21, acceleration sensor 22, geomagnetic sensor 23, position. It includes a legacy input / output unit having a related sensor 24, an image pickup unit 40, and an input / output chip 2070.

The host controller 2082 connects the RAM 2020 to the CPU 2000 and the graphic controller 2075 that access the RAM 2020 at a high transfer rate. The CPU 2000 operates based on the programs stored in the ROM 2010 and the RAM 2020, and controls each part. The graphic controller 2075 acquires image data generated on a frame buffer provided in the RAM 2020 by the CPU 2000 or the like, and displays the image data on the display device 2080. Instead of this, the graphic controller 2075 may internally include a frame buffer for storing image data generated by the CPU 2000 or the like.

The input / output controller 2084 connects the host controller 2082 to the communication interface 2030, the hard disk drive 2040, and the DVD drive 2060, which are relatively high-speed input / output devices. The communication interface 2030 communicates with other devices via a network by wire or wirelessly. In addition, the communication interface functions as hardware for communication. The hard disk drive 2040 stores programs and data used by the CPU 2000 in the computer 1900. The DVD drive 2060 reads a program or data from the DVD 2095 and provides it to the hard disk drive 2040 via the RAM 2020.

Further, the input / output controller 2084 is relatively composed of the above-mentioned infrared sensor 21, acceleration sensor 22, geomagnetic sensor 23, positional relationship sensor 24, imaging unit 40, ROM 2010, flash memory drive 2050, and input / output chip 2070. A low-speed input / output device is connected. The ROM 2010 stores a boot program executed by the computer 1900 at startup, and / or a program depending on the hardware of the computer 1900. The flash memory drive 2050 reads a program or data from the flash memory 2090 and provides it to the hard disk drive 2040 via the RAM 2020. The input / output chip 2070 connects the flash memory drive 2050 to the input / output controller 2084, and inputs / outputs various input / output devices via, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like. Connect to controller 2084.

The program provided to the hard disk drive 2040 via the RAM 2020 is stored in a recording medium such as a flash memory 2090, a DVD 2095, or an IC card and provided by the user. The program is read from the recording medium, installed on the hard disk drive 2040 in the computer 1900 via the RAM 2020, and executed in the CPU 2000.

The programs installed in the computer 1900 and causing the computer 1900 to function as at least a part of the estimation device 10 are the acquisition module, the first calculation module, the second calculation module, the first threshold calculation module, the detection module, the estimation module, and the control module. It has at least one of them. These programs or modules may act on the CPU 2000 or the like to cause the computer 1900 to function as the detection unit 301, the estimation unit 302, and the control unit 303, respectively.

The information processing described in these programs is read into the computer 1900 and acts on the CPU 2000 or the like, which is a concrete means in which the software and the various hardware resources described above cooperate with each other, to detect the computer 1900. It functions as a unit 301, an estimation unit 302, and a control unit 303. Then, by realizing the calculation or processing of information according to the purpose of use of the computer 1900 in the present embodiment by these specific means, a unique estimation device 10 according to the purpose of use is constructed.

A program installed on the computer 1900 that causes the computer 1900 to function as at least a part of the terminal device 1 comprises at least one of an estimation device module, a user interface module, an imaging control module, and an authentication module. These programs or modules may act on the CPU 2000 or the like to cause the computer 1900 to function as an estimation device 10, a user interface 30, an imaging control unit 50, and an authentication unit 60, respectively.

The information processing described in these programs is read into the computer 1900 and acts on the CPU 2000 or the like, which is a concrete means in which the software and the various hardware resources described above cooperate, to estimate the computer 1900. It functions as a device 10, a user interface 30, an imaging control unit 50, and an authentication unit 60. Then, by realizing the calculation or processing of information according to the purpose of use of the computer 1900 in the present embodiment by these specific means, a unique terminal device 1 according to the purpose of use is constructed.

As an example, when communicating between the computer 1900 and an external device or the like, the CPU 2000 executes a communication program loaded on the RAM 2020, and based on the processing content described in the communication program, a communication interface. Instruct 2030 to perform communication processing. Under the control of the CPU 2000, the communication interface 2030 reads the transmission data stored in the transmission buffer area or the like provided on the storage device such as the RAM 2020, the hard disk drive 2040, the flash memory 2090, or the DVD 2095, and transmits the transmission data to the network. Alternatively, the received data received from the network is written to a receive buffer area or the like provided on the storage device. As described above, the communication interface 2030 may transfer the transmitted / received data to / from the storage device by the DMA (direct memory access) method, and instead, the CPU 2000 may transfer the transfer source storage device or the communication interface 2030. The transmitted / received data may be transferred by reading the data from the data and writing the data to the communication interface 2030 or the storage device of the transfer destination.

Further, the CPU 2000 performs DMA all or necessary parts from files or databases stored in an external storage device such as a hard disk drive 2040, a DVD drive 2060 (DVD2095), and a flash memory drive 2050 (flash memory 2090). It is read into the RAM 2020 by transfer or the like, and various processes are performed on the data on the RAM 2020. Then, the CPU 2000 writes the processed data back to the external storage device by DMA transfer or the like. In such processing, the RAM 2020 can be regarded as temporarily holding the contents of the external storage device. Therefore, in the present embodiment, the RAM 2020 and the external storage device are collectively referred to as a memory, a storage unit, a storage device, or the like. For example, the estimation device 10 and the terminal device 1 may appropriately include a storage device that stores data before, during, and after processing of the present embodiment.

Various information such as various programs, data, tables, and databases in the present embodiment are stored in such a storage device and are subject to information processing. The CPU 2000 can also hold a part of the RAM 2020 in the cache memory and read / write on the cache memory. Even in such a form, the cache memory plays a part of the function of the RAM 2020. Therefore, in the present embodiment, the cache memory is also included in the RAM 2020, the memory, and / or the storage device, unless otherwise indicated. To do.

Further, the CPU 2000 includes various operations, information processing, condition determination, information search / replacement, etc., which are specified by the instruction sequence of the program for the data read from the RAM 2020. Is processed and written back to RAM 2020. For example, when the CPU 2000 determines a condition, whether or not various variables shown in the present embodiment satisfy conditions such as large, small, above, below, and equal to other variables or constants. If the condition is satisfied (or not satisfied), it branches to a different instruction sequence or calls a subroutine.

In addition, the CPU 2000 can search for information stored in a file in the storage device, a database, or the like. For example, when a plurality of entries in which the attribute value of the second attribute is associated with the attribute value of the first attribute are stored in the storage device, the CPU 2000 describes the plurality of entries stored in the storage device. By searching for an entry in which the attribute value of the first attribute matches the specified condition and reading the attribute value of the second attribute stored in that entry, it is associated with the first attribute that satisfies the predetermined condition. The attribute value of the second attribute obtained can be obtained.

Further, when a plurality of elements are listed in the description of the embodiment, elements other than the listed elements may be used. For example, when it is described that "X executes Y using A, B and C", X may execute Y using D in addition to A, B and C.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that such modified or improved forms may also be included in the technical scope of the present invention.

The order of execution of operations, procedures, steps, steps, etc. in the devices, systems, programs, and methods shown in the claims, specification, and drawings is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. It's not a thing.

1 terminal device, 10 estimation device, 11 1st housing, 12 2nd housing, 21 infrared sensor, 22 acceleration sensor, 23 geomagnetic sensor, 24 positional relationship sensor, 30 user interface, 40 imaging unit, 50 imaging control unit, 60 Authentication unit, 301 Detection unit, 302 Estimator unit, 303 Control unit, 110 display, 111 display surface, 112 touch sensor, 120 keyboard, 1900 computer, 2000 CPU, 2010 ROM, 2020 RAM, 2030 communication interface, 2040 hard disk drive, 2050 Flash Memory Drive, 2060 DVD Drive, 2070 I / O Chip, 2075 Graphic Controller, 2080 Display, 2082 Host Controller, 2084 I / O Controller, 2090 Flash Memory, 2095 DVD

Claims (21)

Computer,
An estimation unit that estimates the spatial state using the first output value received from the first sensor, which is an infrared sensor provided in the terminal device having a display surface in the first housing , and
A detection unit that detects at least one of the movement and the posture of the terminal device by using the second output value received from the second sensor provided in the terminal device.
It functions as a control unit that controls the estimation of the spatial state by the estimation unit according to at least one of the movement and the posture of the terminal device detected by the detection unit .
Wherein, in response to said detected orientation of the display surface by the detection unit, a program to change the estimation algorithm by the estimation unit. The program according to claim 1, wherein the first sensor is provided in the first housing. The program according to claim 1 or 2, wherein the control unit selects one of a plurality of estimation algorithms having different presupposed spatial states according to the orientation of the display surface detected by the detection unit. Wherein the control unit is configured according to the orientation of the display surface when the movement of the terminal device has been completed, the program according to any one of claims 1 to 3 for changing the estimated algorithm by the estimation unit. When the orientation of the display surface when the movement of the terminal device is completed is obliquely upward, the control unit can use the estimation algorithm of the estimation unit to display the display surface when the movement of the terminal device is completed. The program according to any one of claims 1 to 4, which selects an estimation algorithm different from the case where the orientation is diagonally downward. When the display surface is obliquely upward when the movement of the terminal device is completed, the control unit presents a detection target when the movement of the terminal device is completed as an estimation algorithm by the estimation unit. The program according to claim 5 , wherein the first estimation algorithm is selected on the premise that is in the estimated state. In the estimation unit, when the first estimation algorithm is selected, the first output value is below the first threshold value, or the fluctuation amount of the first output value is below the second threshold value for a certain period of time. The program according to claim 6, wherein the absence of the detection target is estimated accordingly. The control unit is the estimation unit when the direction of the display surface is obliquely downward when the movement of the terminal device is completed and the upper end of the display surface is lower than the lower end of the display surface. According to claims 5 to 7, a second estimation algorithm is selected on the premise that the detection target exists or does not exist when the movement of the terminal device is completed. The program described in any one section. When the movement of the terminal device is completed, the control unit has the display surface oriented from sideways to downward, which is more lateral than the reference orientation, and the upper end of the display surface is the lower end of the display surface. When the height is higher than the above, it is assumed that the estimation algorithm by the estimation unit is in a state where it is unknown whether the detection target exists or does not exist when the movement of the terminal device ends. 2. The program according to any one of claims 5 to 8, which selects an estimation algorithm. The estimation unit responds to the fact that the first output value exceeds the third threshold value or the fluctuation amount of the first output value exceeds the fourth threshold value when the second estimation algorithm is selected. Estimate the existence of the detection target
Claim 8 or 9 for estimating the absence of the detection target according to the fact that the first output value is below the fifth threshold value or the amount of fluctuation of the first output value is below the sixth threshold value for a certain period of time. The program described in. When the movement of the terminal device is completed, the control unit has the display surface oriented downward from the sideways direction to the downward direction, and the upper end of the display surface is the lower end of the display surface. Claims 5 to select a third estimation algorithm that assumes that the detection target does not exist when the movement of the terminal device ends, as the estimation algorithm by the estimation unit when the height is higher than The program according to any one of 10. The estimation unit responds that the first output value exceeds the seventh threshold value or the fluctuation amount of the first output value exceeds the eighth threshold value when the third estimation algorithm is selected. The program according to claim 11, which estimates the existence of a detection target. The terminal device further includes a second housing having a keyboard, which is provided with a variable angle with respect to the first housing.
Wherein the control unit program claimed in any one of 12 to modify the estimate algorithm by the estimation unit in accordance with the angle of the second housing relative to the first body. Wherein, in response to said positional relationship of the display surface by vertical way of the first housing of the terminal device and the first sensor, of claims 1 to 13 to modify the estimate algorithm by the estimator The program described in any one section. The second sensor, the program according to any one of claims 1 1 4 is an acceleration sensor. The program according to claim 15 , wherein the detection unit further uses an output value received from at least one of the geomagnetic sensor and the gyro sensor to detect at least one of the movement and the posture of the terminal device. A computer-readable medium for storing the program according to any one of claims 1 to 16 . An estimation device for storing the program according to any one of claims 1 to 16 . An estimation unit that estimates the spatial state using the first output value received from the first sensor, which is an infrared sensor provided in the terminal device having a display surface in the first housing , and
A detection unit that detects at least one of the movement and the posture of the terminal device by using the second output value received from the second sensor provided in the terminal device.
A control unit that controls the estimation of the spatial state by the estimation unit according to at least one of the movement and the posture of the terminal device detected by the detection unit.
Equipped with a,
Wherein, in response to said detected orientation of the display surface by the detection unit, wherein to change the estimation algorithm by estimating portion estimating apparatus. With the estimation device according to claim 18 or 19 .
With the first housing
With the first sensor provided in the first housing,
With the second sensor
A terminal device comprising. An estimation stage for estimating the spatial state using the first output value received from the first sensor, which is an infrared sensor provided in the terminal device having a display surface in the first housing , and
A detection step of detecting at least one of the movement and the posture of the terminal device using the second output value received from the second sensor provided in the terminal device, and
A control step that controls the estimation of the spatial state by the estimation step according to at least one of the movement and the posture of the terminal device detected by the detection step.
Equipped with a,
Wherein in the control step, in response to the detected orientation of the display surface by the detection step, the estimation method to change the estimation algorithm according to the estimation step.

Images