JP7199207B2 - Air conditioner, air conditioning system, wind direction control method, and program - Google Patents

Description

The present invention relates to an air conditioner, an air conditioning system, a wind direction control method, and a program.

2. Description of the Related Art Conventionally, air conditioners for indoor air conditioning have been widely used in houses, buildings, and the like. For example, the user can operate the remote controller to instruct the air conditioner to perform indoor air conditioning such as cooling or heating.

Recently, a technique of operating an air conditioner by voice is also known. As a result, even a visually impaired user, a user who cannot operate a remote control during work, an elderly user who has difficulty in operating a complicated remote control, etc., can operate the air conditioner by emitting voice. can be done.

A prior art for operating an air conditioner by voice is disclosed in Patent Document 1, for example. Patent Document 1 discloses, for example, an invention of an air conditioner that can perform operations such as "operation start", "operation stop", "set temperature", and "strength of wind speed" by voice.

JP-A-2000-161746

However, the invention of Patent Document 1 only discloses that basic operations in an air conditioner such as the set temperature and wind speed can be instructed by voice, and does not disclose anything about more advanced wind direction settings. rice field. For example, in the air conditioner of Patent Literature 1, an advanced wind direction setting such as a wind blowing operation to control the air blowing to the person or a wind protection operation to control the air not blowing to the person is instructed by voice. couldn't.
Also, it is difficult to visually recognize the direction of the wind from the air conditioner. Therefore, it is difficult for not only visually impaired users and elderly users who have difficulty in complicated remote control operations but also most users to set the wind direction as intended.

The present invention has been made in order to solve such problems, and provides an air conditioner, an air conditioning system, a wind direction control method, and a program capable of instructing the user's intended wind direction setting by voice. intended to provide

In order to achieve the above object, the air conditioner according to the present invention includes:
a voice acquisition means for acquiring a voice uttered by a user present in the air-conditioned space;
air conditioning control means for controlling the wind direction of the conditioned air to be sent to the air conditioned space based on the voice;
voice recognition means for recognizing a target name uttered by the user from the voice;
a photographed image obtaining means for obtaining a photographed image of the air-conditioned space;
an image recognition means for recognizing a target corresponding to the target name from the captured image;
The air conditioning control means controls the wind direction toward the target.

In the air conditioner according to the present invention, the voice acquisition means acquires the voice uttered by the user present in the air-conditioned space. Then, the air conditioning control means controls the wind direction of the conditioned air sent to the air conditioned space based on the sound. For example, when the user speaks a target name, the air conditioning control means controls the wind direction toward the target existing in the air conditioned space.
As a result, the wind direction setting intended by the user can be instructed by voice.

BRIEF DESCRIPTION OF THE DRAWINGS The perspective view which shows an example of the external appearance of the air conditioner which concerns on Embodiment 1 of this invention. Block diagram showing an example of the main configuration of an air conditioner (a) is a graph showing an example of waveform characteristics, and (b) is a graph showing an example of distribution of characteristics. Schematic diagram showing an example of information stored in the target name storage unit Schematic diagram showing an example of information stored in an image storage unit Both (a) and (b) are schematic diagrams for explaining the registration of the target image. 3 is a flowchart for explaining an example of main processing of the air conditioner according to Embodiment 1 of the present invention; Flowchart for explaining an example of speech recognition processing in main processing Flowchart for explaining an example of image recognition processing in main processing Flowchart for explaining an example of air conditioning control processing in main processing Flowchart for explaining an example of learning management processing in main processing Schematic diagram showing an example of other information stored in an image storage unit A block diagram showing an example of the configuration of an air conditioning system according to Embodiment 2 of the present invention. Flowchart for explaining an example of main processing of the air conditioning system according to Embodiment 2 of the present invention Flowchart for explaining an example of image recognition processing in main processing Flowchart for explaining an example of air conditioning control processing in main processing A block diagram showing an example of the configuration of an air conditioner according to another embodiment of the present invention. A block diagram showing an example of the configuration of an air conditioning system according to another embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, in each drawing used in this specification, the same code|symbol shall be attached to a common element. Moreover, the present invention is not limited to the following embodiments, and various modifications can be made without departing from the scope of the present invention.

(Embodiment 1)
FIG. 1 is a perspective view showing an example of the appearance of an air conditioner 100 according to Embodiment 1 of the present invention.

As shown in the figure, the air conditioner 100 is provided with a suction port 101 for sucking indoor air in the upper portion (top surface). Inside the air conditioner 100, an air conditioning mechanism (not shown) (eg, a heat exchanger, a fan, a fan motor, etc.) is arranged. Such an air-conditioning mechanism generates hot or cold conditioned air from the air sucked from the suction port 101 .

In addition, an outlet 102 for blowing out the generated conditioned air into the room is provided at the lower portion of the air conditioner 100 . A vertical wind direction plate 103 and a left/right direction plate 104 are arranged at the outlet 102 .

The vertical wind direction plate 103 adjusts the wind direction of the conditioned air blown into the room from the outlet 102 in the vertical direction (vertical direction). In addition, the left/right wind direction plate 104 adjusts the wind direction of the conditioned air blown into the room from the air outlet 102 in the left/right direction (horizontal direction).
That is, the wind direction of the conditioned air sent out from the outlet 102 is adjusted by the vertical wind direction plate 103 and the left/right wind direction plate 104 .

Further, at the front center of the air conditioner 100, a microphone 10 for inputting indoor sounds and a camera 20 for capturing an indoor image are arranged. The arrangement positions of the microphone 10 and the camera 20 are not limited to the center of the front surface as described above, and may be arranged at positions where the sound in the room and the image in the room can be properly acquired. In addition, the cameras 20 may be arranged, for example, at intervals in the horizontal direction.

The configuration of the air conditioner 100 will be described below with reference to FIG. FIG. 2 is a block diagram showing an example of the main configuration of the air conditioner 100. As shown in FIG.

As shown, the air conditioner 100 includes a microphone 10 that is an example of voice acquisition means, a camera 20 that is an example of a captured image acquisition means, a voice recognition section 30 that is an example of voice recognition means, and a target name storage. a unit 31, an image recognition unit 40 as an example of image recognition means, an image storage unit 41, a learning management unit 50 as an example of learning management means, an air conditioning control unit 60 as an example of air conditioning control means, and a wind direction. A plate drive unit 70 is provided.
Note that the speech recognition unit 30, the image recognition unit 40, the learning management unit 50, and the air conditioning control unit 60 are realized by, for example, a computation processing unit having a CPU (Central Processing Unit) and memory. That is, the voice recognition unit 30, the image recognition unit 40, the learning management unit 50, and the air conditioning control unit 60 are realized by the CPU executing the programs stored in the memory.

The microphone 10 is arranged, for example, in the front center of the air conditioner 100 as described above, and acquires the sound in the room. That is, the microphone 10 acquires the voice uttered by the user present in the air-conditioned space. The microphone 10 supplies the acquired voice data to the voice recognition section 30 .

The camera 20 is arranged, for example, in the center of the front surface of the air conditioner 100 as described above, and acquires a photographed image of the interior of the room. That is, the camera 20 acquires a photographed image of the air-conditioned space. The camera 20 supplies the acquired photographing information to the image recognition section 40 .

The voice recognition unit 30 recognizes the target name uttered by the user from the voice acquired by the microphone 10 .
Specifically, the speech recognition unit 30 first extracts words from speech. For example, the speech recognition unit 30 divides the speech waveform into phonemes, and extracts the frequency features of the phonemes as shown in FIG. 3(a). Then, as shown in FIG. 3(b), the speech recognition unit 30 generates feature distributions (distributions A, B, C, . . . ) of sample phonemes measured in advance, extracted phonemes ) to identify the sample phoneme closest to the phoneme (more specifically, the distance to the center of gravity of the feature distribution). The speech recognition unit 30 performs such identification for each phoneme and connects the identified sample phonemes to extract a word.
Next, if a target name matching the extracted word can be retrieved from the target name storage unit 31 described below, the speech recognition unit 30 recognizes that the target name has been uttered by the user.

Returning to FIG. 2, the target name storage unit 31 stores a dictionary of target names to be subjected to speech recognition (name dictionary).
For example, the target name storage unit 31 stores a plurality of types of target names 31b as shown in FIG. Specifically, the target names 31b include the names of users who use the air-conditioned space air-conditioned by the air conditioner 100, namely “Taro”, “Shota”, and “Hanako”, and the names of objects existing in the air-conditioned space. There is a "sofa" and a "dining table". A name ID 31a for identifying the name is added to such a target name 31b.
As will be described later, the target name storage unit 31 can also store target names arbitrarily registered by the user.

Returning to FIG. 2, the image recognition unit 40 performs image recognition of the target corresponding to the target name recognized by the voice recognition unit 30 from the image captured by the camera 20 .
Specifically, first, the image recognition unit 40 reads the target image associated with the target name whose voice is recognized by the voice recognition unit 30 from the image storage unit 41 described below.
Next, if the image recognition unit 40 can search for an image that matches the read target image from among the captured images acquired by the camera 20, the image recognition unit 40 recognizes that image as the target image. The image search is performed by pattern recognition (pattern matching), template matching, or the like, for example.

The image storage unit 41 stores a dictionary of target images to be subjected to image recognition (that is, an image dictionary).
For example, the image storage unit 41 stores a plurality of types of target images 41b as shown in FIG. An image ID 41a for identifying the image is added to the target image 41b. Furthermore, in the target image 41b, the same ID as any of the name IDs 31a in the target name storage section 31 is added to the target name ID 41c. That is, the target images stored in the image storage section 41 are associated with the target names stored in the target name storage section 31 .
Further, as will be described later, the image storage unit 41 can also store a target image arbitrarily registered by the user.

The learning management unit 50 appropriately updates the target name stored in the target name storage unit 31 and the target image stored in the image storage unit 41 .
For example, when data is updated in a management server that officially manages target names and target images, the learning management unit 50 downloads the updated data from the management server, stores the target name storage unit 31, and The image storage unit 41 is updated.
In addition, when the user requests manual registration, the learning management unit 50 stores the uttered target name in the target name storage unit 31, and stores the target image to be associated with the target name. Stored in unit 41 .
For example, when manual registration of a target name is requested, the learning management unit 50 accepts the voice uttered by the user from the microphone 10 (for example, “cushion”), and uses the word recognized by the voice recognition unit 30 as a new word. is stored in the target name storage unit 31 as a unique target name.
Subsequently, the learning management unit 50 acquires a target image to be associated with the registered target name by one of the two methods described below, and stores it in the image storage unit 41 .

A first method is, for example, a method of transmitting captured images from the air conditioner 100 to a user terminal such as a smartphone or a tablet, and having the user select a target image from the captured images. Specifically, as shown in FIG. 6A, the learning management unit 50 transmits the captured image IM captured by the camera 20 to the user terminal UT for display. At that time, the learning management section 50 may transmit to the user terminal UT a photographed image IM in which the area in which the image recognition section 40 has recognized the object is surrounded by the selection cursors SL (SL1 to SL3). Then, when the user selects an arbitrary area in the captured image IM by encircling it with a finger or by tapping any selection cursor SL (selection cursor SL1 as an example), the learning management unit 50 stores the selected target image in the image storage unit 41 .
The learning management unit 50 can communicate with the user terminal UT using wireless communication such as BLE (Bluetooth (registered trademark) Low Energy) and Wi-Fi.

A second method is to take a picture of the target with the user terminal and have the user send the picture to the air conditioner 100 .
Specifically, the user uses the camera function of the user terminal UT to take a picture PT including the target OB (a cushion as an example), as shown in FIG. 6B. Then, the photograph PT is transmitted to the learning management section 50 from the user terminal UT. Upon receiving the photo PT, the learning management unit 50 stores it in the image storage unit 41 as a target image after performing background deletion, size conversion, and the like.

Returning to FIG. 2, the air conditioning control unit 60 controls the direction of the conditioned air sent out from the air outlet 102 based on the user's voice. The air conditioning control unit 60 appropriately drives the wind direction plate driving unit 70 to control the up/down direction plate 103 and the left/right direction plate 104 respectively, thereby controlling the wind direction of the conditioned air sent out from the outlet 102. .
For example, the air conditioning control unit 60 causes the voice recognition unit 30 to recognize the target name uttered by the user, and the image recognition unit 40 recognizes the target image associated with the target name from the captured image. Then, the wind direction is controlled so that the conditioned air is sent out toward the target existing in the air-conditioned space (in the case of wind directing operation). In the case of the windbreak operation, the wind direction is controlled so that the conditioned air is sent with a deviation of, for example, ±30 degrees from the target existing in the air-conditioned space.
Also, in the case of wind blowing operation, when the size of the target (for example, the area) is larger than the reference value, the air conditioning control unit 60 adjusts the target so that the conditioned air is delivered to the entire target. , perform a swing operation in which the wind direction moves back and forth within a specified range.
In addition, the air conditioning control unit 60 adjusts the temperature of the conditioned air sent from the air outlet 102 according to the temperature setting for cooling operation, heating operation, etc. Adjust the air volume of the conditioned air.

The wind direction plate driving unit 70 is, for example, an actuator, a motor, or the like, and drives the up/down direction plate 103 and the left/right direction plate 104 .

The operation of the air conditioner 100 having such a configuration will be described below with reference to FIGS. 7 to 11. FIG. FIG. 7 is a flowchart for explaining an example of main processing of the air conditioner 100 according to Embodiment 1 of the present invention. FIG. 8 is a flowchart for explaining an example of speech recognition processing in the main processing. Also, FIG. 9 is a flowchart for explaining an example of image recognition processing in the main processing. Also, FIG. 10 is a flowchart for explaining an example of the air conditioning control process in the main process. FIG. 11 is a flowchart for explaining an example of learning management processing in the main processing.

First, the main processing performed by the air conditioner 100 will be described with reference to FIG.

First, the air conditioner 100 determines whether or not the current mode is the management mode (step S100). Note that the management mode is a mode in which information stored in the target name storage unit 31 and the image storage unit 41 is updated as appropriate.

When the air conditioner 100 determines that it is not in the management mode (step S100; No), it determines whether or not a wake word has been detected (step S200). A wake word is a word that serves as a trigger for wind direction setting, and includes, for example, "wind protection" and "wind protection".
That is, the speech recognition unit 30 determines whether such a wake word has been input from the microphone 10 or not.

When the air conditioner 100 determines that no wake word has been detected (step S200; No), it waits for the wake word to be input.

On the other hand, when determining that the wake word has been detected (step S200; Yes), the air conditioner 100 executes voice recognition processing for detecting the target name (step S300). Details of the speech recognition process will be described later.

The air conditioner 100 determines whether or not the target name has been detected by voice recognition processing (step S400).
When determining that the target name could not be detected (step S400; No), the air conditioner 100 returns the process to step S200 described above.

On the other hand, when it is determined that the target name has been detected (step S400; Yes), the air conditioner 100 executes image recognition processing for detecting the target existing in the air-conditioned space (step S500). Details of the image recognition processing will be described later.

The air conditioner 100 determines whether or not the target has been detected by image recognition processing (step S600).
When determining that the target could not be detected (step S600; No), the air conditioner 100 returns the process to step S200 described above.

On the other hand, when it is determined that the target has been detected (step S600; Yes), the air conditioner 100 calculates the size of the target (step S700).
For example, the air conditioner 100 uses the height from the floor to the ceiling, the width and height of windows, and the width and height of doors in the photographed image as references, and considers the depth between the windows, doors, etc. and the target. The size of the target is calculated from the positional relationship.
In addition, when using two cameras 20 arranged with an interval in the horizontal direction, the same photographed image is photographed from each camera 20, and the parallax of the two photographed images and the distance between the cameras 20 are used. The size of the target may be calculated by measuring the distance to the target by trigonometry.

Then, the air conditioner 100 executes air conditioning control processing for controlling the wind direction (step S800). Details of the air conditioning control process will be described later.

Further, in step S100 described above, when it is determined that the management mode is set (step S100; Yes), the air conditioner 100 performs learning management for updating the target name storage unit 31 and the image storage unit 41. Processing is executed (step S900). Details of the learning management process will be described later.

Next, with reference to FIG. 8, the details of the speech recognition processing executed in step S300 of the main processing described above will be described.

First, the air conditioner 100 acquires the voice input by the microphone 10 (step S301).

The air conditioner 100 recognizes words from the acquired voice (step S302). For example, the speech recognition unit 30 divides the speech waveform into phonemes, and extracts the frequency features of the phonemes as shown in FIG. 3A. Then, as shown in FIG. 3B, the speech recognition unit 30 compares the feature distribution of sample phonemes measured in advance with the frequency features of the extracted phonemes, and identifies the sample phoneme closest to the phoneme. do. The speech recognition unit 30 performs such identification for each phoneme and connects the identified sample phonemes to recognize a word.

The air conditioner 100 compares the recognized word with the registered n-th target name (step S303). For example, the speech recognition unit 30 compares the word recognized in step S302 with the n-th target name in the target name 31b of the target name storage unit 31 as shown in FIG. Note that n is a natural number starting from 1.

The air conditioner 100 determines whether or not the recognized word matches the n-th target name (step S304).
When the air conditioner 100 determines that there is a match (step S304; Yes), it returns the name ID of the matched target name (step S305). That is, the air conditioner 100 sets the name ID to the return value, and returns control to the main processing described above.

On the other hand, if it is determined that they do not match (step S304; No), the air conditioner 100 determines whether or not n is smaller than the number of registrations (the total number of target names stored in the target name storage unit 31) (step S306). ). That is, the air conditioner 100 determines whether or not the target name that can be compared remains in the target name storage unit 31 .

When determining that n is smaller than the number of registrations (step S306; Yes), the air conditioner 100 adds 1 to n (step S307) and returns the process to step S303 described above.

On the other hand, when it is determined that n is not smaller than the number of registrations (n is equal to or greater than the number of registrations) (step S306; No), the air conditioner 100 returns no detection (step S308). That is, the air conditioner 100 sets the return value to indicate that the target name is not detected, and returns control to the main processing described above.

Next, with reference to FIG. 9, details of the image recognition processing executed in step S500 of the main processing described above will be described.

First, the air conditioner 100 acquires a captured image captured by the camera 20 (step S501).

The air conditioner 100 reads the target image associated with the returned name ID (step S502).
That is, the image recognition unit 40 reads from the image storage unit 41 the target image associated with the name ID returned in step S305 of the voice recognition process described above.

The air conditioner 100 compares the target image and the image of the search window (search window region) in the captured image by pattern recognition (step S503). That is, the image recognition unit 40 compares the target image read from the image storage unit 41 and the image of the current search window in the captured image by pattern recognition. This search window is an area for scanning while shifting the position from the start point to the end point in the photographed image. In addition to pattern recognition, the image recognition unit 40 may compare the target image and the image of the current search window by template matching.

The air conditioner 100 determines whether or not the target image matches the current search window image (step S504).
When the air conditioner 100 determines that there is a match (step S504; Yes), it returns the position of the search window (step S505). That is, the air conditioner 100 sets the position of the target image in the captured image as the return value, and returns control to the main processing described above.

On the other hand, when determining that they do not match (step S504; No), the air conditioner 100 determines whether or not the search window has reached the end (step S506). In other words, the air conditioner 100 determines whether or not the search has been completed for the entire captured image.

When determining that the search window has not reached the end (step S506; No), the air conditioner 100 moves the search window (step S507) and returns the process to step S503 described above.

On the other hand, when it is determined that the search window has reached the end (step S506; Yes), the air conditioner 100 returns no detection (step S508). That is, the air conditioner 100 sets the non-detection of the target image as the return value, and returns the control to the main processing described above.

Next, with reference to FIG. 10, details of the air conditioning control process executed in step S800 of the main process described above will be described.

First, the air conditioner 100 calculates the target direction toward the target existing in the air-conditioned space from the position of the returned detection window (step S801).
For example, the air-conditioning control unit 60 determines the target direction toward the target existing in the air-conditioned space (for example, air A target direction from the air outlet 102 of the harmony machine 100 to the target) is calculated.

The air conditioner 100 determines whether or not it is in wind blowing operation (step S802). That is, air conditioner 100 determines whether or not the wake word detected in step S200 of the main processing described above was "wind blow".

When the air conditioner 100 determines that it is not in wind blowing operation (wind shielding operation) (step S802; No), it sets the wind direction by, for example, ±30 degrees from the target direction (step S803).
For example, the air-conditioning control unit 60 sets the wind direction by shifting the direction of the target by 30 degrees to the left so that the conditioned air does not hit the target. Note that when the wind direction exceeds the set limit angle, the air-conditioning control unit 60 sets the wind direction shifted rightward by 30 degrees from the target direction.
Note that the shift angle is not limited to 30 degrees to the left and right as described above, and may be another angle. Further, the shifting direction is not limited to left and right, but may be shifted up and down.
Then, the air conditioner 100 returns control to the main processing described above.

On the other hand, when it is determined that the wind blowing operation is being performed (step S802; Yes), the air conditioner 100 determines whether or not the size of the target is larger than the reference value (step S804).
That is, air conditioner 100 determines whether the size of the target calculated in step S700 of the main processing described above is larger than the reference value. Note that the reference value is a threshold at which the conditioned air reaches all the people who use the target when the wind direction is fixed. For example, if the target is a dining table, if multiple people use the dining table (when people sit on both sides of the dining table), if the air direction is fixed, conditioned air cannot be distributed to everyone. . Therefore, a reference value is determined in advance as a threshold between a target that can be used by a plurality of people and a target that can be used by a single person. Note that the user may arbitrarily set the reference value. In this case, the user may designate a target to be set as a threshold value and have the air conditioner 100 measure the size of the target as the reference value, or the user may directly input the reference value.

When the air conditioner 100 determines that the size of the target is not larger than the reference value (the size of the target is equal to or smaller than the reference value) (step S804; No), it sets the wind direction in the direction of the target (step S805). .
That is, the air-conditioning control unit 60 controls the wind direction so that the conditioned air is sent toward the target existing in the air-conditioned space.
Then, the air conditioner 100 returns control to the main processing described above.

On the other hand, when it is determined that the size of the target is larger than the reference value (step S804; Yes), the air conditioner 100 sets the swing so as to swing within a specified range centered on the direction of the target (step S806). .
That is, the air-conditioning control unit 60 performs a swing operation in which the wind direction moves back and forth within a specified range centering on the direction of the target so that the conditioned air hits the entire target.
Then, the air conditioner 100 returns control to the main processing described above.

Next, with reference to FIG. 11, the details of the learning management process executed in step S900 of the main process described above will be described.

First, the air conditioner 100 determines whether or not the target is registered by the user (step S901).

When the air conditioner 100 determines that the target is registered by the user (step S901; Yes), the air conditioner 100 recognizes the voice input by the microphone 10 and registers the recognized word as the target name (step S902).
For example, the learning management unit 50 accepts the voice uttered by the user from the microphone 10, and stores a word voice-recognized by the voice recognition unit 30 in the target name storage unit 31 as a new target name.

The air conditioner 100 provides the captured image captured by the camera 20 to the user terminal (step S903).
For example, the learning management unit 50 transmits the captured image IM captured by the camera 20 to the user terminal UT for display, as shown in FIG. 6A described above. At that time, the learning management section 50 may transmit to the user terminal UT a photographed image IM in which the area in which the image recognition section 40 has recognized the object is surrounded by the selection cursors SL (SL1 to SL3).

The air conditioner 100 registers the image selected by the user from among the captured images as the target image (step S904).
For example, as shown in FIG. 6A, when the user designates an arbitrary image in the captured image IM by surrounding it with a finger or by tapping any of the selection cursors SL to select the image, learning The management unit 50 stores the image in the image storage unit 41 as a target image. At this time, the learning management unit 50 associates the target name stored in the target name storage unit 31 in step S902 with the target image.
Then, the air conditioner 100 returns control to the main processing described above.

Further, when it is determined in step S901 that the target is not registered by the user (step S901; No), the air conditioner 100 determines whether or not there is an update (step S905).
For example, the learning management unit 50 determines whether or not the target name and target image have been updated in the official management server.

When the air conditioner 100 determines that there is no update (step S905; No), it ends the learning management process as it is, and returns control to the main process described above.
On the other hand, when it is determined that there is an update (step S905; Yes), the air conditioner 100 updates the target name storage unit 31 (step S906) and updates the image storage unit 41 (step S906). S907).
That is, the learning management unit 50 downloads updated data from the management server, and updates the target name storage unit 31 and the image storage unit 41, respectively.
Then, the air conditioner 100 returns control to the main processing described above.

In this way, by each process executed by the air conditioner 100, the user can control the direction of the wind toward the target existing in the air-conditioned space simply by speaking the name of the target (in the case of wind blowing operation). In the case of windbreak operation, the wind direction can be controlled by shifting from the target.
As a result, even a visually impaired user, a user who cannot operate a remote control during work, an elderly user who has difficulty in operating a complicated remote control, etc., can intuitively set the wind direction.
In addition, although it is difficult to visually recognize the direction of the airflow from the air conditioner 100, most users, not only visually impaired users and elderly users who have difficulty in performing complicated remote control operations, can make fine adjustments. The wind direction can be pinpointed without need.

As a result, the wind direction setting intended by the user can be instructed by voice.

In the first embodiment described above, the information shown in FIG. 5 is stored in the image storage unit 41. However, the information shown in FIG. You may make it

As shown in FIG. 12, position information 41d and attribute 41e are added to the target image 41b.
The position information 41d is, for example, the most recent position information of the target. For example, when the image recognition unit 40 recognizes the object as a target, the position information 41d is updated. Note that the position information 41d may be updated by searching for a target corresponding to each target image when the air conditioner 100 is operated.

Also, the attribute 41e is the attribute of the target. For example, "person" is an attribute that is highly likely to move in an air-conditioned space. Also, "heavy furniture" is an attribute that has a very low possibility of being moved (changed in location) in an air-conditioned space. "Light furniture" is an attribute with a medium possibility of being moved in an air-conditioned space.

By storing the information as shown in FIG. 12 in the image storage unit 41, for example, even if the image recognition unit 40 cannot recognize the target because it overlaps with the user, there is a possibility that the target will not be moved from the attribute 41e. When it is high, the position information 41d may be used by the air conditioning control unit 60 to control the direction of the wind toward the target (in the case of wind blowing operation).
Specifically, when a large number of people are sitting on a sofa and the user utters "Wind, sofa", the image recognition unit 40 may not be able to recognize the sofa. Even in such a case, the attribute 41e of the sofa is "heavy furniture" and the possibility of it being moved is extremely low. Control the wind direction.

Even in this case, the wind direction setting intended by the user can be instructed by voice.

Further, in the first embodiment, the case of using the camera 20 has been described, but a thermal imaging camera as described below may be used.

(Embodiment 2)
In the first embodiment described above, the case where the air conditioner 100 includes the microphone 10 and the voice recognition unit 30 has been described, but a commercially available smart speaker (AI speaker) may be used.
Hereinafter, an air conditioning system 1 according to Embodiment 2 of the present invention will be described with reference to the drawings.

FIG. 13 is a block diagram showing the configuration of the air conditioning system 1 according to Embodiment 2 of the present invention.
As illustrated, the air conditioning system 1 includes a smart speaker 200 and an air conditioner 100 .

The smart speaker 200 includes a microphone 10 that is an example of voice acquisition means, a voice recognition section 30 that is an example of voice recognition means, and a target name storage section 31 .
The configuration of the microphone 10, the voice recognition unit 30, and the target name storage unit 31 is the same as that of the air conditioner 100 according to the first embodiment described above.

The air conditioner 100 includes a control management unit 80, a thermal image camera 90 as an example of thermal image acquisition means, an image recognition unit 40 as an example of image recognition means, an image storage unit 41, and an example of learning management means. , an air-conditioning control unit 65 as an example of air-conditioning control means, and a wind direction plate driving unit 70 .
The image recognition unit 40, the image storage unit 41, the learning management unit 50, and the wind direction plate driving unit 70 have the same configuration as the air conditioner 100 according to the first embodiment described above.

The control management unit 80 acquires the target name recognized by the voice recognition unit 30 of the smart speaker 200 .

A thermal image camera 90 acquires a thermal image showing the temperature distribution of the air-conditioned space. For example, the thermal image camera 90 has an array of infrared light receiving elements movably supported by a movable (rotating) mechanism, and scans a scanning range from the ceiling to the floor in the room to obtain a thermal image in an air-conditioned space. to detect

The air conditioning control unit 65 causes the smart speaker 200 (voice recognition unit 30) to perform voice recognition of the target name uttered by the user, and the image recognition unit 40 converts the target image associated with the target name into a thermal image. When recognized from within, it controls the direction of the wind so that conditioned air is delivered towards a target existing in the conditioned space.
At that time, if the cooling operation is in progress, the air conditioning control unit 65 controls the wind direction so that the conditioned air is sent toward the hottest point in the thermal image of the target. For example, if the target is a person, the wind direction is controlled so that cool air is blown toward the person's head, face, or the like. Conversely, if heating is in progress, the air-conditioning control unit 65 controls the wind direction so that the conditioned air is sent toward the lowest temperature point in the thermal image of the target. For example, if the target is a person, the wind direction is controlled so that warm air is sent toward the person's feet, hands, or the like.

The operation of the air conditioning system 1 having such a configuration will be described below with reference to FIGS. 14 to 16. FIG. FIG. 14 is a flowchart for explaining an example of main processing of the air conditioning system 1 according to Embodiment 2 of the present invention. Also, FIG. 15 is a flowchart for explaining an example of image recognition processing in the main processing. FIG. 16 is a flowchart for explaining an example of air conditioning control processing in the main processing.
Note that the voice recognition processing and the learning management processing in the main processing are the same as the processing of the air conditioner 100 according to the first embodiment described above.

First, main processing in the air conditioning system 1 will be described with reference to FIG. 14 .

First, the air conditioner 100 determines whether the current mode is the management mode (step S100), and if it determines that it is not the management mode (step S100; No), the smart speaker 200 detects the wake word. It is determined whether or not (step S200).

When the smart speaker 200 determines that the wake word is not detected (step S200; No), it waits for the wake word to be input, and when it determines that the wake word is detected (step S200; ), and performs voice recognition processing for detecting the target name (step S300).

The air conditioner 100 determines whether or not the target name could be detected by the smart speaker 200 (step S400), and if it determines that the target name could not be detected (step S400; No), the process proceeds to step S200 described above. On the other hand, if it is determined that the target name has been detected (step S400; Yes), image recognition processing for detecting the target existing in the air-conditioned space is executed (step S550). Details of the image recognition processing will be described later.

The air conditioner 100 determines whether or not the target has been detected by image recognition processing (step S600), and if it determines that the target has not been detected (step S600; No), returns the processing to step S200 described above.

On the other hand, when it is determined that the target has been detected (step S600; Yes), the air conditioner 100 determines whether or not it is in cooling operation (step S750).

When determining that the air conditioner 100 is in the cooling operation (step S750; Yes), the air conditioner 100 specifies the hottest point on the target (step S760).
For example, if the target is a person, the air conditioner 100 identifies from the thermal image that the person's head, face, etc. have the highest temperature.

On the other hand, when it is determined that the cooling operation is not being performed (the heating operation is being performed) (step S750; No), the air conditioner 100 identifies the lowest temperature point in the target (step S770).
For example, if the target is a person, the air conditioner 100 identifies from the thermal image that the feet and hands of the person have the lowest temperature.

Then, the air conditioner 100 executes air conditioning control processing for controlling the wind direction (step S850). Details of the air conditioning control process will be described later.

Further, in step S100 described above, when it is determined that the management mode is set (step S100; Yes), the air conditioner 100 performs learning management for updating the target name storage unit 31 and the image storage unit 41. Processing is executed (step S900).

Next, with reference to FIG. 15, details of the image recognition processing executed in step S550 of the main processing described above will be described.

First, the air conditioner 100 acquires a thermal image detected by the thermal image camera 90 (step S551).

The air conditioner 100 reads the target image associated with the returned name ID (step S502).

The air conditioner 100 compares the target image and the image of the search window in the thermal image by pattern recognition (step S553). That is, the image recognition unit 40 compares the target image read from the image storage unit 41 and the image of the current search window in the thermal image by pattern recognition. In addition to pattern recognition, the image recognition unit 40 may compare the target image and the image of the current search window by template matching.

The air conditioner 100 determines whether or not the target image matches the current search window image (step S504), and if it determines that they match (step S504; Yes), returns the position of the search window. (Step S505). That is, the air conditioner 100 sets the position of the target image in the thermal image as the return value, and returns control to the main processing described above.

On the other hand, if it is determined that they do not match (step S504; No), the air conditioner 100 determines whether the search window has reached the end (step S506), and if it determines that the search window has not reached the end ( Step S506; No), the search window is moved (step S507), and the process returns to step S553 described above.

On the other hand, when it is determined that the search window has reached the end (step S506; Yes), the air conditioner 100 returns no detection (step S508). That is, the air conditioner 100 sets the non-detection of the target image as the return value, and returns the control to the main processing described above.

Next, with reference to FIG. 16, details of the air conditioning control process executed in step S850 of the main process described above will be described.

First, the air conditioner 100 calculates a pinpoint direction toward the identified target location (step S851).
For example, the air-conditioning control unit 60 obtains the position of the target existing in the air-conditioned space from the position of the returned detection window, and furthermore, the pinpoint direction ( As an example, a pinpoint direction from the outlet 102 of the air conditioner 100 to a specific location) is calculated.

The air conditioner 100 determines whether or not it is in wind blowing operation (step S802), and if it determines that it is not wind blowing operation (wind shielding operation) (step S802; No), it moves from the pinpoint direction, for example, ± The wind direction is set by shifting by 30 degrees (step S853).
Then, the air conditioner 100 returns control to the main processing described above.

On the other hand, if it is determined that the wind blowing operation is being performed (step S802; Yes), the air conditioner 100 sets the wind direction to the pinpoint direction (step S855).
Then, the air conditioner 100 returns control to the main processing described above.

In this way, even with the air conditioning system 1 that uses the commercially available smart speaker 200, the user can control the direction of the wind toward the target in the air-conditioned space simply by speaking the name of the target. in the case of blind driving). In addition, by using the thermal imaging camera 90, the wind direction can be directed toward the hottest point on the target during cooling operation, and conversely toward the coldest point on the target during heating operation. can be controlled (in the case of wind blowing operation). In the case of windbreak operation, the wind direction can be controlled by shifting from the target.
As a result, even a visually impaired user, a user who cannot operate a remote control during work, an elderly user who has difficulty in operating a complicated remote control, etc., can intuitively set the wind direction.
In addition, although it is difficult to visually recognize the direction of the airflow from the air conditioner 100, most users, not only visually impaired users and elderly users who have difficulty in performing complicated remote control operations, can make fine adjustments. The wind direction can be pinpointed without need.

As a result, the wind direction setting intended by the user can be instructed by voice.

(Other embodiments)
In the first and second embodiments described above, the case where the image recognition unit 40 performs image recognition to specify the target position (target direction) has been described. By omitting it, the position of the target may be specified only from the voice.
Hereinafter, an air conditioner 100 according to another embodiment of the present invention will be described with reference to the drawings.

FIG. 17 is a block diagram showing the configuration of an air conditioner 100 according to another embodiment of the invention.
As illustrated, the air conditioner 100 includes an array microphone 15 as an example of voice acquisition means, a voice recognition section 30 as an example of voice recognition means, a target name storage section 31, and an example of air conditioning control means. An air conditioning control unit 66 and a wind direction plate driving unit 70 are provided.
Note that the voice recognition unit 30, the target name storage unit 31, and the wind direction plate driving unit 70 have the same configurations as those of the air conditioner 100 according to the first embodiment described above.

The array microphone 15 has a plurality of microphone elements arranged therein, and not only acquires sound but also detects the direction of the sound.

When the target name uttered by the user is recognized by the voice recognition unit 30, the air conditioning control unit 66 adjusts the wind direction so that conditioned air is sent out in the direction of the voice detected by the array microphone 15. Control.
At that time, if the speech recognition unit 30 continues to recognize a specific keyword (for example, “swing driving”), the air conditioning control unit 66 detects the direction of the voice detected by the array microphone 15 as the center, and the specified range You may perform the swing operation which makes a wind direction come and go with .

Even with the air conditioner 100 having such a configuration, the wind direction setting intended by the user can be instructed by voice.

In addition, in the first and second embodiments, etc., the case where the air conditioner 100 side has all the configurations has been described, but after having an external server have a part of the configuration, it is possible to cooperate with the external server. good too.
Hereinafter, an air conditioning system 1 according to another embodiment of the present invention will be described with reference to the drawings.

FIG. 18 is a block diagram showing the configuration of an air conditioning system 1 according to another embodiment of the invention.
As illustrated, the air conditioning system 1 includes an air conditioner 100 and a cloud server 300 . Note that the air conditioner 100 and the cloud server 300 are connected via the Internet N so as to be communicable.

The air conditioner 100 includes a microphone 10 that is an example of sound acquisition means, a camera 20 that is an example of a captured image acquisition means, a communication section 95, an air conditioning control section 67 that is an example of air conditioning control means, and a wind direction plate drive. a portion 70;
The configuration of the microphone 10, the camera 20, and the wind direction plate driving section 70 is the same as that of the air conditioner 100 according to the first embodiment described above.

The communication unit 95 communicates with the cloud server 300 via the Internet N. FIG.
For example, the communication unit 95 transmits the voice input by the microphone 10 and the captured image captured by the camera 20 to the cloud server 300 . Also, the communication unit 95 receives the position of the target sent from the cloud server 300 .

When the position of the target is sent from the cloud server 300, the air conditioning control unit 67 controls the wind direction so that the conditioned air is sent toward the target existing in the air conditioned space (in the case of wind blowing operation). In the case of the windbreak operation, the wind direction is controlled so that the conditioned air is sent with a deviation of, for example, ±30 degrees from the target existing in the air-conditioned space.

The cloud server 300 also includes a receiving unit 301, a speech recognition unit 30 as an example of speech recognition means, a target name storage unit 31, an image recognition unit 40 as an example of image recognition means, and an image storage unit 41. , a learning management unit 50 which is an example of learning management means, and a transmission unit 302 .
Note that the voice recognition unit 30, the target name storage unit 31, the image recognition unit 40, the image storage unit 41, and the learning management unit 50 have the same configuration as the air conditioner 100 according to the first embodiment described above.

The receiving unit 301 receives audio and captured images sent from the air conditioner 100 .
Then, the transmission unit 302 transmits the position of the target recognized by the image recognition unit 40 to the air conditioner 100 .

Even with the air conditioning system 1 having such a configuration, it is possible to instruct the wind direction setting intended by the user by voice.

Further, in the first and second embodiments described above, for example, the program executed in the air conditioner 100 is a CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), MO (Magneto-Optical Disc) , a USB memory, a memory card, or other computer-readable recording medium for distribution. By installing such a program in a specific or general-purpose computer, the computer can be made to function as the air conditioner 100 in the above embodiment.

Alternatively, the above program may be stored in a disk device of a server device on a communication network such as the Internet, superimposed on a carrier wave, and downloaded to a computer. The above processing can also be achieved by starting and executing the program while transferring it via a communication network. Furthermore, the above processing can also be achieved by causing all or part of a program to be executed on a server device and executing the program while the computer transmits and receives information regarding the processing via a communication network.

If the above functions are shared by the OS (Operating System), or if the OS and applications work together, only the parts other than the OS are stored in the above recording medium and distributed. You can also download it to your computer.

The present invention is capable of various embodiments and modifications without departing from its broader spirit and scope. Moreover, the above-described embodiments are for explaining the present invention, and do not limit the scope of the present invention. In other words, the scope of the present invention is indicated by the claims rather than the embodiments. Various modifications made within the scope of the claims and within the meaning of equivalent inventions are considered to be within the scope of the present invention.

1 air conditioning system 10 microphone 15 array microphone 20 camera 30 voice recognition unit 31 target name storage unit 40 image recognition unit 41 image storage unit 50 learning management unit 60, 65, 66, 67 air conditioning control Unit 70 Wind direction plate drive unit 80 Control management unit 90 Thermal image camera 95 Communication unit 100 Air conditioner 101 Suction port 102 Outlet 103 Up/down direction plate 104 Left/right direction plate 200 Smart speaker 300 cloud server, 301 receiving unit, 302 transmitting unit

Claims (10)

a voice acquisition means for acquiring a voice uttered by a user present in the air-conditioned space;
air conditioning control means for controlling the wind direction of the conditioned air to be sent to the air conditioned space based on the voice;
voice recognition means for recognizing a target name uttered by the user from the voice;
a photographed image obtaining means for obtaining a photographed image of the air-conditioned space;
an image recognition means for recognizing a target corresponding to the target name from the captured image;
The air conditioning control means controls the wind direction toward the target.
Air conditioner. The air conditioning control means swings the wind direction within a specified range centered on the target when the size of the target exceeds a reference value.
The air conditioner according to claim 1 . a voice acquisition means for acquiring a voice uttered by a user present in the air-conditioned space;
air conditioning control means for controlling the wind direction of the conditioned air to be sent to the air conditioned space based on the voice;
voice recognition means for recognizing a target name uttered by the user from the voice;
a photographed image obtaining means for obtaining a photographed image of the air-conditioned space;
an image recognition means for recognizing a target corresponding to the target name from the captured image ;
The air conditioning control means controls the wind direction by shifting it from the target during windbreak operation.
Air conditioner. a voice acquisition means for acquiring a voice uttered by a user present in the air-conditioned space;
air conditioning control means for controlling the wind direction of the conditioned air to be sent to the air conditioned space based on the voice;
voice recognition means for recognizing a target name uttered by the user from the voice;
a thermal image acquiring means for acquiring a thermal image showing the temperature distribution of the air-conditioned space;
an image recognition means for recognizing a target corresponding to the target name from the thermal image ;
The air conditioning control means controls the wind direction toward the target.
Air conditioner. The air conditioning control means controls the direction of the wind toward the hottest point on the target during cooling operation, and toward the coldest point on the target during heating operation.
The air conditioner according to claim 4 . further comprising learning management means for updating a name dictionary used for recognition by the speech recognition means and an image dictionary used for recognition by the image recognition means,
The air conditioner according to any one of claims 1 to 5 . An air conditioning system including a smart speaker and an air conditioner,
The smart speaker
a voice acquisition means for acquiring a voice uttered by a user present in the air-conditioned space;
voice recognition means for recognizing the target name uttered by the user from the voice,
The air conditioning system is
an image acquisition means for acquiring a photographed image of the air-conditioned space or a thermal image showing the temperature distribution of the air-conditioned space;
image recognition means for recognizing the target corresponding to the target name from the captured image or the thermal image;
air conditioning control means for controlling the wind direction of the conditioned air sent to the air conditioned space toward the target or shifted from the target ;
air conditioning system. a voice acquisition means for acquiring a voice uttered by a user present in the air-conditioned space;
air conditioning control means for controlling the wind direction of the conditioned air to be sent to the air conditioned space based on the voice;
voice recognition means for recognizing a target name uttered by the user from the voice;
an image acquisition means for acquiring a photographed image of the air-conditioned space or a thermal image showing the temperature distribution of the air-conditioned space;
an image recognition means for recognizing the target corresponding to the target name from the captured image or the thermal image;
The air conditioning control means controls the direction of the wind toward the target or away from the target.
air conditioning system. an audio acquisition step for acquiring audio input by a microphone;
a voice recognition step of recognizing a target name uttered by a user present in the air-conditioned space from the voice;
an image acquisition step of acquiring a photographed image of the air-conditioned space or a thermal image showing the temperature distribution of the air-conditioned space;
an image recognition step of recognizing a target corresponding to the target name from the captured image or the thermal image;
an air conditioning control step of controlling the wind direction of the conditioned air sent to the air conditioned space toward the target or shifted from the target ;
A wind direction control method comprising: the computer,
a speech recognition unit that recognizes the target name uttered by the user in the air-conditioned space from the voice input by the microphone;
an image recognition unit that recognizes a target corresponding to the target name from a captured image captured by a camera or a thermal image captured by a thermal imaging camera;
an air conditioning control unit that controls the wind direction of the conditioned air sent to the air conditioned space toward the target or shifted from the target ;
A program that acts as a

Images