JP7481174B2 - Display device - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies. Venue: Secom Co., Ltd. (1) Headquarters (1-5-1 Jingumae, Shibuya-ku, Tokyo) (2) Secom TE Center (6-11-23 Shimorenjaku, Mitaka-shi, Tokyo) Dates: (1) June 1 to 5, 2020 (2) June 18 to 25, 2020 [Publications, etc.] Website addresses: https://www.secom.co.jp/corporate/release/2020/nr_20200601.html https://www.secom.co.jp/corporate/release/2020/pdf/nr_20200601.pdf https://prtimes.jp/ jp/main/html/rd/p/000000022.000036837.html Date of publication: June 1, 2020 [Publications, etc.] Website address: https://www3.nhk.or.jp/news/html/20200601/k10012453261000.html Date of publication: June 1, 2020 [Publications, etc.] Website address: https://news.tv-asahi.co.jp/news_economy/articles/000185369. html Posted on: June 3, 2020 [Publications, etc.] Broadcast program: News (Kanto) Broadcast date: June 1, 2020 [Publications, etc.] Broadcast program: Super J Channel Broadcast date: June 2, 2020 [Publications, etc.] Interview location (email destination): Nihon Keizai Shimbun Inc. Interview date: June 1, 2020 [Publications, etc.] Interview location (email destination): Security Services Times Co., Ltd. Interview date: June 2, 2020 [Publications, etc.] Interview location (email destination): Nihon Keizai Shimbun Inc. Interview date: June 2, 2020

The present invention relates to a display device.

In recent years, interest in preventing the spread of infectious diseases has increased. For infectious diseases accompanied by fever, such as pneumonia caused by the new coronavirus, people suspected of infection can be easily identified by taking their temperature. Therefore, in facilities such as office buildings, hospitals, and commercial facilities, the facility manager takes the temperature of facility users near the entrance of the facility, isolates users suspected of having a fever from other users, and restricts entry to the facility. However, when taking temperatures, the manager comes into close proximity to or comes into contact with many users, which creates a problem in that both the manager and the users are at risk of infection.

Patent Document 1 describes a body temperature measuring device that uses a thermal camera to measure the body temperature of facility users, and if the measured body temperature exceeds a preset maximum body temperature, displays a message indicating that the user is suspected of having a fever, and allows entry to the facility if the body temperature does not exceed the maximum body temperature. By using the body temperature measuring device described in Patent Document 1, it becomes possible for an administrator to take the temperature of users without coming into close proximity or contact with them, and to identify users suspected of having a fever.

JP 2011-67371 A

However, users who are unfamiliar with the facility, such as those visiting the facility for the first time, are unable to determine how to act even if a message is displayed indicating that they may have a suspected fever. Therefore, the body temperature measuring device described in Patent Document 1 may not be able to properly guide people suspected of having a fever.

The present invention has been made to solve the above-mentioned problems, and aims to provide a display device that can appropriately guide people suspected of having a fever in a specified direction while reducing the risk of infection during temperature checks.

The display device according to the present invention has a display unit that displays an image of a character, a measurement unit that measures a person's body temperature without contact, a judgment unit that judges whether the measured body temperature is normal or not, a detection unit that detects a person in a waiting area, and a control unit that, when a person is detected in the waiting area, controls the behavior of the character to respond to the person, and is characterized in that, as a response, the control unit controls the behavior of the character to indicate the direction in which the person should proceed according to the result of the judgment.

The display device according to the present invention preferably further includes a memory unit that stores a first direction in which a person whose body temperature has been determined to be normal should move, and a second direction different from the first direction in which a person whose body temperature has been determined to be abnormal should move, and the control unit controls the movement of the character to indicate the first direction or the second direction depending on the result of the determination.

In addition, in the display device according to the present invention, it is preferable that the person is a user of a facility, the first direction is a direction of entering the facility, and the second direction is a direction toward an examination area where the person will be examined.

In addition, in the display device according to the present invention, it is preferable that the detection unit detects the moving direction of the person, and the control unit outputs a warning when the character's movement is controlled to indicate a second direction as the direction in which the person should move, and when the detection unit detects the first direction as the moving direction.

In addition, in the display device according to the present invention, it is preferable that the control unit sequentially attends to people in the attendance area, and when attending to a person whose body temperature is determined to be abnormal, attends to other people after a predetermined time has elapsed since attending to that person.

In addition, in the display device according to the present invention, it is preferable that the control unit controls the movement of the character so that the movement of the character when the body temperature is determined to be abnormal is smaller than the movement of the character when the body temperature is determined to be normal.

In addition, in the display device according to the present invention, it is preferable that the control unit outputs a sound as the character's speech and controls the volume of the speech so that the volume of the speech when the body temperature is determined to be abnormal is lower than the volume of the speech when the body temperature is determined to be normal.

The display device of the present invention makes it possible to appropriately guide people suspected of having a fever in a specified direction while reducing the risk of infection during temperature checks.

1 is a diagram illustrating an example of a schematic configuration of a response system 1. FIG. FIG. 2 is a diagram showing an example of a schematic configuration of a display device 2. FIG. 4 is a diagram showing an example of a first guide screen 400. FIG. 5 is a diagram showing an example of a second guide screen 500. FIG. 6 is a diagram showing an example of a body temperature display screen 600. 13 is a diagram showing an example of the data structure of an action table 271. FIG. 13 is a diagram showing an example of the data structure of a person table 272. FIG. FIG. 11 is a flowchart showing an example of the flow of a first storage process. FIG. 11 is a flow diagram showing an example of a process flow of normal control. FIG. 4 is a flow chart showing an example of a processing flow of emergency control. FIG. 1 is a diagram illustrating an example of a schematic configuration of a response system 1a. FIG. 2 is a diagram showing an example of a schematic configuration of a display device 2a. FIG. 13 is a diagram showing an example of the data structure of a person table 272a. FIG. 11 is a flow diagram showing an example of the flow of a second response process. FIG. 11 is a flowchart showing an example of the flow of an instruction information output process.

Various embodiments of the present invention will be described below with reference to the drawings. Please note that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions described in the claims and their equivalents.

First Embodiment
1 is a diagram showing an example of a schematic configuration of a response system 1 according to a first embodiment. The response system 1 is provided near an entrance 11 of a facility such as an office building, a hospital, or a commercial facility. The response system 1 includes a rope partition 12 arranged along a traffic line D1 of a user of the facility who entered through the entrance 11, a display device 2 arranged on the traffic line D1, and a partition 13 arranged to the side of the display device 2 and away from the traffic line D1. The display device 2 is an example of a response device.

A temperature measurement area A1 is formed in front of the display device 2. Inside the temperature measurement area A1, a reception area A2 that is smaller than the temperature measurement area A1 is formed close to the display device 2. Note that in the example shown in FIG. 1, the temperature measurement area A1 includes the reception area A2, but the two may be adjacent to each other or separated from each other. To the side of the display device 2, an examination area A3 is provided, separated by a partition 13, where users suspected of having a fever can be examined.

When facility users enter the facility through entrance 11, they move along flow line D1 into temperature measurement area A1, wait there, and then move one by one into reception area A2. Display device 2 measures the body temperature of users in temperature measurement area A1 and determines whether the body temperature is normal or not. A normal body temperature means that there is no suspicion of fever, and for example, that the body temperature is below a predetermined value (e.g., 37.5°C).

When the user moves into the reception area A2, the display device 2 indicates the direction in which the user should proceed according to the result of the determination. If the body temperature is determined to be normal, the display device 2 indicates the direction D2 for entering the facility as the direction in which the user should proceed, and if the body temperature is determined to be abnormal, the display device 2 indicates the direction D3 for the examination area A3 as the direction in which the user should proceed. Note that the directions D2 and D3 are different directions, and may be, for example, opposite directions on either side of the display device, or directions that can be distinguished by the guidance of the character C. The direction D2 for entering the facility is, for example, the direction toward the reception desk or the entrance gate. Note that the direction of entry is not limited to a certain direction, and multiple directions may be set according to the purpose of the user of the facility. In other words, the user is not limited to the direction toward the reception desk or the entrance gate, but may be guided, for example, toward the elevator hall or the waiting area according to the purpose of the user's visit.

The user proceeds in the direction D2 or D3 indicated on the display device 2. The user is shown the direction D3 to the testing area, and undergoes testing such as re-temperature checks and interviews by an examiner waiting in the testing area A3. Users who are determined to be free of infectious diseases proceed in direction D4 as instructed by the examiner and enter the facility. Users who are determined to be suspected of infectious diseases proceed in direction D5 as instructed by the examiner and exit through the exit 14. In addition to testing by an examiner, the testing area A3 may be configured to perform unattended re-temperature checks using a non-contact thermometer such as a thermal imaging camera, or to have the user perform a re-temperature check, and to guide the user to enter or exit depending on whether the re-temperature check is normal or not. When performing unattended re-temperature checks, the user to be tested is guided on where to stand and where to face, and the position and orientation of the user's face are strictly controlled to obtain more accurate temperature check results. The testing area may be configured so that an examiner waiting in a separate room performs the test remotely using a video call or the like. In this way, temperature checks and other tests can be performed unmanned, further reducing the risk of infection.

In addition, if the body temperature of a user waiting in the temperature measurement area A1 is determined to be abnormal, the display device 2 stops serving the user in the service area A2 and serves the user in the temperature measurement area A1 whose body temperature is determined to be abnormal. When serving the user in the temperature measurement area A1, the display device 2 indicates the direction to the testing area A3 as the direction the user should proceed.

Figure 2 is a front view of the display device 2, and Figure 3 is a diagram showing an example of the schematic configuration of the display device 2. The display device 2 has a first display unit 21, a sensor unit 22, an image capture unit 23, a thermal image capture unit 24, a second display unit 25, an audio output unit 26, a memory unit 27, and a processing unit 28.

The first display unit 21 is a component for displaying images provided on the front of the display device 2, and includes a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The first display unit 21 displays an image based on display data supplied from the processing unit 28. The first display unit 21 displays a character C that resembles a security guard as an image. The first display unit 21 is large enough to enable the character C to be displayed at approximately the same size as a real security guard. Note that the character C is not limited to one that resembles a security guard, and may be any character.

The first display unit 21 may be a mirror-type display in which a reflective member such as a half-mirror film is attached to the display. This allows the character C representing a security guard to be superimposed on the reflected scenery inside the facility, giving the impression that the security guard is actually present.

The sensor unit 22 is provided above the first display unit 21 and is configured to measure the distance to an object around the display device 2, and includes, for example, a Time of Flight (TOF) type laser distance measuring sensor. The sensor unit 22 irradiates pulse-modulated laser light in each direction within the measurement range, and calculates the distance to the reflection point in each direction based on the time it takes to detect the reflected light. The sensor unit 22 generates distance image data in which distance information is associated with each pixel based on the direction in which the laser light is irradiated and the calculated distance, and supplies the data to the processing unit 28.

The imaging unit 23 is provided alongside the sensor unit 22 on the upper side of the first display unit 21 and is a component for capturing an image of a subject, and includes, for example, a camera. The imaging unit 23 generates an image signal corresponding to an image of the subject formed by concentrating light rays from the subject within the imaging range using an optical lens or the like, converts it into image data in a predetermined format, and supplies it to the processing unit 28.

The thermal image capturing unit 24 is provided to the side of the first display unit 21 and is configured to generate a thermal image showing the temperature of the subject in the temperature measurement area A1, and includes, for example, a thermal imaging camera. The thermal image capturing unit 24 calculates the intensity of infrared rays incident from each direction within the measurement range using a thermopile or the like. The thermal image capturing unit 24 generates thermal image data in which temperature information is associated with each pixel based on the incident direction of infrared rays and the temperature corresponding to the infrared intensity, and supplies the data to the processing unit 28.

The measurement range of the sensor unit 22, the imaging range of the imaging unit 23, and the measurement range of the thermal image imaging unit 24 are all set to include the temperature measurement area A1. In addition, the correspondence between each pixel of the distance image data generated by the sensor unit 22, each pixel of the image data generated by the imaging unit 23, and each pixel of the thermal image data generated by the thermal image imaging unit 24 is set in advance and stored as data in the memory unit 27 described later.

The second display unit 25 is a component for displaying images, provided to the side of the first display unit 21, and includes a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The second display unit 25 displays images based on the display data supplied from the processing unit 28.

The audio input/output unit 26 is provided on the left and right sides of the first display unit 21, and is configured to output and collect audio, and includes, for example, a speaker and a microphone. The audio input/output unit 26 outputs audio based on audio data supplied from the processing unit 28, and also converts audio input to the microphone into digital input audio data and supplies it to the processing unit 28.

The storage unit 27 is configured to store programs and data, and includes, for example, a semiconductor memory. The storage unit 27 stores operating system programs, driver programs, application programs, data, etc., used for processing by the processing unit 28. Programs are installed into the storage unit 27 using a setup program or the like from a computer-readable, non-transient portable storage medium such as a CD (Compact Disc)-ROM (Read Only Memory).

The memory unit 27 stores an action table 271 and a person table 272 as data.

The processing unit 28 is configured to comprehensively control the operation of the display device 2, and includes one or more processors and their peripheral circuits. The processing unit 28 includes, for example, a CPU (Central Processing Unit), an LSI (Large Scale Integration), or an ASIC (Application Specific Integrated Circuit). The processing unit 28 may also include a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), etc. The processing unit 28 controls the operation of each component of the display device 2 so that the processing of the display device 2 is appropriately performed based on the program stored in the memory unit 27, and executes the processing.

The processing unit 28 has a measurement unit 281, a determination unit 282, a detection unit 283, a normal control unit 284, and an emergency control unit 285 as functional blocks. Each of these units is a functional module realized by a program executed by the processing unit 28. Each of these units may be implemented in the display device 2 as firmware.

Figure 4 is a diagram showing an example of the first guidance screen 400 displayed on the first display unit 21. The first guidance screen 400 is displayed to indicate the direction D2 of facility entry as the direction in which the user should proceed when the user's body temperature is determined to be normal. In the example shown in Figure 1, the direction D2 of facility entry is to the right as seen by the user, so the first guidance screen 400 includes a character C whose movement is controlled to indicate the right direction as seen by the user. Note that when the direction of entry is clear (for example, when it is known how users will enter an office or the like, or when an entrance gate is installed near the display device), the character C does not specifically indicate the direction, but may indicate the direction in which the user should proceed by outputting a voice message such as "Welcome, please proceed" to encourage entry.

Figure 5 is a diagram showing an example of the second guidance screen 500 displayed on the first display unit 21. The second guidance screen 500 is displayed to indicate the direction D3 to the examination area as the direction in which the user should proceed if the user's body temperature is determined to be abnormal. In the example shown in Figure 1, the direction D3 to the examination area is to the left as seen by the user, so the second guidance screen 500 includes a character C whose behavior is controlled to indicate the left as seen by the user.

The movements of character C on the second guide screen 500 are controlled to be smaller than the movements of character C on the first guide screen 400. Smaller movements of character C mean that fewer parts of character C are moving, or that the area on the screen that corresponds to the range of movement of character C is smaller. In the example shown in Figures 4 and 5, the forearms and upper arms of character C are moving on the first guide screen 400, whereas only the forearms of character C are moving on the second guide screen 500. In other words, the parts of character C moving on the second guide screen 500 are controlled to be smaller than the parts of character C moving on the first guide screen 400.

Figure 6 is a diagram showing an example of a body temperature display screen 600 displayed on the second display unit 25. The body temperature display screen 600 includes a captured image 601, a person detection object 602, and a body temperature display object 603.

The captured image 601 is an image of the user captured by the imaging unit 23. The person detection object 602 is an object that indicates an area in the captured image 601 that corresponds to the user. In the example shown in FIG. 6, the person detection object 602 indicates an area including the user's face, but it may indicate an area including the user's entire body or other body parts. The body temperature display object 603 is an object that indicates the body temperature of the user indicated by the person detection object 602, and is displayed near each of the person detection objects 602. Note that an image of the user captured by the thermal image capturing unit 24 may be used as the captured image 601.

Figure 7 is a diagram showing an example of the data structure of the action table 271 stored in the storage unit 27. The action table 271 stores the determination result, the guidance direction, the action, the sound, the volume, etc. in association with each other.

The judgment result is information indicating the result of judging whether the user's body temperature is normal or not. The guidance direction is the direction in which the user should proceed according to the judgment result. In the example shown in FIG. 7, since the direction D2 for entering the facility is to the right as seen by the user, the judgment result "body temperature is normal" is associated with the guidance direction of "rightward." In addition, since the direction D3 to the examination area is to the left as seen by the user, the judgment result "body temperature is not normal" is associated with the guidance direction of "leftward."

The action is information indicating the action of the character C to show the user the direction in which to proceed. When the character C is controlled to make a "large guiding action to the right", the first guide screen 400 is displayed. When the character C is controlled to make a "small guiding action to the left", the second guide screen 500 is displayed. As the action, a moving image file or the like may be stored, or data indicating the movement of the character's joints or the like may be stored. The sound is information indicating the sound as the character speaks to show the user the direction in which to proceed. As the sound, for example, a sound file or the like is stored.

The volume is information indicating the volume at which the associated voice is output. In the example shown in FIG. 7, a "high" volume is associated with the determination result of "body temperature is normal," and a "low" volume is associated with the determination result of "body temperature is not normal." In other words, the volume of the speech when the body temperature is determined to be not normal is controlled to be lower than the volume of the speech when the body temperature is determined to be normal. Note that in the case of a "low" volume, there is a risk that the user to whom the speech is addressed may miss it, so the display output on the second display unit 25 may be displayed as a warning (for example, in red).

Each data item in the operation table 271 is preset according to the structure of the facility and stored in the memory unit 27.

Figure 8 is a diagram showing an example of the data structure of the person table 272 stored in the storage unit 27. The person table 272 stores a person ID, person area, body temperature, judgment result, position, and traveling direction, etc., in association with each other.

The person ID is identification information for identifying a user detected in an image captured by the imaging unit 23. The person area is information that identifies the person area detected in the captured image, and is, for example, information indicating the center position of the person area. The information that identifies the person area may include information indicating the size and shape of the area. The body temperature is information that indicates the detected body temperature of the user. The judgment result is information that indicates the result of judging whether the user's body temperature is normal or not. The position is information that indicates the position of the user's feet. The direction of travel is information that indicates the direction of travel of the user.

Each data item in the person table 272 is generated by the display device 2 in the first storage process described below, and is stored in the storage unit 27.

The display device 2 executes the first storage process, normal control, and emergency control in parallel and asynchronously, as described below. These processes are executed repeatedly on a regular or irregular basis while the processing unit 28 is performing display control to display a character C resembling a security guard on the first display unit 21. These processes are realized by the processing unit 28 working in cooperation with each component of the display device 2 based on a program stored in the memory unit 27.

First, the display control of the character C will be described. The processing unit 28 controls the display of the character C by executing a program. The processing unit 28 detects a user, as described later in the description of the first storage process. When a user is not detected, the processing unit 28 controls the image display of the character C so that the character C performs a lookout action of moving the line of sight and body direction within a predetermined range (for example, 45 degrees from the front), and when a user is detected outside the response area A2, the processing unit 28 controls the image display of the character C so that the character C performs a tracking action of directing the line of sight toward the detected user. This allows an image that resembles a real security guard performing guard duty to be displayed, and the presence of a security guard is exerted. In addition, when the processing unit 28 detects a user or a speech from a user within the response area A2, the processing unit 28 controls the voice output so that the character C imitating a security guard converses with the user, thereby accepting and guiding the facility user.

The processing unit 28 may also determine whether the user intends to enter the facility, and control the display of the character C to provide guidance according to the temperature measurement results if the user intends to enter. For example, the processing unit 28 detects the user's moving direction, as described below. The processing unit 28 may determine whether the user intends to enter the facility based on whether the detected moving direction of the user is the direction of entering the facility, or may determine whether the user intends to enter based on the content of the user's utterance (for example, "I'm here to have a meeting with Mr. X"). When serving a user who does not intend to enter, that is, when serving is not performed based on the result of the temperature measurement, a predetermined utterance (for example, a greeting) is made to the user detected in the serving area A2. When receiving a question from the user (for example, "Where is the restroom?"), guidance is provided by voice output or display control according to a dialogue table that links the content of the question and the answer to the question stored in advance in the storage unit 27 (for example, "Please proceed to the passage on the right to get to the restroom"). In detecting the contents of the utterance, as described later in the explanation of normal control, the contents of the utterance are detected by applying known voice recognition technology and natural language processing technology to the input voice acquired from the voice input/output unit 26. Depending on the keywords contained in the detected contents of the utterance, the character C is made to perform a predetermined movement or vocalize.

Figure 9 is a flow diagram showing an example of the flow of the first storage process executed by the display device 2.

First, the measurement unit 281 measures the body temperature of one or more users in the temperature measurement area A1 (S101). The measurement unit 281 acquires a captured image captured by the imaging unit 23. The measurement unit 281 detects a person area corresponding to the user in the captured image by inputting the captured image to a trained model that has been trained to output a person area indicating a person in the captured image when the captured image is input. The trained model is, for example, a neural network for object detection such as R-CNN (Region with Convolutional Neural Network) or SSD (Single Shot Detector).

The measurement unit 281 acquires the thermal imaging data generated by the thermal image capturing unit 24. The measurement unit 281 identifies an area of the thermal imaging data that corresponds to the person area of the detected captured image. The measurement unit 281 acquires the temperature of the identified area of the thermal imaging data as the body temperature of the user. The measurement unit 281 displays a body temperature display screen 600 on the second display unit 25 based on the acquired captured image, the detected person area, and the acquired body temperature.

The measurement unit 281 identifies a user who is the same as the user indicated by the detected person area from among the users stored in the person table 272. For example, the measurement unit 281 calculates the difference between the center position of the detected person area and the center position of the person area stored in the person table 272, and identifies a user whose difference is equal to or less than a predetermined value as the same user as the user indicated by the detected person area. The measurement unit 281 updates the person table 272 by storing information identifying the detected person area and the acquired body temperature in the person table 272 in association with the person ID of the identified user. In this way, by performing a tracking process of users extracted across multiple input images and previously storing the temperature measurement results for each tracked user in association with each other in the person table 272, it becomes possible to promptly provide guidance according to the temperature measurement results of the user at the time the user enters the response area A2.

If the same user as the user indicated by the detected person area is not stored in the person table 272, the measurement unit 281 associates the information identifying the detected person area and the acquired body temperature with a new person ID and stores them in the person table 272.

Then, the determination unit 282 determines whether the measured body temperature is normal or not (S102). If the body temperature stored in the person table 272 is less than a predetermined value, the determination unit 282 determines that the measured body temperature is normal. If the body temperature stored in the person table 272 is equal to or greater than a predetermined value, the determination unit 282 determines that the measured body temperature is not normal. The determination unit 282 stores the result of the determination in the person table 272 in association with the body temperature.

Then, the detection unit 283 calculates the position and traveling direction of the user within the temperature measurement area (S103) and ends the detection process. The detection unit 283 identifies an area of distance image data that corresponds to the person area stored in the person table 272. The detection unit 283 calculates the position of the user based on distance information associated with the identified area of distance image data. The detection unit 283 calculates the traveling direction of the user from the difference between the calculated position of the user and the position stored in the person table 272. The detection unit 283 stores the calculated position and traveling direction of the user in the person table 272.

Figure 10 is a flow diagram showing an example of a normal control flow in which people in the service area are served in sequence and served based on the measurement results for each person.

First, the detection unit 283 determines whether a user in the response area has been detected (S201). The detection unit 283 determines whether a user in the response area has been detected by determining whether each of the positions stored in the person table 272 is included in a range that has been set in advance as the response area. If a user in the response area has not been detected (S201-No), the process of determining whether a user in the response area has been detected is repeated.

When a user is detected in the service area (S201-Yes), the normal control unit 284 determines whether the body temperature of the detected user in the service area is normal (S202). The normal control unit 284 refers to the determination result in the person table 272 and determines whether the body temperature of the detected user is normal. In addition, the normal control unit 284 responds to the user in order starting with the person closest to the display device 2, so that even if multiple people are detected, the user can be quickly served.

If it is determined that the detected body temperature of the user is normal (S202-Yes), the normal control unit 284 responds to the detected user by controlling the character C to indicate the direction D2 in which the user should enter the facility as the direction in which the user should proceed (S203). The normal control unit 284 refers to the action table 271 and acquires the action, voice, and volume associated with the determination result of "body temperature is normal". The normal control unit 284 controls the character C to perform the acquired action, and causes the first guidance screen 400 to be displayed on the first display unit 21. The normal control unit 284 also causes the acquired voice to be output from the voice output unit 26 at the acquired volume as the speech of the character C.

The normal control unit 284 then determines whether a first time has elapsed since controlling the character C to indicate the direction D2 for entering the facility (S204). The first time is a time that is predetermined to be sufficient for a user who has been shown the direction to proceed to leave the reception area, and is, for example, 5 seconds. If it is determined that the first time has not elapsed (S204-No), the reception process returns to S204. In other words, the normal control unit 284 waits until the first time has elapsed.

When it is determined that the first time has elapsed (S204-Yes), the normal control unit 284 guides the other user to the service area (S205), ends the normal control for serving the user, and serves the next user. The normal control unit 284 guides the other user to the service area by outputting a voice such as "Next, please" from the voice output unit 26 as the speech of the character C. That is, the normal control unit 284 serves the other user after the first time has elapsed since serving the detected user. In addition, in the normal control, the determination of whether the first time has elapsed or not may be omitted, and the next user may be promptly served after serving the user. Note that since multiple people may exist in the service area, when multiple people exist in the service area, the process of guiding the other user to the service area (S205) is omitted, and the normal control for serving the next user is executed in sequence.

If it is determined in S202 that the body temperature of the user detected is not normal (S202-No), the normal control unit 284 responds to the detected user by controlling the character C to indicate the direction D3 to the examination area as the direction in which the user should proceed (S206). The normal control unit 284 refers to the action table 271 and acquires the action and sound associated with the determination result of "body temperature is not normal". The normal control unit 284 controls the character C to perform the acquired action, and causes the second guidance screen 500 to be displayed on the first display unit 21. The normal control unit 284 also causes the acquired sound to be output from the sound output unit 26 at the acquired volume as the speech of the character C.

Next, the normal control unit 284 determines whether the detected user's traveling direction is the direction D2 of entering the facility (S207). The normal control unit 284 refers to the person table 272 to determine whether the traveling direction is the direction D2 of entering the facility.

If the traveling direction is the direction D2 for entering the facility (S207-Yes), the normal control unit 284 outputs a warning (S208). The warning is issued by outputting a voice as the speech of the character C. For example, the normal control unit 284 outputs the warning by repeatedly outputting a voice such as "Please go left" from the voice output unit 26. The warning may be issued by controlling the movement of the character C or by outputting a sound effect. Note that the case where the traveling direction is the direction D2 for entering the facility does not necessarily mean that the traveling direction exactly matches the direction D2, but means that the traveling direction is included in a predetermined angle range including the direction D2. If the traveling direction is not the direction D2 for entering the facility (S207-No), the normal control proceeds to S209.

The normal control unit 284 then determines whether a second time has elapsed since controlling the character C to indicate the direction D3 to the inspection area (S209). The second time is a time longer than the first time that is preset so as to be sufficient for a user who has been shown the direction to proceed to move to the inspection area, e.g., 10 seconds. If it is determined that the second time has not elapsed (S209-No), the response process returns to S207. In other words, the normal control unit 284 waits until the second time has elapsed.

When it is determined that the second time has elapsed (S209-Yes), the normal control unit 284 guides the other user to the service area (S205), ends the normal control of serving the user, and moves to serving the next user. That is, the normal control unit 284 serves the other user after the second time has elapsed since serving the detected user. Note that even when the determination of whether the first time has elapsed is omitted, the second time can be a time that is preset so that it is sufficient for the user who has been shown the direction to proceed as described above to move to the inspection area.

Figure 11 is a flow diagram showing an example of the processing flow of emergency control.

First, the emergency control unit 285 judges whether the body temperature of the user in the temperature measurement area is normal or not (S301). The emergency control unit 285 judges whether the body temperature of the user is normal or not by referring to the judgment result of the person table 272. If it is judged that the body temperature of the user in the temperature measurement area is normal (S301-Yes), the process of judging whether the body temperature of the user in the temperature measurement area but outside the response area is normal or not is repeated.

If it is determined that the body temperature of the user in the temperature measurement area is not normal (S301-No), the emergency control unit 285 suspends normal control (S302). The emergency control unit 285 waits until the normal control being executed for the person currently being served is completed, and then suspends normal control by controlling so that normal control processing for the next person is not executed. The emergency control unit 285 may immediately suspend the normal control currently being executed. In this case, a voice such as "Please wait a moment" is output as the speech of character C, and the user in the service area is made to wait.

Then, the emergency control unit 285 controls the character C to indicate the direction D3 to the testing area as a response to the user whose body temperature has been determined to be abnormal (S303). The emergency control unit 285 refers to the action table 271 to obtain the action, voice, and volume associated with the determination result of "body temperature is abnormal." The emergency control unit 285 controls the character C to perform the obtained action, and causes the second guidance screen 500 to be displayed on the first display unit 21.

The emergency control unit 285 also refers to the position in the person table 272 to identify the positional relationship of the user whose body temperature has been determined to be abnormal relative to other users. The positional relationship of other users is, for example, information indicating the position in line of the user whose body temperature has been determined to be abnormal when multiple users are lined up in the temperature measurement area. The emergency control unit 285 outputs the acquired voice with the identified positional relationship attached as the speech of character C from the voice output unit 26 at the acquired volume. For example, the emergency control unit 285 outputs a voice such as, "Third in line, please proceed to the left and be examined."

Then, the emergency control unit 285 determines whether the user whose body temperature is determined to be abnormal is traveling in the direction D3 toward the inspection area (S304). The emergency control unit 285 refers to the traveling direction in the person table 272 to determine whether the traveling direction is the direction D3 toward the inspection area. If the traveling direction is not the direction D3 toward the inspection area (S304-No), the response process returns to S304. In other words, the emergency control unit 285 waits until it is determined that the traveling direction of the user whose body temperature is determined to be abnormal is the direction D3 toward the inspection area.

If the direction of travel of the user with an abnormal body temperature is the direction D3 toward the testing area (S304-Yes), the emergency control unit 285 guides other users to the waiting area (S305), and the emergency control ends. Note that if there are multiple people in the waiting area A2 in S305, the people are served in order starting with the person closest to the display device.

As described above, the display device 2 measures the body temperature of facility users, and controls the character C to indicate the direction the user should go depending on the result of judging whether the measured body temperature is normal, thereby executing normal control to respond to the user. In this way, the display device 2 can appropriately guide people suspected of having a fever, reducing the risk of infection.

In addition, under normal control, the display device 2 controls the behavior of the character C to indicate the direction D2 for entering the facility if the user's body temperature is determined to be normal, and to indicate the direction D3 to the testing area if the user's body temperature is determined to be abnormal. This allows appropriate guidance according to the temperature measurement results, reducing the risk of infection.

In addition, when the direction D3 toward the inspection area is indicated as the direction in which the user should proceed, and the user's proceeding direction is the direction D2 toward entering the facility, the display device 2 outputs a warning. This enables the display device 2 to more appropriately guide people suspected of having a fever and reduce the risk of infection.

Furthermore, if the display device 2 determines that the user's body temperature is normal, it responds to other users after a first time has elapsed since serving the user, and if the body temperature is determined to be abnormal, it responds to other users after a second time, which is longer than the first time, has elapsed since serving the user. This allows other users to be guided into the service area only after the user suspected of having a fever has sufficiently left the service area, thereby reducing the risk of the person suspected of having a fever coming into close proximity with other users.

The display device 2 also controls the movement of the character C on the second guide screen 500 so that it is smaller than the movement of the character C on the first guide screen 400. In other words, the display device 2 controls the movement of the character C so that the movement of the character C when it is determined that the user's body temperature is not normal is smaller than the movement of the character C when it is determined that the body temperature is normal. This reduces the possibility that the content of the response to the user suspected of having a fever will be seen by others, and the privacy of the person suspected of having a fever can be protected.

The display device 2 also controls the volume of the speech of the character C when the body temperature is determined to be abnormal, so that the volume of the speech is lower than the volume of the speech when the body temperature is determined to be normal. This reduces the possibility that the content of the response to a user suspected of having a fever will be overheard by others, and the privacy of the person suspected of having a fever can be protected.

In addition, when the display device 2 determines that the body temperature of another user in the service area other than the detected user is not normal, it suspends normal control to serve the detected user and executes emergency control to serve the other user. This makes it possible for the display device 2 to quickly guide people suspected of having a fever.

In addition, in emergency control, if the direction of travel of the attended user is the direction D3 toward the testing area, the display device 2 ends the emergency control and executes normal control. This allows other users to be attended to after the person suspected of having a fever has started to move in accordance with the guidance in emergency control, making it possible to reduce the risk of infection between users.

In the above description, the display device 2 executes the first storage process, normal control, and emergency control in parallel, but it may execute only the first storage process and normal control in parallel without executing emergency control.

In the above description, the normal control unit 284 determines whether the body temperature of the person detected in S202 of normal control is normal or not, but this is not limited to an example. The normal control unit 284 may control the character C to indicate the direction D2 of entering the facility as the direction in which the user should proceed. In other words, since the person suspected of having a fever is dealt with under emergency control while waiting in the temperature measurement area A1, the user who moves to the response area A2 is considered not to be a person suspected of having a fever. Therefore, by doing this, it is possible to quickly isolate the person suspected of having a fever from other users of the facility while reducing the processing load on the display device 2.

In the above description, the normal control unit 284 provides guidance depending on whether the detected body temperature of the person is normal or not, but in addition to this, it may determine whether the user is wearing a mask from the captured image, and if it is determined that the user is not wearing a mask, it may output audio or control the display to encourage the user to wear a mask, or provide guidance to deny entry. Similarly, the emergency control unit 285 may also respond depending on the result of determining whether the user is wearing a mask or not.

In the above description, the first time and the second time are preset times, but are not limited to this example. For example, the display device 2 may determine that the first time has elapsed when the direction D2 of entry into the facility is detected as the traveling direction of the served user. The display device 2 may also determine that the second time has elapsed when the served user reaches the inspection area. This prevents a user not suspected of having a fever from coming into close proximity with the next user moving into the serving area, and makes it possible to more appropriately isolate a person suspected of having a fever from other users.

In the above description, the determination unit 282 determines whether the body temperature is normal or not based on whether the body temperature is below a predetermined value, but this is not limited to the example. For example, the determination unit 282 may determine whether the body temperature is normal or not by comparing the measured body temperature with a normal body temperature preset for each user.

In this case, the storage unit 27 stores information about the user's characteristics in advance in association with the user's normal body temperature. The user's characteristics are information about characteristics that can identify the user from the captured image, such as the relative positions of the eyes and nose. Then, in S101 of the first storage process, when the measurement unit 281 extracts a person area from the captured image, it further extracts the user's characteristics that correspond to the extracted person area. Also, in S102, the determination unit 282 determines the user's normal body temperature based on the extracted user's characteristics, and determines whether the body temperature is normal by comparing the measured body temperature with the determined normal body temperature. For example, the determination unit 282 determines that the body temperature is normal when the difference between the measured body temperature and the normal body temperature is less than a predetermined value.

This makes it possible to identify users suspected of having a fever with high accuracy in facilities where the number of users is limited, such as office buildings. The normal body temperature of a user may be set in advance by the user, or may be calculated by the measuring unit 281 by measuring the user's body temperature for a predetermined period of time.

Second Embodiment
12 is a diagram showing an example of a schematic configuration of the response system 1a according to the second embodiment. Hereinafter, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted as appropriate. The response system 1a is provided near the entrance 11 of the facility, similar to the response system 1.

Response system 1a has a display device 2a arranged on the flow line D1 of facility users who enter through entrance 11, and a partition 13 arranged to the side of display device 2a. Response system 1a differs from response system 1 in that it does not have a rope partition 12.

When facility users enter the facility through entrance 11, they wait in temperature measurement area A1 according to traffic flow D1, and then move one by one into reception area A2. Display device 2a measures the body temperature of users in temperature measurement area A1 and determines whether the body temperature is normal or not. When a user moves into reception area A2, display device 2a indicates the direction in which the user should proceed, either the direction D2 for entering the facility or the direction D3 to examination area A3, depending on the result of the determination.

Also, since no rope partition is provided, in addition to users waiting in the temperature measurement area A1, there are other users, such as users moving around. If the display device 2 determines that the body temperature of a user in the temperature measurement area A1 is not normal, it outputs instruction information including characteristic information indicating a characteristic that allows the user to be uniquely identified by speech from among the multiple users and to move the user to the inspection area A3.

Figure 13 is a diagram showing an example of the schematic configuration of display device 2a. Display device 2a differs from display device 2 in that storage unit 27 stores person table 272a as data instead of person table 272. Display device 2a also differs from display device 2 in that display device 2a has processing unit 28a instead of processing unit 28.

The processing unit 28a is configured to comprehensively control the operation of the display device 2a, and includes one or more processors and their peripheral circuits. The processing unit 28a controls the operation of each component of the display device 2a so that the processing of the display device 2 is executed appropriately based on the program stored in the storage unit 27, and also executes the processing.

The processing unit 28a includes a measurement unit 281a, an extraction unit 282a, a determination unit 283a, a detection unit 284a, a normal control unit 285a, an identification unit 286a, and an emergency control unit 287a as functional blocks. Each of these units is a functional module realized by a program executed by the processing unit 28a. Each of these units may be implemented in the display device 2a as firmware.

Figure 14 is a diagram showing an example of the data structure of the person table 272a stored in the storage unit 27. The person table 272a stores a person ID, person area, body temperature, judgment result, position, direction of travel, feature information, and identification feature information in association with each other.

Characteristic information is information that indicates the characteristics of multiple users, and is information about the users' clothing, belongings, etc. Identifying characteristic information is characteristic information that can uniquely identify one user from multiple users by their speech. The characteristic information and identifying characteristic information are generated by the display device 2a in the second response process described below, and are stored in the person table 272a.

The display device 2a executes the second storage process described below, the normal control described with reference to FIG. 10, and the instruction information output process described below in parallel and asynchronously. These processes are executed repeatedly on a regular or irregular basis while the first display unit 21 displays the character C that resembles a security guard. These processes are realized by the processing unit 28a working in cooperation with each component of the display device 2a based on the program stored in the memory unit 27.

Figure 15 shows an example of the flow of the second storage process executed by the display device 2a.

First, the measurement unit 281a measures the body temperatures of multiple users in the temperature measurement area (S401). The measurement unit 281a acquires an image captured by the imaging unit 23. The measurement unit 281a detects a person area corresponding to the user in the captured image. The measurement unit 281a acquires the temperature of an area of the thermal imaging data generated by the thermal image capturing unit 24 that corresponds to the detected person area as the user's body temperature. The measurement unit 281 sets a person ID for the detected user and stores it in the person table 272 in association with the acquired body temperature.

Next, the extraction unit 282a extracts identification feature information indicating features that can uniquely identify one user by speech from among multiple users in the temperature measurement area (S402). The extraction unit 282a detects feature information indicating features related to the user's clothing or belongings by inputting the captured image to a trained model that has been trained to output features related to the user's clothing or belongings when a captured image is input. The feature information is indicated by a combination of a pre-set type of clothing or belongings, such as "clothing, black" or "bag, checkered pattern," and features related to the appearance of the clothing or belongings. The extraction unit 282a associates the detected feature information with the user based on the positional relationship with the person area in the captured image, and stores it in the person table 272a. Note that the detection of feature information may be performed simultaneously with the detection of the person area in S401.

The extraction unit 282a extracts, from the stored feature information, identification feature information indicating features that can uniquely identify one user from multiple users through speech. In the example shown in FIG. 14, only the user with the person ID of "0001" has the feature information of "clothes, red", so the user with the person ID of "0001" can be uniquely identified by the feature information of "clothes, red". Therefore, the extraction unit 282a extracts the feature information of "clothes, red" as identification feature information for the user with the person ID of "0001". Similarly, the extraction unit 282a extracts the feature information of "bag, red" as identification feature information for the user with the person ID of "0002". Note that, for the user with the person ID of "0003", uniquely identifiable feature information has not been detected, so identification feature information is not extracted. The extraction unit 282a stores the extracted identification feature information in the person table 272a.

Then, the determination unit 283a determines whether the measured body temperature is normal or not (S403). The determination unit 283a stores the result of the determination in the person table 272a in association with the body temperature.

Next, the detection unit 284a calculates the position and direction of travel of the user within the temperature measurement area (S404) and ends the second storage process. The detection unit 284a calculates the position and direction of travel of the user in the same manner as in S103, and stores them in the person table 272a.

Figure 16 is a flow diagram showing an example of the flow of instruction information output processing.

First, the identification unit 286a identifies users whose body temperature has been determined to be abnormal and from whom identification feature information has been extracted (S501). The identification unit 286a refers to the determination result in the person table 272a and identifies users whose body temperature has been determined to be abnormal. If the identified users include a user from whom identification feature information has been extracted, the identification unit 286a identifies that user as a user whose body temperature has been determined to be abnormal and from whom identification feature information has been extracted. Note that if the extracted users do not include a user from whom identification feature information has been extracted, the identification unit 286a does not identify a user (S501-No) and the instruction information output process ends.

When a user whose body temperature is determined to be abnormal and whose identification feature information has been extracted is identified (S501-Yes), the emergency control unit 287a determines whether the identified person's travel direction is a direction to exit the facility (S502). The emergency control unit 287a refers to the person table 272a and determines whether the direction to exit the facility has been detected as the travel direction of the identified person. The direction to exit the facility is the direction from the user's current position toward the exit 14.

If the direction of travel is not the direction of exiting the facility (S502-No), the emergency control unit 287a suspends normal control (S503).

Next, the emergency control unit 287a outputs instruction information that includes the identification feature information and causes the identified user to move to the inspection area (S504). For example, if the identification feature information is "bag, red", the emergency control unit 287a outputs a voice such as "Anyone with a red bag, please move left and move to the inspection area" from the voice output unit 26.

Then, the emergency control unit 287a determines whether the identified user's traveling direction is the direction D3 toward the inspection area (S505). The emergency control unit 287a refers to the person table 272a to determine whether the traveling direction is the direction D3 toward the inspection area. If the traveling direction is not the direction D3 toward the inspection area (S505-No), the instruction information output process returns to S505. That is, the emergency control unit 287a waits until it is determined that the identified user's traveling direction is the direction D3 toward the inspection area.

Next, the emergency control unit 287a guides the other user to the reception area (S506). Note that since multiple people may be present in the reception area, if multiple people are present in the reception area, the process of guiding the other user to the reception area (S506) is omitted, and the process moves on to the next user in sequence.

In S502, if the travel direction is the direction of exiting the facility (S502-Yes), the instruction information output process ends. In other words, if the identified user's travel direction is the direction of exiting the facility, the emergency control unit 287a does not output instruction information.

As described above, the display device 2a measures the body temperature of facility users and extracts feature information indicating features that can uniquely identify a user from the speech of multiple users. Furthermore, when the display device 2a determines that the body temperature is not normal and identifies a person from whom identifying feature information has been extracted, it outputs instruction information that includes the identifying feature information and causes the identified person to move to an inspection area. In this way, the display device 2a makes it possible to appropriately guide people suspected of having a fever and reduce the risk of infection.

In other words, when there are other users, such as users who are moving, in addition to the users waiting in the temperature measurement area, it is difficult to properly convey instruction information to those users even if they are suspected of having a fever. By including identification feature information in the instruction information, the display device 2a makes it easier for users suspected of having a fever to recognize that the output instruction information is intended for them, making it possible to properly guide those suspected of having a fever and reduce the risk of infection.

The display device 2a also executes normal control to respond to people in the response area, and when a person whose body temperature is determined to be abnormal and whose identification feature information has been extracted is identified, the display device 2a interrupts normal control and outputs instruction information. This enables the display device 2a to appropriately guide people suspected of having a fever while providing guidance to users who are not suspected of having a fever, thereby reducing the risk of infection.

In addition, if the identified user is moving in the direction of exiting the facility, the display device 2a does not output instruction information to the identified user. This prevents the display device 2a from prompting an unnecessary inspection to a user who is exiting the facility.

In the above description, in S501, the identification unit 286a identifies users whose body temperature has been determined to be abnormal and whose identification feature information has been extracted, but this is not limited to an example. In S501, the identification unit 286a may identify only users outside the service area among users whose body temperature has been determined to be abnormal and whose identification feature information has been extracted. This allows the display device 2a to quickly guide people suspected of having a fever outside the service area, thereby reducing the risk of infection.

In the above description, the identification feature information is information about the user's clothing or belongings, but is not limited to this example. The identification feature information may include information about the clothing or belongings of other users and information indicating the relationship with the other users. For example, the display device 2a outputs a voice such as "Please stand behind the person wearing red clothing" as instruction information. This makes it possible to output instruction information to a user who does not have characteristic information that can be uniquely identified by speech from among multiple users, and it becomes possible to properly isolate a user suspected of having a fever from other users. In addition, when identification feature information is not extracted from the characteristic information of a user, the display device 2a may generate identification feature information including information about the clothing or belongings of other users and information indicating the relationship with the other users. In addition, the positional relationship of the user whose body temperature is determined to be abnormal with respect to other users as described above may be used as identifiable information. In addition, if the face information of a user is registered in advance in an office or the like, the user may be identified by face authentication and the name of the user may be used as the identification feature information.

Those skilled in the art should understand that various changes, substitutions, and modifications can be made to the present invention without departing from the spirit and scope of the present invention. For example, the processing of each part described above may be performed in a different order as appropriate within the scope of the present invention. Furthermore, the above-described embodiments and modifications may be implemented in combination as appropriate within the scope of the present invention.

2 Display device 21 First display unit 22 Sensor unit 23 Imaging unit 24 Thermal image imaging unit 25 Second display unit 26 Audio output unit 27 Memory unit 28 Processing unit 281 Measurement unit 282 Determination unit 283 Detection unit 284 Normal control unit 285 Emergency control unit

Claims (7)

A display unit that displays an image of a character;
A measurement unit that measures a person's body temperature non-contact;
A determination unit that determines whether the measured body temperature is normal or not;
A detection unit that detects a person within the response area;
a control unit that, when a person is detected within the interaction area, controls the movement or voice of the character to execute a normal interaction with the person,
The display device is characterized in that when the control unit determines that the body temperature of the detected person is not normal, instead of the normal response, it executes a fever response that controls the movement or voice of the character to instruct the person in the direction in which to go. a storage unit that stores a first direction in which the person whose body temperature is determined to be normal should move, and a second direction different from the first direction in which the person whose body temperature is determined to be abnormal should move,
The control unit controls an action of the character to indicate the first direction or the second direction depending on a result of the determination.
The display device according to claim 1 . The person is a user of the facility,
The first direction is a direction of entry into the facility,
the second direction is a direction to a screening area where the person will be screened;
The display device according to claim 2 . The detection unit detects a traveling direction of the person,
the control unit outputs a warning when a movement of the character is controlled to indicate the second direction as a direction in which the person should move and when the moving direction of the person is the first direction.
The display device according to claim 2 or 3. the control unit sequentially attends to the people in the attendance area, and when attending to a person whose body temperature is determined to be abnormal, attends to another person after a predetermined time has elapsed since attending to the person.
A display device according to any one of claims 1 to 4. the control unit controls the movement of the character such that the movement of the character when the body temperature is determined to be abnormal is smaller than the movement of the character when the body temperature is determined to be normal.
A display device according to any one of claims 1 to 5. the control unit outputs a sound as an utterance of the character, and controls a volume of the utterance such that a volume of the utterance when the body temperature is determined to be abnormal is smaller than a volume of the utterance when the body temperature is determined to be normal.
A display device according to any one of claims 1 to 6.

Images