JP2021173935A - Information processing system, information processing method, and program - Google Patents

Abstract

To provide an information processing system that can detect the activity of participants for each of a plurality of groups.SOLUTION: An information processing system is an information processing system comprising an information processing apparatus 1 and a terminal 2 communicably connected with the information processing apparatus, and comprises an activity detection unit that detects the activity of participants for each of a plurality of groups based on at least either one of images of the participants belonging to the groups or voices uttered by the participants.SELECTED DRAWING: Figure 2

Description

The present application relates to information processing systems, information processing methods and programs.

Conventionally, there is known an information processing system that detects the raising hand of a student such as a student who takes a lesson in a classroom and detects the activity of the student (see, for example, Patent Document 1).

Further, an information processing system for determining a student's proficiency level based on a student's image taken by a camera is disclosed (see, for example, Patent Document 2).

However, in a conventional information processing system, when a student constitutes a plurality of groups, the activity level of the student may not be detected for each of the plurality of groups.

An object of the present invention is to detect the activity of a student in each of a plurality of groups.

The information processing system according to one aspect of the present invention is an information processing system including an information processing device and a terminal communicably connected to the information processing device, and is an image of a student belonging to a group or the above-mentioned. An activity detection unit for detecting the activity of the student in each of a plurality of groups based on at least one of the voices emitted by the student is provided.

According to the present invention, the activity of students can be detected in each of a plurality of groups.

It is a figure of the configuration example of the lesson support system which concerns on 1st Embodiment. It is a figure of the arrangement example of the lesson support system which concerns on 1st Embodiment. It is a block diagram of the hardware configuration example of the web server which concerns on embodiment. It is a block diagram of the hardware configuration example of the smartphone which concerns on embodiment. It is a block diagram of the hardware configuration example of the spherical camera which concerns on embodiment. It is a block diagram of the functional configuration example of the lesson support system which concerns on 1st Embodiment. It is a figure which shows the display screen example of a smartphone, (a) is a figure of a UI screen, (b) is a figure of the activity detection result screen. It is a figure of the operation example of the lesson support system which concerns on 1st Embodiment. It is a figure of the configuration example of the lesson support system which concerns on 2nd Embodiment. It is a block diagram of the functional configuration example of the lesson support system which concerns on 2nd Embodiment. It is a block diagram of the functional configuration example of the lesson support system which concerns on 3rd Embodiment.

Hereinafter, modes for carrying out the invention will be described with reference to the drawings. In each drawing, the same components are designated by the same reference numerals, and duplicate description will be omitted as appropriate. Further, the embodiments shown below exemplify an information processing system for embodying the technical idea of the present invention, and the present invention is not limited to the embodiments shown below.

The information processing system according to the embodiment includes an information processing device and a terminal communicatively connected to the information processing device, and is based on at least one of an image of a student belonging to a group or a voice emitted by the student. Detect the activity of students in each group.

Here, the activity level is an index showing the degree of activity of the student. For example, the higher the number of hands raised by the students and the number of voices uttered by the students, the higher the activity. A group is a group composed of one or more students. Furthermore, a student is a person who receives an explanation. In the case of a class or lecture, the student who receives the explanation given by the teacher or university professor corresponds to the student, and in the case of the seminar, the seminar participant who receives the explanation given by the lecturer corresponds to the student. In the product description, the customer who receives the explanation given by the product explainer corresponds to the student, and in the case of a meeting, the person who receives the explanation of the presenter corresponds to the student.

In the following, an embodiment will be described using a lesson support system that supports a lesson that a teacher gives to a student (an example of a student) in a school classroom as an example of an information processing system. In the embodiment, the activity of the student is detected for each of a plurality of groups composed of the students, and the detection result of the activity is displayed on the terminal provided by the teacher. This allows the teacher to recognize the student activity in multiple groups via the terminal.

[First Embodiment]
First, the lesson support system according to the first embodiment will be described.

<Structure example of class support system 100>
FIG. 1 is a diagram showing an example of the configuration of the lesson support system 100 according to the present embodiment. As shown in FIG. 1, the lesson support system 100 includes a web server 1, a smartphone 2, and three spherical cameras 3a, 3b, and 3c. Each of these is provided with a means of wireless communication or a means of wired communication, and is connected to each other so as to be able to communicate with each other via a network N such as a LAN (Local Area Network) or the Internet.

The web server 1 is a server computer that provides a service for displaying objects such as HTML (HyperText Markup Language) and images to a web browser, which is client software, in accordance with HTTP (Hypertext Transfer Protocol). This is an example.

However, the information processing device is not limited to the web server, and may be a PC (Personal Computer) or the like as long as it can communicate via the network. However, using a web server as an information processing device is preferable because objects such as HTML (HyperText Markup Language) and images can be displayed in parallel on the screens of a plurality of terminals connected to the network N. be.

The number of information processing devices is not limited to one, and a plurality of information processing devices may be connected to the lesson support system 100. Further, the function of the information processing device described later may be realized by processing by one information processing device, or may be realized by distributed processing by a plurality of information processing devices.

The smartphone 2 is a terminal that is communicably connected to the web server 1. The smartphone 2 is provided with a screen as a display unit for displaying the activity detection result. The teacher can give a lesson while holding the smartphone 2 in a state where the screen of the smartphone 2 can be visually recognized during the lesson.

However, the terminal is not limited to a smartphone, and may be a tablet, a PC (Personal Computer), a mobile phone, or the like. Further, the number of terminals is not limited to one, and a plurality of terminals may be connected to the lesson support system 100. In this embodiment, since it is a lesson support system, the teacher uses the terminal, but the present invention is not limited to this, and the person who gives the explanation can use the terminal.

The spherical cameras 3a, 3b, and 3c are photographing units that capture images of students in groups of a plurality of groups, and can capture images such as still images or streaming moving images in a 360-degree field of view in all directions of up, down, left, and right.

In this embodiment, the students who take the lesson are divided into three groups, and one spherical camera 3a, 3b, and 3c is provided for each of the three groups. The spherical cameras 3a, 3b, and 3c can photograph one or more students belonging to each group.

The number of shooting units is not limited to three, and may be increased or decreased. In addition, the photographing unit is not limited to the spherical camera, and it is sufficient that the images of the students can be photographed by a plurality of groups. However, if a spherical camera is used, images of students belonging to a group can be taken in a wide field of view, and it is possible to avoid limiting the position of students with respect to the shooting section and requiring multiple cameras in one group. Therefore, it is suitable. In the following, for convenience of explanation, the spherical cameras 3a, 3b, and 3c will be referred to as the spherical camera 3 when not particularly distinguished.

In addition to the configuration shown in FIG. 1, another configuration having a communication function may be connected to the lesson support system 100. Other configurations such as this include PJs (Projectors), IWBs (Interactive White Boards), output devices such as digital signage, and HUDs (Head Up Display). Devices, industrial machines, imaging devices, sound collectors, medical devices, network home appliances, automobiles (Connected Cars), notebook PCs (Personal Computers), mobile phones, smartphones, tablet terminals, game machines, PDA (Personal Digital Assistant), digital Examples include a camera, a wearable PC, a desktop PC, and the like.

<Example of placement of class support system 100>
Next, the arrangement of the lesson support system 100 will be described. FIG. 2 is a diagram showing an example of the arrangement of the lesson support system 100. FIG. 2 shows a state in which a lesson support system 100 is arranged in a classroom 10 and a teacher 20 gives a lesson to a student 30 using the lesson support system 100.

The web server 1 is located outside the classroom 10. However, the web server 1 may be arranged in the classroom 10. Further, in FIG. 2, the web server 1 is arranged outside the classroom 10 and in the vicinity of the classroom 10, but the web server 1 may be installed in a remote place farther from the classroom 10.

The teacher 20 holds the smartphone 2 so that the screen of the smartphone 2 can be visually recognized. However, it is not always necessary to hold the smartphone 2, and the teacher 20 may place the smartphone 2 on a desk or the like while the screen of the smartphone 2 can be visually recognized.

Desks 4a, 4b, and 4c (hereinafter referred to as desks 4 when not distinguished) are installed in the classroom 10. A plurality of students 30 are seated around each of these desks 4. In the example of FIG. 2, six students 30 are seated around one desk, and six students form one group. That is, 6 students sitting around the desk 4a form a group 5a, 6 students sitting around the desk 4b form a group 5b, and 6 students sitting around the desk 4c form a group. It constitutes 5c.

The number of students in the group is not limited to 6 and may increase or decrease. The number of people forming a group may be the same or different for each group, and the number of groups is not limited to three and may increase or decrease. Further, the position where the desk 4 is arranged in the classroom 10 is not particularly limited, and the positions of the desks 4a, 4b, and 4c may be changed by movement.

On the desk 4a, a spherical camera 3a is installed so as to be able to photograph six students forming the group 5a. Similarly, on the desk 4b, the spherical camera 3b is installed so as to be able to photograph the six students forming the group 5b, and on the desk 4c, the spherical camera 3c constitutes the group 5c. It is set up so that 6 students can be photographed.

The images captured by the spherical cameras 3a, 3b, and 3c are transmitted to the web server 1, and the activity levels of the groups 5a, 5b, and 5c are detected by the processing by the web server 1. For shooting with the spherical camera 3, it is sufficient that at least a part of the body of each of the six students is shot. Further, the position of the spherical camera 3 on the desk 4 may be changed by moving. However, in order to detect the activity, the omnidirectional camera 3 is installed at a position on the desk 4 so that the faces of all six students are included in the image, and the omnidirectional camera 3 is optically used. It is preferable to set an appropriate shooting magnification. In the present embodiment, since the number of raised hands of the student is detected as the activity level as described later, the spherical camera 3 is configured and arranged so that the state of the student's arm can be determined.

Further, since the spherical camera 3 has a built-in microphone, it is possible to transmit the collected ambient sound data to the web server 1. The web server 1 can also detect the number of utterances by the student 30 and the total volume (loudness of voice) uttered by the student 30 as the activity level based on the voice data uttered by the student 30 included in the received sound data. ..

Since the spherical camera 3 is provided for each group, the lesson support system 100 acquires images of students belonging to the group and audio data emitted by the students for each group, and based on the acquired images and audio data, Student activity can be detected in groups. In other words, the lesson support system 100 can detect the activity of the students for each group based on at least one of the images of the students belonging to the group and the voices emitted by the students.

<Hardware configuration example of Web server 1>
Next, the hardware configuration of the web server 1 will be described. FIG. 3 is a block diagram showing an example of the hardware configuration of the web server 1.

As shown in FIG. 3, the web server 1 is constructed by a computer, and has a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, and an HD (Hard Disdk). ) 504, an HDD (Hard Disk Drive) controller 505, a display 506, and an external device connection I / F (Interface) 508. It also includes a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

Of these, the CPU 501 controls the operation of the entire web server 1. The ROM 502 stores a program used to drive the CPU 501 such as an IPL. The RAM 503 is used as a work area of the CPU 501. The HD504 stores various data such as programs.

The HDD controller 505 controls reading or writing of various data to the HD 504 according to the control of the CPU 501. The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like.

The network I / F 509 is an interface for data communication using the network N. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG.

Further, the keyboard 511 is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions and the like. The pointing device 512 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like.

The DVD-RW drive 514 controls reading or writing of various data to the DVD-RW 513 as an example of the removable recording medium. In addition, it is not limited to DVD-RW, and may be DVD-R or the like. The media I / F 516 controls reading or writing (storage) of data to a recording medium 515 such as a flash memory.

<Hardware configuration example of smartphone 2>
Next, the hardware configuration of the smartphone 2 will be described. FIG. 4 is a block diagram showing an example of the hardware configuration of the smartphone 2.

As shown in FIG. 4, the smartphone 2 receives CPU 401, ROM 402, RAM 403, EEPROM 404, CMOS sensor 405, image sensor I / F 406, acceleration / orientation sensor 407, media I / F 409, and GPS reception. It is provided with a unit 411.

Of these, the CPU 401 controls the operation of the entire smartphone 2. The ROM 402 stores a program used to drive the CPU 401 such as the CPU 401 and the IPL. The RAM 403 is used as a work area for the CPU 401. The EEPROM 404 reads or writes various data such as a smartphone program under the control of the CPU 401.

The CMOS (Complementary Metal Oxide Semiconductor) sensor 405 is a kind of built-in imaging means for acquiring image data by imaging a subject (mainly a self-portrait) under the control of the CPU 401. Instead of a CMOS sensor, it may be an imaging means such as a CCD (Charge Coupled Device) sensor. The image sensor I / F 406 is a circuit that controls the drive of the CMOS sensor 405. The acceleration / orientation sensor 407 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect the geomagnetism. The media I / F 409 controls reading or writing (storage) of data to a recording medium 408 such as a flash memory. The GPS receiving unit 411 receives GPS signals from GPS satellites.

Further, the smartphone 2 includes a long-distance communication circuit 412, a CMOS sensor 413, an image sensor I / F 414, a microphone 415, a speaker 416, a sound input / output I / F 417, a display 418, and an external device connection I /. It includes an F419, a short-range communication circuit 420, an antenna 420a of the short-range communication circuit 420, and a touch panel 421.

Of these, the telecommunications circuit 412 is a circuit that communicates with other devices via the network N. The CMOS sensor 413 is a kind of built-in imaging means for capturing an image of a subject and obtaining image data under the control of the CPU 401.

The image sensor I / F 414 is a circuit that controls the drive of the CMOS sensor 413. The microphone 415 is a built-in circuit that converts sound into an electric signal. The speaker 416 is a built-in circuit that converts an electric signal into physical vibration to produce sounds such as music and voice. The sound input / output I / F 417 is a circuit that processes sound signal input / output between the microphone 415 and the speaker 416 under the control of the CPU 401.

The display 418 is a kind of display means such as a liquid crystal display or an organic EL (Electro Luminescence) for displaying an image of a subject, various icons, and the like. The external device connection I / F419 is an interface for connecting various external devices. The short-range communication circuit 420 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 421 is a kind of input means for operating the smartphone 2 by the user pressing the display 418.

<Hardware configuration example of spherical camera 3>
Next, the hardware configuration of the spherical camera 3 will be described. FIG. 5 is a block diagram showing an example of the hardware configuration of the spherical camera 3.

In the following, the omnidirectional camera 3 is an omnidirectional (omnidirectional) camera using two image pickup elements, but the number of image pickup elements may be two or more. In addition, it does not necessarily have to be a camera dedicated to omnidirectional shooting, and by attaching a retrofitted omnidirectional imaging unit to a normal digital camera, smartphone, etc., it has substantially the same function as the omnidirectional camera 3. You may.

As shown in FIG. 5, the spherical camera 3 includes an image pickup unit 601, an image processing unit 604, an image pickup control unit 605, a microphone 608, a sound processing unit 609, a CPU 611, and a ROM 612. .. In addition, SRAM (Static Random Access Memory) 613, DRAM (Dynamic Random Access Memory) 614, operation unit 615, external device connection I / F 616, long-distance communication circuit 617, antenna 617a, and acceleration / orientation sensor 618. A gyro sensor 619, an acceleration sensor 620, and a concave terminal 621 for Micro USB are included.

Of these, the image sensor 601 is a wide-angle lens (so-called fisheye lens) 602a, 602b each having an angle of view of 180 ° or more for forming a hemispherical image, and two provided corresponding to each wide-angle lens. It includes image pickup elements 603a and 603b.

The image pickup elements 603a and 603b include an image sensor such as a CMOS sensor or a CCD sensor that converts an optical image obtained by the fisheye lenses 602a and 602b into image data of an electric signal and outputs the image, and a horizontal or vertical synchronization signal or pixel clock of the image sensor. It is provided with a timing generation circuit for generating the image sensor and a group of registers in which various commands and parameters necessary for the operation of the image sensor are set.

The image sensors 603a and 603b of the image pickup unit 601 are connected to the image processing unit 604 by a parallel I / F bus, respectively. On the other hand, the image pickup elements 603a and 603b of the image pickup unit 601 are connected to the image pickup control unit 605 by a serial I / F bus (I2C bus or the like). The image processing unit 604, the image pickup control unit 605, and the sound processing unit 609 are connected to the CPU 611 via the bus 610. Further, ROM 612, SRAM 613, DRAM 614, an operation unit 615, an external device connection I / F (Interface) 616, a telecommunications circuit 617, an acceleration / direction sensor 618, and the like are also connected to the bus 610.

The image processing unit 604 takes in the image data output from the image pickup elements 603a and 603b through the parallel I / F bus, performs predetermined processing on each image data, and then synthesizes these image data. , Create data for a regular distance cylindrical projected image.

The image pickup control unit 605 generally uses the image pickup control unit 605 as a master device and the image pickup elements 603a and 603b as slave devices, and sets commands and the like in the register group of the image pickup elements 603a and 603b by using the I2C bus. Necessary commands and the like are received from the CPU 611. Further, the image pickup control unit 605 also uses the I2C bus to take in the status data and the like of the register group of the image pickup elements 603a and 603b and send them to the CPU 611.

Further, the image pickup control unit 605 instructs the image pickup devices 603a and 603b to output image data at the timing when the shutter button of the operation unit 615 is pressed. Some spherical cameras 3 may have a preview display function on a display (for example, a display of a smartphone) or a function corresponding to a moving image display. In this case, the image data is continuously output from the image sensors 603a and 603b at a predetermined frame rate (frames / minute).

Further, as will be described later, the image pickup control unit 605 also functions as a synchronization control means for synchronizing the output timings of the image data of the image pickup elements 603a and 603b in cooperation with the CPU 611. In the present embodiment, the spherical camera 3 is not provided with a display, but a display means may be provided.

The microphone 608 converts sound into sound (signal) data. The sound processing unit 609 takes in the sound data output from the microphone 608 through the I / F bus, and performs predetermined processing on the sound data.

The CPU 611 controls the overall operation of the spherical camera 3 and executes necessary processing. The ROM 612 stores various programs for the CPU 611. The SRAM 613 and the DRAM 614 are work memories, and store programs executed by the CPU 611, data in the middle of processing, and the like. In particular, the DRAM 614 stores image data during processing by the image processing unit 604 and data of the processed equirectangular projection image.

The operation unit 615 is a general term for operation buttons such as the shutter button 615a. By operating the operation unit 615, the user inputs various shooting modes, shooting conditions, and the like. This user is a vehicle used by an administrator who manages the lesson support system 100, or a teacher or student who uses the lesson support system 100.

The external device connection I / F 616 is an interface for connecting various external devices. The external device in this case is, for example, a USB memory, a PC, or the like. The equirectangular projection image data stored in the DRAM 614 is recorded on an external medium via the network I / F 616, or an external terminal (device) such as a smartphone via the network I / F 616 as needed. It is sent to.

The long-range communication circuit 617 uses short-range wireless communication technology such as Wi-Fi, NFC (Near Field Communication) and Bluetooth (registered trademark) via an antenna 617a provided on the spherical camera 3 to provide a smartphone or the like. Communicate with an external terminal (device). The long-distance communication circuit 617 can also transmit the equirectangular projection image data to an external terminal (device) such as a smartphone.

The acceleration / direction sensor 618 calculates the direction of the spherical camera 3 from the magnetism of the earth and outputs the direction information. This orientation information is an example of related information (metadata) along with Exif, and is used for image processing such as image correction of a captured image. The related information also includes each data of the shooting date and time of the image and the data capacity of the image data.

The acceleration / direction sensor 618 is a sensor that detects changes in angles (Roll angle, Pitch angle, Yaw angle) that accompany the movement of the spherical camera 3. The change in angle is an example of related information (metadata) along with Exif, and is used for image processing such as image correction of a captured image.

Further, the acceleration / direction sensor 618 is a sensor that detects acceleration in the three axial directions. The omnidirectional camera 3 calculates the posture (angle with respect to the direction of gravity) of its own device (omnidirectional camera 3) based on the acceleration detected by the acceleration / direction sensor 618. By providing the omnidirectional camera 3 with an acceleration / orientation sensor 618, the accuracy of image correction is improved.

<Example of functional configuration of class support system 100>
Next, the functional configuration of the lesson support system 100 will be described. FIG. 6 is a block diagram showing an example of the functional configuration of the lesson support system 100.

(Web server 1)
As shown in FIG. 6, the web server 1 in the lesson support system 100 includes a transmission / reception unit 11, an inference unit 12, an activity detection unit 13, an activity data storage unit 14, and a management unit 15. .. Of these, the function of the transmission / reception unit 11 is realized by the network I / F 509 or the like in FIG. 3, the function of the activity data storage unit 14 is realized by the HD 504 or the like in FIG. 3, and the inference unit 12 and the activity detection unit 13 are realized. The function is realized by the CPU 501 of FIG. 3 executing a predetermined program or the like.

The transmission / reception unit 11 transmits / receives signals and data to / from the smartphone 2 and the spherical camera 3. Specifically, when the teacher 20 operates the smartphone 2 to instruct the start of shooting by the spherical camera 3 at the start of the lesson, the transmission / reception unit 11 sends the instruction signal (start) of the start of shooting from the smartphone 2. Instructions) are accepted. Further, the transmission / reception unit 11 transmits a shooting start instruction signal to the spherical camera 3 in response to the shooting start instruction signal received from the smartphone 2.

Further, when the teacher 20 operates the smartphone 2 to instruct the end of shooting by the spherical camera 3 at the end of the lesson, the transmission / reception unit 11 receives an instruction signal (end) from the smartphone 2 to end shooting. Instructions) are accepted. Further, the transmission / reception unit 11 can output a shooting end instruction signal to the spherical camera 3 in response to the shooting end instruction signal received from the smartphone 2.

Further, the transmission / reception unit 11 transmits predetermined lesson information corresponding to the request for lesson information from the smartphone 2 to the smartphone 2. The transmission / reception unit 11 can transmit / receive various signals and data other than those described above.

The inference unit 12 receives the image captured by the spherical camera 3 via the transmission / reception unit 11, and determines whether or not the student has raised his / her hand in the captured image based on the learning data acquired in advance. Classify. The learning data is data obtained by machine learning patterns hidden in a large number of images taken by students acquired in advance.

Here, machine learning is a technique for making a computer acquire learning ability like a human being, and the computer autonomously generates an algorithm necessary for judgment such as data identification from learning data taken in advance. , Refers to a technology that applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-teacher learning, intensive learning, and deep learning, and may be a learning method that combines these learning methods. It doesn't matter what learning method you use.

The inference unit 12 cuts out a person (student) in the received image, classifies the states of raising or not raising hands of a plurality of students included in the image by inference based on the learning data, and detects the activity of the classification result. Output to unit 13.

The activity detection unit 13 detects the number of raised hands of students (the number of students raising hands) obtained from the classification result by the inference unit 12 as the activity level. The inference unit 12 outputs the classification result for each group based on the image taken by the spherical camera 3 to the activity detection unit 13, and the activity detection unit 13 is used as a terminal number for identifying the spherical cameras 3a, 3b, 3c. By counting the number of hands raised in association with each other, the number of hands raised in each group is detected.

Since the number of hands raised by students changes in chronological order, the reasoning unit 12 classifies the students by group from the streaming moving images of the students by the spherical camera 3 in chronological order, and the activity detection unit 13 classifies the students as activity. The number of hands raised is detected in chronological order. The "time series" here means that the classification by the inference unit 12 and the detection by the activity detection unit 13 are performed for each frame of the spherical camera 3. For example, if the frame rate of the spherical camera 3 is 30 fps (flames oer second), information on the number of raised hands of 30 can be obtained per second. However, the processing is not limited to each frame, and the classification by the inference unit 12 and the detection by the activity detection unit 13 may be performed for each predetermined number of frames of the spherical camera 3.

The activity detection unit 13 associates the information on the number of hands raised for each detected group with the time stamp indicating the time when the information on the number of hands raised is detected and the terminal number indicating the spherical camera in which the image was taken. It is output to the data storage unit 14 and stored.

Here, Table 1 below shows an example of the data stored in the activity data storage unit 14 of the web server 1.

表１の最も左側の列はタイムスタンプを示している。タイムスタンプは、授業を行っている最中のものであり、表１では２０２０年４月２０日１０時００分０１秒から２０２０年４月２０日１０時０５分１５秒までのタイムスタンプが示されている。但し、タイムスタンプの単位は、時刻が分かるものであれば何でもよく、ＵＮＩＸ（登録商標）時間等であってもよい。

The leftmost column in Table 1 shows the time stamps. The time stamps are during the class, and Table 1 shows the time stamps from 10:00:01 on April 20, 2020 to 10:05:15 on April 20, 2020. Has been done. However, the unit of the time stamp may be any unit as long as the time can be known, and may be UNIX (registered trademark) time or the like.

The middle column of Table 1 shows the terminal numbers of the spherical camera 3. For example, "1" represents the spherical camera 3a, and "2" represents the spherical camera 3b. The spherical camera 3a is for the group 5a, and the spherical camera 3b is for the group 5b. The rightmost column shows the number of hands raised.

In the example of Table 1, at 10:00:01 on April 20, 2020, the number of hands raised is "0" in the image taken by the spherical camera 3a, and 10:00:02 on April 20, 2020. Then, the image taken by the spherical camera 3b shows that the number of hands raised is "3".

In addition, the activity detection unit 13 displays the activity detection result using the information on the number of hands raised, the time stamp, and the terminal number stored in the activity data storage unit 14 in response to the request from the smartphone 2. The display screen to be displayed can be generated, and the generated display screen can be transmitted to the smartphone 2. The display screen will be described below with reference to FIG. 7.

The management unit 15 manages the lesson information registered by the teacher 20 or the manager of the lesson support system 100. This lesson information is information about the lesson such as the start / end time of the lesson, the location of the classroom to be held, the students (grade, class, number of people, etc.) to be given the lesson, and the number of all-sky cameras 3 installed in the lesson. This is information associated with the teacher 20 who conducts the lesson. In response to the request from the smartphone 2, the management unit 15 transmits the lesson information corresponding to the teacher 20 to the smartphone 2 via the transmission / reception unit 11.

(Smartphone 2)
As shown in FIG. 6, the smartphone 2 includes an operation unit 21, a transmission / reception unit 22, and a display unit 23. Of these, the function of the operation unit 21 is realized by the touch panel 421 or the like of FIG. 4, and the function of the transmission / reception unit 22 is realized by the long-distance communication circuit 412 or the short-range communication circuit 420 of FIG. Further, the function of the display unit 23 is realized by the display 418 or the like.

Application software for class support (hereinafter referred to as a class support application) is pre-installed on the smartphone 2. When the teacher 20 starts the lesson, the teacher 20 instructs the start of shooting by the spherical camera 3 by activating the lesson support application via the operation unit 21. In response to this instruction, the transmission / reception unit 22 transmits an instruction signal for starting shooting to the web server 1.

The operation unit 21 receives various operations on the smartphone 2 and various application software installed on the smartphone 2. The transmission / reception unit 22 sends / receives signals and data to / from the web server 1.

The display unit 23 displays the student hand-raising information for each group transmitted from the web server 1 on the screen of the display 418 of the smartphone 2. The display unit 23 displays, for example, a graph showing the time transition of the number of hands raised. By visually recognizing such a graph, the teacher can recognize which group of students is activated and which group of students is not activated.

(Spherical camera 3)
As shown in FIG. 6, the spherical camera 3 includes a transmission / reception unit 31, a terminal number storage unit 32, a photographing unit 33, and a sound collecting unit 34. The function of the transmission / reception unit 31 is realized by the long-distance communication circuit 617 or the like shown in FIG. 5, and the function of the terminal number storage unit 32 is realized by the ROM 612 or the like. Further, the function of the photographing unit 33 is realized by the image pickup unit 601 or the like shown in FIG. 5, and the function of the sound collecting unit 34 is realized by the microphone 608 or the like.

The omnidirectional camera 3 receives a shooting start instruction signal from the web server 1 via the transmission / reception unit 31 at the start of shooting, and starts shooting in response to the instruction signal. Further, the captured moving image is transmitted to the web server 1 via the transmission / reception unit 31. At the end of shooting, the web server 1 receives an instruction signal for the end of shooting, and the shooting ends in response to this instruction signal.

The transmission / reception unit 31 sends / receives signals and data to / from the web server 1. The terminal number storage unit 32 stores the terminal number for identifying the spherical cameras 3a, 3b, and 3c. The spherical camera 3 can transmit its own terminal number stored in the terminal number storage unit 32 together with the captured image to the web server 1 via the transmission / reception unit 31.

In response to the instruction signal for starting shooting from the web server 1, the shooting unit 33 captures a 360-degree omnidirectional image, and the sound collecting unit 34 collects sound. The captured image by the photographing unit 33 and the audio data by the sound collecting unit 34 are transmitted to the web server 1 via the transmitting / receiving unit 31.

<Display screen example>
FIG. 7 is a diagram showing an example of a display screen displayed by the display unit 23 of the smartphone 2. (A) is a diagram showing a UI (User Interface) screen 230a, and (b) is a diagram showing an activity detection result screen 230b.

On the UI screen 230a, a shooting start button 231 for instructing the start of shooting, a shooting end button 232 for instructing the end of shooting, and a result viewing button 233 for viewing the detection result of the activity are displayed. The teacher 20 can perform a touch operation or the like on these buttons.

In FIG. 7A, when the teacher 20 operates the shooting start button 231, the spherical camera 3 starts shooting, and when the teacher 20 operates the shooting end button 232, the spherical camera 3 ends shooting. do. When the teacher 20 operates the result viewing button 233, the activity detection result by the activity detection unit 13 is displayed.

When the teacher 20 operates the result viewing button 233, the detection result screen 230b of FIG. 7B is displayed on the screen of the smartphone 2.

Graph 234 shows the time transition of the cumulative number of hands raised by students for each group as the activity level. The solid line graph is the time transition of the cumulative number of hands raised by the students belonging to group 5a, the broken line graph is the time transition of the cumulative number of hands raised by the students belonging to group 5b, and the alternate long and short dash line graph is the time transition of the cumulative number of hands raised by the students belonging to group 5c. The transition is shown respectively. The horizontal axis of the graph 234 represents time, and each graph extends to the right with the passage of time.

Table 235 shows the number of hands raised in each group for each time stamp. The display is updated with the passage of time, and the number of hands raised in the group in which the hands were raised is displayed.

By visually recognizing the graph 234 and the table 235 on the screen of the smartphone 2 at hand, the teacher 20 can grasp the number of hands raised for each group, and can easily recognize which group is activated and which group is not activated. ..

The example of FIG. 7 is an example of the activity display method, and the display method can be appropriately modified according to the purpose and preference.

<Operation example of class support system 100>
Next, the operation of the lesson support system 100 will be described. FIG. 8 is a sequence chart showing an example of the operation of the lesson support system 100.

First, in step 81, the teacher 20 uses the smartphone 2 to perform an operation to start using the lesson support system 100. This system use operation corresponds to the operation of activating the lesson support application installed on the smartphone 2.

Subsequently, in step S82, the smartphone 2 requests the web server 1 to transmit the lesson information corresponding to the teacher 20 such as the identification information of the teacher 20 in response to the system use start operation.

Subsequently, in step S83, the web server 1 responds to the request and transmits the lesson information corresponding to the teacher 20 to the smartphone 2.

Subsequently, in step S84, the smartphone 2 displays the UI screen 230a (see FIG. 7) in response to the received lesson information. When there is a plurality of lesson information corresponding to the teacher 20, a list of a plurality of lessons is displayed once, and the UI screen 230a is displayed after the teacher 20 selects a desired lesson from the list.

Subsequently, in step S85, the teacher 20 performs an operation such as a touch operation on the shooting start button 231 (see FIG. 7) of the smartphone 2.

Subsequently, in step S86, the smartphone 2 outputs a shooting start instruction signal to the web server 1 in response to the operation to instruct the web server 1 to start shooting.

Subsequently, in step S87, the web server 1 instructs all the spherical cameras 3 used in the class to start shooting in response to the instruction signal.

Subsequently, in step S88, each of the spherical cameras 3 captures a still image or a streaming moving image of the students belonging to the group.

Subsequently, in step S89, each of the spherical cameras 3 transmits the captured image to the web server 1.

Subsequently, in step S90, the web server 1 performs a process of cutting out a person (student) from the received image.

Subsequently, in step S91, the web server 1 classifies whether or not the student raises his / her hand for each student that has been cut out.

Subsequently, in step S92, the web server 1 detects the number of raised hands of the students for each group.

Subsequently, in step S93, the web server 1 stores the detected number of hands raised information, the time stamp, and the terminal number in association with each other.

The above operations of steps S88 to S93 are repeated until the shooting is completed. The Loop period in the repetition is, for example, the frame rate of the spherical camera 3, but the period is not limited to this, and a period that is a multiple of the frame rate or the like can be appropriately selected.

Subsequently, in step S94, the teacher 20 performs an operation such as a touch operation on the result viewing button 233 (see FIG. 7) of the smartphone 2. The timing of this step S94 is an arbitrary timing at which the teacher 20 desires to view the detection result of the activity.

Subsequently, in step S95, the smartphone 2 requests the web server 1 to transmit the activity detection result in response to the result viewing operation.

Subsequently, in step S96, the web server 1 generates a detection result screen 230b (see FIG. 7) that displays the activity detection result in response to the request. At this time, the web server 1 generates the detection result screen 230b using the latest detection result of the activity of the corresponding lesson stored. Also, during class, polling is performed to acquire the latest activity detection result, and the detection result screen 230b is generated (updated) at a predetermined cycle.

Subsequently, in step S97, the web server 1 transmits the generated detection result screen 230b to the smartphone 2.

Subsequently, in step S98, the smartphone 2 displays the received detection result screen 230b on the display 418 (see FIG. 4) of the smartphone 2. The teacher 20 can visually recognize the displayed detection result screen 230b and recognize the activity level of the students for each group at a desired timing.

Subsequently, in step S99, the teacher 20 performs an operation such as a touch operation on the shooting end button 232. The timing of this step S99 is an arbitrary timing in which the teacher 20 desires the end of the lesson.

Subsequently, in step S100, the smartphone 2 outputs a shooting end instruction signal to the web server 1 in response to the operation to instruct the shooting end.

Subsequently, in step S101, the web server 1 instructs all the spherical cameras 3 used in the class to end the shooting in response to the instruction signal.

Subsequently, in step S102, the spherical camera 3 ends the shooting.

Note that FIG. 8 shows a flow in which steps S99 to S102 related to the end of shooting are performed after steps S94 to S98 related to viewing the detection result, but steps S99 to S102 are performed before steps S94 to S98 are performed. You may be broken.

In this way, the lesson support system 100 can detect the activity of the students for each group and display them on the screen of the smartphone 2 provided by the teacher 20.

<Effects of class support system 100>
Next, the operation and effect of the lesson support system 100 will be described.

Conventionally, when a teacher recognizes the number of hands raised by a student as the activity of a student such as a student in a class, the teacher can only recognize the activity of the student nominated by himself and keeps a record of the activity. It was difficult.

An information processing system that detects the student's raised hand and detects the student's activity level, and an information processing system that determines the student's proficiency level based on the student's image taken by the camera are disclosed. In these information processing systems, it may be difficult to detect the activity level of a student in each of a plurality of groups composed of the student.

In the present embodiment, the activity of the students is detected for each of a plurality of groups based on the images of the students belonging to the groups. Specifically, an omnidirectional camera 3 (shooting unit) is provided for each of the plurality of groups, and images of the students are photographed for each of the plurality of groups. The student's hand-raised state in the captured image is classified based on the learning data acquired in advance, and the number of student's hand-raised hands acquired from the classification result is detected as the activity. By doing so, it is possible to detect the activity of the students in each of a plurality of groups.

Since no photographing device or detection device for detecting the raised hand is provided for each student, it is possible to avoid complicating the system and making the system expensive. In addition, since it is not necessary to attach a sensor such as an acceleration sensor for detecting the raised hand to a part of the body, the student can take the lesson without feeling any trouble.

Further, in the present embodiment, since the smartphone 2 is provided with a display unit for displaying the activity detection result, the teacher 20 recognizes the student activity in each group by visually recognizing the display unit of the smartphone 2. can do. In the present embodiment, an example in which the display unit is provided on the smartphone 2 is shown, but the present invention is not limited to this. For example, a display device such as a display may be provided at a position where the teacher 20 can see, and the display device may include a display unit for displaying the detection result of the activity.

Further, in the present embodiment, since the web server is used as the information processing device, objects such as HTML (HyperText Markup Language) and images can be displayed in parallel on the screens of a plurality of terminals connected to the network N. , Not only teachers but also many others can see the activity of students in each group in class.

Further, in the present embodiment, since the photographing unit 33 captures an omnidirectional image of 360 degrees, it is possible to capture an image of the students belonging to the group in a wide field of view, the position of the student with respect to the photographing unit is restricted, and a plurality of students in one group. You can avoid the need for a camera.

Further, in the present embodiment, since the student's hand raising state in the captured image is classified based on the learning data acquired in advance, the number of student's hand raising is detected with high reliability and the activity is highly accurate. Can be detected.

Further, in the present embodiment, since the time transition of the number of hands raised by the students in each group is detected and displayed on the screen of the smartphone 2, the teacher 20 not only shows the activity level of the student each time, but also the time transition of the activity level. It can also be recognized.

In the present embodiment, an example of detecting the activity of the student based on the image taken by the spherical camera 3 is shown, but the sound emitted by the student is transmitted by using the sound collecting unit 34 of the spherical camera 3. Sound is collected, and the activity level of the students can be detected for each group based on the collected voice. If the activity is detected using both the number of hands raised by the student and the voice, the amount of information increases, so that the activity can be detected with higher reliability.

[Second Embodiment]
Next, the lesson support system 100a according to the second embodiment will be described. In the present embodiment, by performing classification by inference and detection of activity with a notebook PC provided for each group, it is not necessary to send the image of the student to the web server, thereby providing security and personal information protection. Provide a more suitable lesson support system from the viewpoint.

<Structure example of class support system 100a>
FIG. 9 is a diagram showing an example of the configuration of the lesson support system 100a. As shown in FIG. 9, the lesson support system 100a includes three notebook PCs 6a, 6b, and 6c. The notebook PCs 6a, 6b, and 6c are provided with wireless communication means or wired communication means, respectively, and are used as a web server 1, a smartphone 2, and a spherical camera via a network N such as a LAN (Local Area Network) or the Internet. It is connected to 3 so that it can communicate with each other.

The notebook PCs 6a, 6b, and 6c are examples of a plurality of information processing devices provided for each group. The lesson support system 100a is provided with three notebook PCs 6a, 6b, 6c in accordance with the three groups 5a, 5b, 5c. However, the number of notebook PCs is not limited to this, and the number of notebook PCs can be increased or decreased according to the increase or decrease in the number of groups.

The notebook PC 6a is installed on the desk 4a on which the students belonging to the group 5a are seated, the notebook PC 6b is installed on the desk 4b on which the students belonging to the group 5b are seated, and the notebook PC 6c is the desk on which the students belonging to the group 5ac are seated. It is installed on 4c.

The notebook PCs 6a, 6b, and 6c are each constructed by a computer and can have the hardware configuration shown in FIG. Further, in the following, when the notebook PCs 6a, 6b and 6c are not particularly distinguished, they are referred to as the notebook PC6, and the same applies to the configurations provided by each of the notebook PCs 6a, 6b and 6c.

<Example of functional configuration of class support system 100a>
Next, the functional configuration of the lesson support system 100a will be described. FIG. 10 is a block diagram showing an example of the functional configuration of the lesson support system 100a.

As shown in FIG. 10, the notebook PC 6 includes a transmission / reception unit 61, an inference unit 62, an activity detection unit 63, and a terminal number storage unit 64. Of these, the functions of the transmission / reception unit 61 are realized by the network I / F 509 and the like in FIG. 3, and the functions of the inference unit 62 and the activity detection unit 63 are realized by the CPU 501 in FIG. 3 executing a predetermined program or the like. NS. Further, the function of the terminal number storage unit 64 is realized by the HD504 or the like shown in FIG.

The transmission / reception unit 61 transmits / receives signals and data to / from the web server 1 and the spherical camera 3. Specifically, it is possible to receive the captured image by the spherical camera 3 and transmit the detection result of the activity based on the captured image and the terminal number for identifying the spherical camera 3 to the web server 1.

The function of the inference unit 62 is the same as the function of the inference unit 12 in FIG. 6, and the function of the activity detection unit 63 is the same as the function of the activity detection unit 13 in FIG.

Specifically, the inference unit 62a of the notebook PC 6a classifies the raised hand state with respect to the image captured by the spherical camera 3a, and the inference unit 62b of the notebook PC 6b raises the hand with respect to the image captured by the spherical camera 3b. The state is classified, and the reasoning unit 62c of the notebook PC 6c classifies the raised hand state with respect to the image captured by the spherical camera 3c.

Further, the activity detection unit 63a of the notebook PC 6a detects the activity from the classification result by the inference unit 62a, and the activity detection unit 63b of the notebook PC 6b detects the activity from the classification result by the inference unit 62b, and the notebook PC 6c The activity detection unit 63c detects the activity from the classification result by the inference unit 62c.

Since the notebook PC 6 has the functions of the inference unit and the activity detection unit in this way, the web server 1 does not have to have the functions of the inference unit and the activity detection unit. Further, since the web server 1 does not have to have the function of the inference unit, it is not necessary to receive the image captured by the spherical camera 3.

The function of the terminal number storage unit 64 is the same as the function of the terminal number storage unit 32 in FIG. Therefore, the spherical camera 3 does not have to have the function of the terminal number storage unit.

<Effects of class support system 100a>
The class support system 100 shown in the first embodiment transmits an image captured by the spherical camera to the web server 1, and the web server 1 performs classification by inference and detection of activity. Since the web server 1 can provide images to the browsers of all terminals such as PCs connected to the network N, images including students such as students may be provided to these terminals, and security and personal information may be provided. It may not be preferable from the viewpoint of protection.

Further, if the processing of classification by inference and detection of activity is performed only by the web server 1, the processing load may increase when the number of students or the number of groups is large, and the processing time may increase.

In the present embodiment, since the notebook PC 6 provided for each group performs the classification by inference and the detection of the activity, it is not necessary to transmit the image of the student to the web server 1. This makes it possible to provide a lesson support system that is more suitable from the viewpoint of security and personal information protection. Further, since the processing can be distributed and executed by a plurality of notebook PCs 6, the processing load can be reduced and the processing time can be shortened even when the number of students or the number of groups is large.

In the present embodiment, an example in which one notebook PC 6 is provided for each group is shown, but the present invention is not limited to this, and two or more notebook PCs may be provided for each group. A number of notebook PCs smaller than the number of groups may be provided by processing the images taken by the students in one group with one notebook PC. Further, an information processing device other than the notebook PC may be provided instead of the notebook PC.

Further, instead of the web server 1, an information processing device such as a desktop PC may be provided, and the desktop PC may be used as a master unit and the notebook PC 6 may be used as a slave unit to distribute processing such as classification by inference and detection of activity. .. With such a configuration, the processing load can be reduced and the processing time can be shortened.

The effects other than those described above are the same as those described in the first embodiment.

[Third Embodiment]
Next, the lesson support system 100b according to the third embodiment will be described.

In the present embodiment, the individual authentication of the student is performed to identify the student who raised his / her hand and detect the activity of each student as well as the group. Teachers will be able to recognize the activity of each student as well as the group.

As the configuration of the lesson support system 100b, any configuration of the lesson support system 100 according to the first embodiment and the lesson support system 100a according to the second embodiment can be applied. In the present embodiment, a case where the web server has an authentication function in the same configuration as the lesson support system 100 will be described as an example.

<Example of functional configuration of class support system 100b>
FIG. 11 is a block diagram illustrating an example of the functional configuration of the lesson support system 100b. As shown in FIG. 11, the web server 1b in the lesson support system 100b includes an authentication information storage unit 16 and an authentication unit 17. Of these, the function of the authentication information storage unit 16 is realized by the HD504 or the like shown in FIG. 3, and the function of the authentication unit 17 is realized by the CPU 501 of FIG. 3 executing a predetermined program or the like.

The authentication information storage unit 16 stores in advance authentication information for authenticating a student who takes a lesson using the lesson support system 100b. This authentication information is, for example, a face image for each student for performing face recognition. However, since it is sufficient that the image taken by the spherical camera 3 can be used for authentication, information on the body other than the face may be stored in the authentication information storage unit 16 and used for authentication. Further, when the activity level is detected by voice, the student's voice information can be stored in the authentication information storage unit 16 in order to perform the authentication using the voiceprint or the like, and can be used for the voiceprint authentication.

The authentication unit 17 inputs an image captured by the spherical camera 3 via the transmission / reception unit 11 and cuts out a face image of a person in the image. Then, the cut out face image and the face image stored in the authentication information storage unit 16 are collated to execute the authentication process. The authentication result is output to the activity detection unit 13 as personal identification information or the like, and the activity detection unit 13 authenticates by associating the personal identification information and detecting the number of hands raised (number of hands raised) as the activity. It is possible to detect the activity level of each student certified by the part 17. As a result, the activity level of each student can be detected and displayed on the display unit 23 of the smartphone 2 so that the teacher can visually recognize it.

In this way, in the present embodiment, by performing personal authentication of the student, the student who raised his / her hand can be identified, and the activity of each student as well as the group can be detected. Teachers will be able to recognize the activity of each student as well as the group.

The effects other than those described above are the same as those described in the first embodiment.

Although the embodiments have been described above, the present invention is not limited to the above-described embodiments specifically disclosed, and various modifications and changes can be made without departing from the scope of claims. ..

For example, in the above-described embodiment, the lesson support system is shown as an example of the information processing system, but the support target is not limited to the lesson. The information processing system according to this embodiment can be applied to various seminars, product briefings, various presentations, and the like.

Further, in the present embodiment, an example in which the lesson support system 100 is provided with the spherical camera 3 is shown, but the lesson support system 100 is not provided with the spherical camera 3, and at least one of the image or the sound of the student is externally provided. It can also be configured to receive from the device.

Further, storage units such as the activity data storage unit 14, the authentication information storage unit 16, and the terminal number storage unit 32 are provided outside the lesson support system 100, and the lesson support system 100 is configured to receive the information stored by these. May be good.

The numbers such as the ordinal number and the quantity used above are all exemplified for the purpose of concretely explaining the technique of the present invention, and the present invention is not limited to the exemplified numbers. Further, the connection relationship between the constituent elements is exemplified for concretely explaining the technique of the present invention, and the connection relationship for realizing the function of the present invention is not limited to this.

Further, the division of blocks in the functional block diagram is an example, and even if a plurality of blocks are realized as one block, one block is divided into a plurality of blocks, and / or some functions are transferred to another block. good. In addition, a single piece of hardware or software may process the functions of a plurality of blocks having similar functions in parallel or in a time division manner.

Further, each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array) and conventional circuit modules.

The embodiment also includes an information processing method. For example, the information processing method is an information processing method by an information processing system including an information processing device and a terminal communicably connected to the information processing device, and is an image of a student belonging to a group or the student. A step of detecting the activity of the student is performed in each of a plurality of groups based on at least one of the voices emitted by the student. By such an information processing method, the same effect as the above-mentioned lesson support system can be obtained.

The embodiment also includes a program. For example, the program is a program executed by an information processing system including an information processing device and a terminal communicatively connected to the information processing device, and an image of a student belonging to a group or the student A computer is made to execute a process of detecting the activity of the student in each of a plurality of groups based on at least one of the emitted voices. With such a program, the same effect as the above-mentioned lesson support system can be obtained.

1 Web server (an example of information processing device)
2 Smartphone (example of terminal)
3 Spherical camera 4 Desk 5 Group 6 Notebook PC (example of multiple information processing devices)
10 Classroom 11 Transmission / reception unit 12 Reasoning unit 13 Activity detection unit 14 Activity data storage unit 15 Management unit 16 Authentication information storage unit 17 Authentication unit 21 Operation unit 22 Transmission / reception unit 23 Display unit 31 Transmission / reception unit 32 Terminal number storage unit 33 Imaging unit 34 Sound collecting unit 61 Transmission / reception unit 62 Inference unit 63 Activity detection unit 64 Terminal number storage unit 100 Class support system (example of information processing system)
230a UI screen 230b Detection result screen 234 Graph 235 Table 608 Microphone N network

Japanese Unexamined Patent Publication No. 2005-346016 JP-A-2017-102153

Claims (13)

An information processing system including an information processing device and a terminal communicatively connected to the information processing device.
An information processing system including an activity detection unit that detects the activity of the student in each of a plurality of groups based on at least one of the image of the student belonging to the group or the voice emitted by the student. The information processing system according to claim 1, further comprising a display unit that displays the detection result of the activity. The information processing device is a web server.
The information processing system according to claim 1 or 2, wherein the activity detection unit is provided in the web server. The information processing system according to any one of claims 1 to 3, further comprising a photographing unit that captures images of the students in groups of a plurality of groups. The information processing system according to claim 4, wherein the photographing unit captures a 360-degree omnidirectional image. It is provided with an inference unit that classifies the raised hand state of the student in the image based on the learning data acquired in advance.
The information processing system according to claim 4 or 5, wherein the activity detection unit detects the number of hands raised by the student obtained from the classification result by the inference unit as the activity. The information processing system according to any one of claims 1 to 6, further comprising a sound collecting unit that collects sounds emitted by the students in groups of a plurality of groups. The information processing system includes a plurality of information processing devices.
The information processing system according to any one of claims 1 to 7, wherein the activity detection unit is provided in each of the plurality of information processing devices. Equipped with an authentication unit that authenticates the students
The information processing system according to any one of claims 1 to 8, wherein the activity detection unit detects the activity for each student authenticated by the authentication unit. The information processing system according to any one of claims 1 to 9, wherein the activity detection unit detects a time transition of the activity. The information according to any one of claims 1 to 10, wherein the activity detection unit starts detection of the activity in response to a start instruction and ends detection of the activity in response to an end instruction. Processing system. An information processing method using an information processing system including an information processing device and a terminal communicatively connected to the information processing device.
An information processing method that performs a step of detecting the activity of the student in each of a plurality of groups based on at least one of the image of the student belonging to the group or the voice emitted by the student. A program executed by an information processing system including an information processing device and a terminal communicatively connected to the information processing device.
A program that causes a computer to execute a process of detecting the activity of a student in each of a plurality of groups based on at least one of an image of a student belonging to the group or a voice emitted by the student.

Images