WO2022024487A1 - 解析システム、解析システムの制御プログラム、制御プログラム、および解析システムの制御方法 - Google Patents

解析システム、解析システムの制御プログラム、制御プログラム、および解析システムの制御方法 Download PDF

Info

Publication number
WO2022024487A1
WO2022024487A1 PCT/JP2021/016969 JP2021016969W WO2022024487A1 WO 2022024487 A1 WO2022024487 A1 WO 2022024487A1 JP 2021016969 W JP2021016969 W JP 2021016969W WO 2022024487 A1 WO2022024487 A1 WO 2022024487A1
Authority
WO
WIPO (PCT)
Prior art keywords
small data
analysis
data
server
certainty
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/016969
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
智也 岡▲崎▼
希武 田中
直樹 池田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Konica Minolta Inc
Original Assignee
Konica Minolta Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Konica Minolta Inc filed Critical Konica Minolta Inc
Priority to JP2021576462A priority Critical patent/JP7044215B1/ja
Priority to US18/006,865 priority patent/US12518534B2/en
Publication of WO2022024487A1 publication Critical patent/WO2022024487A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/033—Recognition of patterns in medical or anatomical images of skeletal patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training

Definitions

  • the present invention relates to an analysis system, an analysis system control program, a control program, and an analysis system control method.
  • the object area is extracted from the image data frame constituting the captured image data, the object type is identified based on the first feature amount in the object area, and the second feature amount in the object area is identified.
  • the object type certainty is calculated based on.
  • the data frame is transmitted to the server and stored in association with the object type and the object type conviction. Then, according to the user's request, the image of the object corresponding to the designated object type and the object type conviction are displayed in descending order of the object type conviction. This makes it possible to efficiently search for a specific type of object from the video without omission.
  • the present invention has been made to solve such a problem. That is, analysis systems, analysis system control programs, control programs, and analysis systems that can obtain new data with high reliability and added value by transmitting small data with reduced size and analyzing at the destination.
  • the purpose is to provide a control method.
  • the present inventors solve the problems of communication volume, privacy protection, and data management cost by transmitting small data (processed data) with reduced size to the server side without transmitting the original data to the server side.
  • small data processed data
  • An analysis system having a terminal and a server, wherein the terminal has an acquisition unit for acquiring original data, small data having a size smaller than the original data, and certainty of the small data.
  • the server has a conversion unit that converts at least the small data into, and a transmission unit that transmits at least the small data to the server.
  • the server has a receiving unit that receives at least the small data from the terminal, and a plurality of received units.
  • An analysis system having an analysis unit that executes analysis using small data and an output unit that outputs analysis results by the analysis unit.
  • the transmitting unit transmits the combination of the small data and the certainty of the small data
  • the receiving unit receives the combination of the small data and the certainty of the small data.
  • the analysis system according to (1) above wherein the analysis unit executes an analysis using the plurality of the small data and the certainty of each of the plurality of small data.
  • the analysis unit weights the small data using the certainty of the small data, and executes an analysis using the plurality of weighted small data, according to the above (3) or ().
  • the analysis system according to 4 The analysis system according to 4).
  • the analysis unit executes an analysis based on the plurality of the small data and the certainty of each of the plurality of the small data using the machine-learned trained model. Described analysis system.
  • the conversion unit converts the original data into the small data and the certainty of the small data by using the machine-learned trained model, and the small data corresponding to the original data.
  • the control unit further has a reception unit that accepts the correct answer of the above, and the control unit relearns the trained model using the combination of the original data and the correct answer of the small data corresponding to the original data as teacher data.
  • control unit changes the first threshold value based on the small data, the certainty degree, and the correct answer.
  • the analysis unit has the small data and the certainty degree, and the certainty degree by the analysis unit based on the small data and the correct answer.
  • the analysis unit has a plurality of said parts converted from a plurality of the original data by the conversion unit.
  • a trained model of the analysis unit is generated in which the certainty of each of the small data and the plurality of small data, and the combination of the correct answers of the analysis results corresponding to the plurality of original data are learned as teacher data.
  • a control method for an analysis system having a terminal and a server wherein the original data is acquired in the terminal (a), the original data is divided into small data having a size smaller than the original data, and the said.
  • a control method for an analysis system comprising (d) and a step (e) for outputting the analysis result in the step (d).
  • FIG. 1 is a diagram showing a schematic configuration of the analysis system 10.
  • the example of the analysis system 10 shown in FIG. 1 includes a terminal 100, a photographing device 200, a communication network 300, and a server 400.
  • the photographing device 200 is an example of an edge device.
  • the edge device includes an image sensor, a microphone, and a scanner, in addition to the photographing device 200.
  • Microphones include speakers that support interactive voice operations, such as smart speakers, that have a built-in microphone.
  • the data acquired by the edge device includes image data, sound data (including data whose sound is frequency-analyzed), tactile data, olfactory data, documents (text data, etc.), and spoken language (spoken voice). (Including words, etc.) is included.
  • the data acquired by the edge device constitutes the original data.
  • the image captured by the photographing apparatus 200 (hereinafter, also simply referred to as “photographed image 250”) (see FIGS. 5 and 6) is an example of the original data.
  • the original data will be described assuming that the captured image is 250.
  • Terminal 100 The terminal 100 is connected to the photographing device 200 so as to be able to communicate with each other by the communication network 300.
  • the terminal 100 may be built in the photographing device 200.
  • the terminal 100 is further communicably connected to the server 400 via the Internet 600.
  • FIG. 2 is a block diagram showing the hardware configuration of the terminal 100.
  • the terminal 100 includes a control unit 110, a storage unit 120, a display unit 130, an input unit 140, and a communication unit 150. These components are connected to each other via a bus.
  • the control unit 110 is configured by a CPU (Central Processing Unit), and controls and performs arithmetic processing of each unit of the terminal 100 according to a program.
  • a CPU Central Processing Unit
  • the storage unit 120 may be composed of a RAM (Random Access Memory), a ROM (Read Only Memory), and an SSD (Solid State Drive).
  • the RAM temporarily stores programs and data as a work area of the control unit 110.
  • the ROM stores various programs and various data in advance.
  • the SSD stores various programs including an operating system and various data.
  • the display unit 130 is, for example, a liquid crystal display and displays various information.
  • the input unit 140 is composed of, for example, a touch panel and various keys.
  • the input unit 140 is used for various operations and inputs.
  • the communication unit 150 is an interface for communicating with an external device.
  • a network interface according to standards such as Ethernet (registered trademark), SATA, PCI Express, USB, and IEEE 1394 can be used.
  • a wireless communication interface such as Bluetooth (registered trademark), 802.11, or 4G may be used for communication.
  • the photographing device 200 is arranged, for example, on the ceiling or the upper part of the wall of the living room of the subject 500, or is arranged on the ceiling or the upper part of the wall of a store, a factory, or the like.
  • the photographing device 200 photographs a predetermined imaging range from a position where the subject 500 can be overlooked, and outputs a captured image 250.
  • the captured image 250 includes an image including the subject 500.
  • the photographing apparatus 200 includes a wide-angle camera.
  • the photographing device 200 may be a stereo camera capable of photographing a three-dimensional photographed image 250.
  • the photographing device 200 can photograph a shooting area as a shot image 250 of a moving image having a frame rate of, for example, 15 fps to 30 fps.
  • the captured image 250 includes a moving image and a still image.
  • the captured image 250 is, for example, a black-and-white image and may be an image of 128 pixels ⁇ 128 pixels.
  • the captured image 250 may be a color image.
  • the photographing device 200 transmits the photographed image 250 to the terminal 100.
  • Communication network 300 For the communication network 300, a network interface based on a wired communication standard such as Ethernet (registered trademark) may be used. For the communication network 300, a network interface based on a wireless communication standard such as Bluetooth (registered trademark) or 802.11 may be used.
  • a wired communication standard such as Ethernet (registered trademark)
  • a network interface based on a wireless communication standard such as Bluetooth (registered trademark) or 802.11 may be used.
  • FIG. 3 is a block diagram showing a hardware configuration of the server 400.
  • the server 400 includes a control unit 410, a storage unit 420, and a communication unit 430. These components are connected to each other via a bus.
  • the server 400 constitutes an analysis device.
  • the communication unit 430 constitutes a reception unit. Since the control unit 410, the storage unit 420, and the communication unit 430, which are the components of the server 400, are the same as the corresponding components of the terminal 100, the description thereof will be omitted.
  • the control unit 410 constitutes an analysis unit.
  • the control unit 410 constitutes an output unit together with the communication unit 430.
  • the server 400 is, for example, a cloud server virtually configured by a plurality of servers arranged on the Internet 600.
  • the server 400 may be configured by one computer.
  • FIG. 4 is a functional block diagram showing the functions of the control unit 110 and the server 400, which are a part of the terminal 100.
  • FIG. 5 is a diagram showing a person area 252 and a peripheral object area 253 on the captured image 250.
  • FIG. 6 is a diagram showing a joint point 251 on the captured image 250.
  • the terminal 100 includes a conversion unit 111 and a transmission unit 112 as functions of the control unit 110.
  • the control unit 110 also functions as an acquisition unit for acquiring the original data from the edge device.
  • the conversion unit 111 refers to the captured image 250, which is the original data, as small data (also referred to as processed data) having a smaller size (data amount) than the original data and the certainty of the small data (hereinafter, simply referred to as “confidence”). ) And.
  • the conversion unit 111 has learned a neural network (hereinafter referred to as “NN”) in which the combination of the captured image 250 which is the original data and the correct answer of the small data corresponding to the captured image 250 is learned as the teacher data. Using the model, the captured image 250, which is the original data, is converted into small data and certainty.
  • NN neural network
  • the small data includes data such that the object cannot be identified in the image, specifically, each coordinate of the joint point 251 (hereinafter, also simply referred to as “joint point 251”) and an object of each category.
  • the position of the region (hereinafter referred to as “object region”) is included.
  • the small data will be described as being at least one of a human joint point 251 and a non-human object region (hereinafter, also simply referred to as “peripheral object region 253”).
  • the conversion unit 111 converts the captured image 250, which is the original data, into the small data and the certainty of the small data as follows.
  • the conversion unit 111 detects an object area in which an object exists from the photographed image 250 (more specifically, each frame of the photographed image 250) acquired by receiving from the photographing apparatus 200, and determines the detected object area. Estimate the category (class) of the contained object.
  • the object categories include, for example, people, wheelchairs, beds, and walkers.
  • the object category can be estimated by calculating the likelihood for each predetermined category including a person for each object area and specifying the category with the highest likelihood.
  • the detection of the object region and the calculation of the likelihood for each category can be performed by using a known machine learning technique using NN such as Faster R-CNN.
  • the object category is added as a tag to the small data of the peripheral object area 253 and used for analysis in the server 400.
  • the wheelchair, the bed, and the walker are detected as the peripheral object area 253 on the captured image 250.
  • the target person 500 who is a resident is detected as an object area (hereinafter, referred to as "person area 252") in which the object category is "person”.
  • person area 252 an object area in which the object category is "person”.
  • the behavior of the subject 500 trying to get into the wheelchair is assumed.
  • the conversion unit 111 further detects the joint point 251 for the person area 252.
  • the joint point 251 can generate a heat map for each individual joint point 251 based on the person region 252, and can be estimated for each individual joint point 251 as the coordinates having the highest likelihood.
  • the heat map can be generated, for example, by a known machine learning technique using an hourglass network.
  • the conversion unit 111 detects the joint point 251 including the height information and the depth information.
  • the conversion unit 111 may detect the posture of the subject 500 based on the joint point 251 and further using a known machine learning technique using NN. The detected posture is added as a tag to the small data of the joint point 251 and can be used for analysis on the server 400.
  • two joint points 251 are detected respectively.
  • the joint point 251 of the resident who is the subject 500 is shown in black.
  • the joint point 251 of the care staff, which is another subject 500 is shown in gray. From the relationship between the two joint points 251, it is assumed that one subject 500 is receiving care from the other subject 500.
  • the conversion unit 111 calculates the conviction of small data.
  • the certainty is the average value of the maximum likelihood values in the heat map of each joint point 251 or the importance of each joint point 251 (for example, the joint point 251 of the head has the highest importance. It can be a weighted mean value that takes into account (which can be set).
  • the conviction may be the total conviction of the entire plurality of joint points 251 (combination of the joint points 251) detected from one captured image 250 (frame).
  • the certainty may be the individual certainty of the plurality of joint points 251 detected from one captured image 250 (frame).
  • the conviction can be the likelihood for each category.
  • the transmission unit 112 controls the transmission of small data and certainty by the communication unit 150 to the server 400. Specifically, the transmission unit 112 transmits small data having a certainty degree of 1st threshold value or more to the server 400 and transmissions small data having a certainty degree of less than the 1st threshold value to the server 400 without transmitting the original data to the server side. Do not control. As a result, it is possible to send small data with a certain degree of certainty or higher without sending the original data to the server side, and it is possible to solve problems of communication volume, privacy protection, data management cost, and even higher accuracy. Small data can be sent to the server side.
  • the combination of the small data (1) and the conviction (1), the combination of the small data (2) and the conviction (2), and the combination of the small data (3) and the conviction (3) are temporal. Small data and convictions converted from the frames of the captured image 250 adjacent to, respectively.
  • the conviction (2) of the small data (2) is less than the first threshold value, the combination of the small data (1) and the conviction (1), and the small data (3) and the conviction. It is controlled that the combination of (3) is transmitted to the server 400, and the combination of the small data (2) and the certainty (2) is not transmitted to the server 400.
  • the transmission unit 112 may detect a predetermined action (event) of the target person 500 and selectively transmit only the small data of the captured image 250 in which the predetermined action is detected to the server 400.
  • the predetermined behavior can be detected by a known method based on the plurality of joint points 251 detected from the plurality of frames of the captured image 250, respectively.
  • the predetermined behavior may be detected based on the plurality of joint points 251 using, for example, a trained model of the NN trained as a combination teacher data of the plurality of joint points 251 and the correct answer of the behavior. In this case, out of the small data of the captured image 250 in which the predetermined behavior is detected, only the small data having a certainty degree of the first threshold value or more can be transmitted to the server 400.
  • FIG. 7 is an explanatory diagram showing an example in which small data is selectively transmitted from the terminal 100 to the server 400.
  • predetermined actions turning over, getting up, and getting into a wheelchair are exemplified.
  • Predetermined actions can include falls and falls.
  • the analysis by the function f is executed using the received small data (1) and the small data (3), which are a plurality of (two) small data, and the analysis result can be accessed by the server 400. Output by sending to the terminal, etc.
  • the server 400 can analyze the activity amount of the subject 500 who is a resident based on, for example, the joint point 251 of the small data (1) and the joint point 251 of the small data (3).
  • the server 400 can output the analysis result based on the plurality of small data by using the trained model of the NN trained using the combination of the plurality of small data and the correct answer of the analysis result as the teacher data.
  • the server 400 may output an analysis result based on a plurality of small data by multiple regression analysis.
  • the function f uses a plurality of small data (1) to small data (3), and the activity amount, sleep time, nighttime getting out of bed (number of times, time), wake-up time, walking speed, abnormal behavior, and It can be a function that analyzes behavior patterns.
  • the function f may be a function that analyzes the posture of the subject 500.
  • the small data is, for example, the joint point 251 but may be a combination of a plurality of joint points 251 detected from one captured image 250 (frame). It may be an individual joint point 251 out of a plurality of joint points 251 detected from the above. Further, the small data may be a combination of a plurality of joint points 251 detected from one captured image 250 (frame) and a combination of a peripheral object region 253.
  • the original data is the data of a document related to chemical substance regulation and the small data (n) converted from the data of the document is the data of the regulation content, it can be described in the future by the calculation by the function f. Chemicals may be analyzed.
  • the analysis result by the server 400 includes, for example, sleeping time, getting out of bed at night (number of times, time), wake-up time, walking speed, abnormal behavior, and behavior pattern in addition to the above-mentioned amount of activity.
  • 8 to 10 are explanatory diagrams showing an example of using the analysis result by the server 400.
  • the daytime care plan is changed, such as increasing the daytime exercise amount of the subject 500 by using the analysis result, nighttime getting out of bed, nighttime activity amount, and sleep time. There is. As a result, the burden on the night staff can be reduced.
  • the care record may be transmitted from the care staff's mobile terminal or the like to the server 400 and stored in the storage unit 420.
  • the cognitive decline of the subject 500 was detected at an early stage.
  • the QOL of residents and staff can be improved.
  • FIG. 11 is a flowchart showing the operation of the analysis system 10. This flowchart can be executed by causing the terminal 100 and the server 400 to operate in cooperation with each other by a program executed by the control unit 110 of the terminal 100 and the control unit 410 of the server 400.
  • the control unit 110 acquires the captured image 250, which is the original data, by receiving it from the photographing device 200 (S101).
  • the control unit 110 converts the captured image 250, which is the original data, into small data such as the joint point 251 and the certainty of the small data (S102).
  • the control unit 110 transmits the joint point 251 and the like, which are small data whose conviction is equal to or higher than the first threshold value, to the server 400 without transmitting the original data to the server side (S103).
  • the control unit 410 executes an analysis using a plurality of small data such as joint points 251 received by the server 400 (S104).
  • the control unit 410 can store the received small data in the storage unit 420 and execute the analysis using the small data for a predetermined period.
  • the predetermined period may be, for example, one day, one week, January, June, or one year, depending on the analysis content.
  • the control unit 410 outputs the analysis result (S105).
  • the second embodiment will be described.
  • the difference between this embodiment and the first embodiment is as follows.
  • the transmission of small data from the terminal 100 to the server 400 is controlled based on the degree of certainty, and the server 400 performs an analysis using the received small data.
  • a combination of small data and certainty is transmitted from the terminal 100 to the server 400 regardless of the certainty, and the server 400 executes an analysis using the small data and the certainty. Since the present embodiment is the same as the first embodiment in other respects, duplicate description will be omitted or simplified.
  • FIG. 12 is a functional block diagram showing the functions of the control unit 110 and the server 400, which are a part of the terminal 100.
  • the terminal 100 includes a conversion unit 111 and a transmission unit 112 as functions of the control unit 110.
  • the conversion unit 111 converts the captured image 250, which is the original data, into small data and certainty.
  • the transmission unit 112 transmits the small data to the server 400 in combination with the conviction.
  • the combination of the small data (1) and the certainty (1), the combination of the small data (2) and the certainty (2), and the combination of the small data (3) and the certainty (3) are temporal. It is a combination of small data and certainty converted from each frame of the captured image 250 adjacent to.
  • not transmitting the original data to the server side is the same as in the first embodiment, but the small data (1) and the certainty (1) converted from the captured image 250 are not related to the certainty.
  • the combination of small data (2) and certainty (2), and the combination of small data (3) and certainty (3) are transmitted to the server 400. That is, all the combinations of small data and certainty converted from the captured image 250 are transmitted to the server 400.
  • small data can be sent with certainty without sending the original data to the server side, problems of communication volume, privacy protection, data management cost can be solved, and the accuracy is high on the server side. Can process data.
  • the server 400 executes an analysis using the received small data and the certainty, and outputs the analysis result. Specifically, the server 400 converts the small data (n) into small data (n)'(n is, for example, 1 to 3) based on the certainty (n) by, for example, the function g, and the function. By f, analysis using a plurality of converted small data (1)'to small data (3)' is executed.
  • the function g can be, for example, an operation of selection based on the certainty of small data. That is, the function g may be a function that does not output small data whose conviction is less than the second threshold value to the arithmetic module of the subsequent function f.
  • the function g may be a function of weighting small data according to the degree of certainty.
  • the function f uses a plurality of converted small data (1)'to small data (3)', and uses the activity amount, sleeping time, nighttime leaving (number of times, time), wake-up time, and walking speed of the subject 500.
  • Abnormal behavior and can be a function that analyzes behavior patterns.
  • the function f may be a function that analyzes the posture of the subject 500.
  • the server 400 uses a trained model of NN trained using a combination of a plurality of small data and convictions and a correct answer of a predetermined analysis result as teacher data, and analyzes the analysis results based on the plurality of small data and convictions. May be output.
  • the server 400 may output the analysis result based on a plurality of small data and certainty by multiple regression analysis.
  • the small data is, for example, the joint point 251 but may be a combination of a plurality of joint points 251 detected from one captured image 250 (frame). It may be an individual joint point 251 out of a plurality of joint points 251 detected from the above. Further, the small data may be a combination of a plurality of joint points 251 detected from one captured image 250 (frame) and a combination of a peripheral object region 253.
  • the original data is the data of a document related to chemical substance regulation and the small data (n) converted from the data of the document is the data of the regulation content
  • the calculation by the function g and the function f will be performed in the future.
  • the chemicals that can be described may be analyzed.
  • FIG. 13 is a flowchart showing the operation of the analysis system 10. This flowchart can be executed by causing the terminal 100 and the server 400 to operate in cooperation with each other by a program executed by the control unit 110 of the terminal 100 and the control unit 410 of the server 400.
  • the control unit 110 acquires the captured image 250, which is the original data, by receiving it from the photographing device 200 (S201).
  • the control unit 110 converts the captured image 250, which is the original data, into small data such as the joint point 251 and the certainty of the small data (S202).
  • the control unit 110 transmits the small data such as the joint point 251 to the server 400 in combination with the conviction without transmitting the original data to the server side (S203).
  • the control unit 410 executes an analysis using a plurality of small data such as joint points 251 and certainty received by the server 400 (S204).
  • the control unit 410 can store the received small data and the certainty in the storage unit 420, and execute the analysis using the small data and the certainty for a predetermined period.
  • the predetermined period may be, for example, one day, one week, January, June, or one year, depending on the analysis content.
  • the control unit 410 outputs the analysis result (S205).
  • the third embodiment will be described.
  • the difference between this embodiment and the first embodiment is as follows.
  • the trained model of the conversion unit 111 is not retrained.
  • the trained model of the conversion unit 111 is relearned using the combination of the original data and the correct answer of the small data corresponding to the original data as the teacher data.
  • the conviction for small data becomes more accurate, so that the accuracy of the analysis result is improved. Since the present embodiment is the same as the first embodiment in other respects, duplicate description will be omitted or simplified.
  • FIG. 14 is a functional block diagram showing the functions of the control unit 110, the reception unit 160, and the server 400, which are a part of the terminal 100.
  • the terminal 100 includes a conversion unit 111 and a transmission unit 112 as functions of the control unit 110.
  • the conversion unit 111 converts the captured image 250, which is the original data, into small data and certainty using the trained model of NN.
  • the reception unit 160 receives correct answers such as joint points 251 which are small data corresponding to the captured image 250 which is the original data, and sends the received correct answers to the control unit 110.
  • the reception unit 160 may exist independently of the terminal 100, or may exist in the terminal 100 or the server 400.
  • the reception unit 160 may be configured by, for example, a computer terminal communicably connected to the terminal 100.
  • the conversion unit 111 relearns the trained model using the combination of the correct answer such as the captured image 250 which is the original data and the joint point 251 which is the small data corresponding to the captured image 250 as the teacher data.
  • the conversion unit 111 includes the joint points 251 and the like, which are small data obtained by converting the captured image 250 by the trained model before re-learning, and the joint points 251 and the like received by the reception unit 160.
  • the first threshold value can be changed by comparing with the correct answer and depending on the relationship between the comparison result and the certainty.
  • the first threshold can be lowered.
  • the fourth embodiment will be described.
  • the difference between this embodiment and the second embodiment is as follows.
  • the reception unit 160 receives the correct answer of the small data and further transmits the correct answer of the small data to the server 400 side, so that the certainty of the server 400 is determined based on the correct answer of the small data.
  • the analysis method (function g) used can be changed. According to the present invention, the accuracy of the analysis result can be further improved by adding the correct answer data to the server 400 side. Since the present embodiment is the same as the second embodiment in other respects, duplicate description will be omitted or simplified.
  • FIG. 15 is a functional block diagram showing the functions of the control unit 110, the reception unit 160, and the server 400, which are a part of the terminal 100.
  • the terminal 100 includes a conversion unit 111 and a transmission unit 112 as functions of the control unit 110.
  • the conversion unit 111 converts the captured image 250, which is the original data, into small data and certainty using the trained model of NN.
  • the reception unit 160 receives the correct answer such as the joint point 251 which is the small data corresponding to the captured image 250 which is the original data, and transmits the received correct answer to the server 400.
  • the reception unit 160 may exist independently of the terminal 100, or may exist in the terminal 100 or the server 400.
  • the reception unit 160 may be composed of, for example, a computer terminal communicably connected to the terminal 100 and the server 400. Further, the reception unit 160 may be a function of the control unit 410 of the server 400.
  • the reception unit 160 receives the correct answer such as the joint point 251 which is the small data corresponding to the captured image 250 which is the original data.
  • the server 400 receives the correct answer such as the joint point 251 from the reception unit 160, and can change the analysis method (function g) using the conviction in the server 400 based on the received correct answer.
  • the server 400 includes the joint points 251 and the like, which are small data obtained by converting the captured image 250 by the trained model of the conversion unit 111, and the correct answers of the joint points 251 and the like received by the reception unit 113.
  • the analysis method (function g) using the certainty can be changed according to the relationship between the comparison result and the certainty.
  • the analysis method (function g) can be changed according to the result.
  • the method itself may be changed, or the threshold value may be simply adjusted.
  • the server 400 may change the analysis method (function g) using the conviction in consideration of the analysis content (function f). For example, even if the certainty is relatively low, the difference between the small data such as the joint point 251 and the correct answer such as the joint point 251 received by the reception unit 160 is relatively small (for example, it is equal to or less than a predetermined threshold value). ), The second threshold value for selection of the joint point 251 and the like, which is small data by the function g, can be lowered.
  • a fifth embodiment will be described.
  • the difference between this embodiment and the second embodiment is as follows.
  • a trained model of NN prepared in advance on the server 400 using a combination of a plurality of small data and convictions and a correct answer of a predetermined analysis result as teacher data is used.
  • the analysis result is output based on a plurality of small data acquired from the conversion unit 111 and the certainty.
  • the present embodiment is different from the second embodiment in the teacher data for generating the learning model in the server 400, and in the present embodiment, the teacher data is used for generating the trained model in the server 400.
  • FIG. 16 is a functional block diagram showing the functions of the control unit 110, the reception unit 160, and the server 400, which are a part of the terminal 100.
  • the terminal 100 includes a conversion unit 111 and a transmission unit 112 as functions of the control unit 110.
  • the conversion unit 111 converts the captured image 250, which is the original data, into small data and certainty using the trained model of NN.
  • the reception unit 160 receives the correct answer of the analysis result by the server 400 corresponding to the captured image 250 which is the original data, and transmits it to the server 400.
  • the correct answer of the analysis result is, for example, the amount of activity of the subject 500.
  • the amount of activity, which is the correct answer of the analysis result can be measured by using an acceleration sensor or the like attached to the subject 500.
  • the reception unit 160 may exist independently of the terminal 100, or may exist in the terminal 100 or the server 400.
  • the reception unit 160 may be composed of, for example, a computer terminal communicably connected to the terminal 100 and the server 400. Further, the reception unit 160 may be a function of the control unit 410 of the server 400.
  • the server 400 is convinced that the plurality of captured images 250, which are the original data, are the joint points 251 and the like, which are the plurality of small data obtained by converting the plurality of captured images 250, which are the original data, by the conversion unit 111, respectively, using the trained model of the conversion unit 111.
  • a trained model of the server 400 is generated in which the combination of the correct answers of the analysis results corresponding to the plurality of captured images 250 is learned as the teacher data.
  • the generated trained model is used for analysis by the server 400.
  • This embodiment has the following effects.
  • small data with a certainty of more than the first threshold is transmitted to the server, and small data with a certainty of less than the first threshold is controlled not to be transmitted to the server.
  • the amount of communication and the amount of data stored in the server can be further reduced.
  • the combination of the small data and the conviction of the small data is transmitted to the server, and the server executes the analysis using the small data and the conviction of the small data.
  • the server executes the analysis using the small data and the conviction of the small data.
  • the server Furthermore, on the server, perform an analysis using a plurality of small data whose conviction is equal to or higher than the second threshold value. As a result, the reliability of the analysis result by the server can be further improved.
  • the small data is weighted using the conviction, and the analysis using the weighted multiple small data is executed. As a result, the reliability of the analysis result by the server can be further improved.
  • the trained model is used to perform analysis based on small data and the certainty of the small data. As a result, the reliability of the analysis result by the server can be further improved.
  • the trained model is used to convert the original data into small data and certainty, accept the correct answer of the small data corresponding to the original data, and correct the original data and the small data corresponding to the original data.
  • the trained model is retrained using the combination with and as teacher data. As a result, it is possible to appropriately reduce the amount of data communication and improve the reliability of the analysis result by the server.
  • the first threshold value is changed based on the small data, the degree of certainty, and the correct answer. As a result, it is possible to more appropriately reduce the amount of data communication and improve the reliability of the analysis result by the server.
  • the correct answer of the small data corresponding to the original data is accepted, and the analysis method using the conviction in the server is changed based on the small data and the conviction, and the above correct answer. As a result, it is possible to appropriately improve the reliability of the analysis result by the server.
  • the correct answer of the analysis result by the server corresponding to the original data is accepted, and the multiple small data and the certainty of each small data converted from the multiple original data by the trained model of the terminal, and the multiple elements.
  • a trained model of the server is generated by learning the combination of correct answers of the analysis results corresponding to each data as teacher data.
  • the configuration of the analysis system 10 described above is the main configuration described in explaining the features of the above-described embodiment, and is not limited to the above-mentioned configuration and may be variously modified within the scope of the claims. can. Moreover, it does not exclude the configuration provided in a general analysis system.
  • the function realized by the NN in the above-described embodiment may be realized by using a machine learning means other than the NN or an approximate function.
  • the means and methods for performing various processes in the analysis system 10 described above can be realized by either a dedicated hardware circuit or a programmed computer.
  • the program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versaille Disc) -ROM, or may be provided online via a network such as the Internet.
  • the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk.
  • the above program may be provided as a single application software, or may be incorporated into the software of a device such as a detection unit as a function.
  • 10 Analysis system 100 terminals, 110 Control unit, 111 converter, 112 transmitter, 120 storage, 130 Communication Department, 140 input section, 150 communication department, 160 reception department, 200 shooting equipment, 250 shot images, 251 Joint Point, 252 person area, 253 Peripheral object area, 300 communication networks, 400 servers, 500 Target people.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
PCT/JP2021/016969 2020-07-30 2021-04-28 解析システム、解析システムの制御プログラム、制御プログラム、および解析システムの制御方法 Ceased WO2022024487A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2021576462A JP7044215B1 (ja) 2020-07-30 2021-04-28 解析システム、解析システムの制御プログラム、制御プログラム、および解析システムの制御方法
US18/006,865 US12518534B2 (en) 2020-07-30 2021-04-28 Analysis system and control method for analysis system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020129137 2020-07-30
JP2020-129137 2020-07-30

Publications (1)

Publication Number Publication Date
WO2022024487A1 true WO2022024487A1 (ja) 2022-02-03

Family

ID=80038025

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/016969 Ceased WO2022024487A1 (ja) 2020-07-30 2021-04-28 解析システム、解析システムの制御プログラム、制御プログラム、および解析システムの制御方法

Country Status (3)

Country Link
US (1) US12518534B2 (https=)
JP (1) JP7044215B1 (https=)
WO (1) WO2022024487A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2023119906A1 (https=) * 2021-12-23 2023-06-29
WO2025224795A1 (ja) * 2024-04-22 2025-10-30 Ｎｔｔ株式会社 行動認識システムおよび方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011107765A (ja) * 2009-11-12 2011-06-02 Masafumi Hagiwara 不審行動検知方法および不審行動検知装置
JP2020027405A (ja) * 2018-08-10 2020-02-20 コニカミノルタ株式会社 物体検出プログラム、および物体検出装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6088381B2 (ja) 2013-08-02 2017-03-01 株式会社日立国際電気 物体検索システム
US10460600B2 (en) * 2016-01-11 2019-10-29 NetraDyne, Inc. Driver behavior monitoring
US11631241B2 (en) * 2020-04-08 2023-04-18 Micron Technology, Inc. Paired or grouped drones

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011107765A (ja) * 2009-11-12 2011-06-02 Masafumi Hagiwara 不審行動検知方法および不審行動検知装置
JP2020027405A (ja) * 2018-08-10 2020-02-20 コニカミノルタ株式会社 物体検出プログラム、および物体検出装置

Also Published As

Publication number Publication date
JPWO2022024487A1 (https=) 2022-02-03
US20230267741A1 (en) 2023-08-24
JP7044215B1 (ja) 2022-03-30
US12518534B2 (en) 2026-01-06

Similar Documents

Publication Publication Date Title
CN110291489B (zh) 计算上高效的人类标识智能助理计算机
US11058209B2 (en) Beauty counseling information providing device and beauty counseling information providing method
US20190216333A1 (en) Thermal face image use for health estimation
CN113743292B (zh) 基于视频序列的活动能力评估方法、装置、介质及设备
US20210304001A1 (en) Multi-head neural network model to simultaneously predict multiple physiological signals from facial RGB video
KR102274581B1 (ko) 개인화된 hrtf 생성 방법
US12267570B2 (en) Machine learning device, image processing device, machine learning method, and machine learning program
JP7044215B1 (ja) 解析システム、解析システムの制御プログラム、制御プログラム、および解析システムの制御方法
WO2022009652A1 (ja) データ収集システム、センサ装置、データ収集装置、及びデータ収集方法
US20240161254A1 (en) Information processing apparatus, information processing method, and program
JP2021033359A (ja) 感情推定装置、感情推定方法、プログラム、情報提示装置、情報提示方法及び感情推定システム
CN112084959B (zh) 一种人群图像处理方法及装置
CN105593903A (zh) 生物体确定装置及方法
Xiang et al. Remote safety monitoring for elderly persons based on omni-vision analysis
CN113658713A (zh) 传染趋势预测方法、装置、设备及存储介质
JPWO2019193899A1 (ja) 演算装置の学習方法、学習装置、学習プログラム、および学習済みモデル
US20250014184A1 (en) Computer program, information processing method, and information processing device
US12244924B2 (en) Information processing device and information processing method
JP7147942B1 (ja) データ収集装置、データ収集システム、データ収集方法
KR102868612B1 (ko) 디스플레이장치 및 그 제어방법
WO2021033453A1 (ja) 画像処理システム、画像処理プログラム、および画像処理方法
CN116868243B (zh) 实现用于监测人的ml模型的训练
JP7124746B2 (ja) 物体の部分位置推定プログラム、物体の部分位置推定用ニューラルネットワーク構造、物体の部分位置推定方法、および物体の部分位置推定装置
CN120126214A (zh) 一种久坐提醒方法、智能久坐提醒系统、设备及存储介质
WO2024247097A1 (ja) 学習装置、学習方法及び記憶媒体

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021576462

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21851087

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21851087

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18006865

Country of ref document: US