WO2022050093A1 - ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム - Google Patents

ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム Download PDF

Info

Publication number
WO2022050093A1
WO2022050093A1 PCT/JP2021/030654 JP2021030654W WO2022050093A1 WO 2022050093 A1 WO2022050093 A1 WO 2022050093A1 JP 2021030654 W JP2021030654 W JP 2021030654W WO 2022050093 A1 WO2022050093 A1 WO 2022050093A1
Authority
WO
WIPO (PCT)
Prior art keywords
pet
unit
estimation
information
situation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/030654
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
友香 中村
宏彰 大眉
靖 上坂
真史 佐藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2022546235A priority Critical patent/JP7479019B2/ja
Priority to US18/022,074 priority patent/US12446554B2/en
Priority to CN202180050178.5A priority patent/CN115885313B/zh
Publication of WO2022050093A1 publication Critical patent/WO2022050093A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
    • A01K29/00—Other apparatus for animal husbandry
    • A01K29/005—Monitoring or measuring activity
    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
    • A01K29/00—Other apparatus for animal husbandry
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • This disclosure generally relates to pet status estimation systems, pet cameras, servers, pet status estimation methods, and programs. More specifically, the present disclosure relates to a pet situation estimation system for a pet as a subject in image data, a pet camera provided with the pet situation estimation system, a server, a pet situation estimation method, and a program.
  • Patent Document 1 discloses a detection device that recognizes and detects an animal and a person from an image, respectively.
  • This detection device includes an animal detection unit that detects an animal from an image and a person detection unit that detects a person from the image. Further, the detection device further includes a detection result output unit that outputs information indicating that the target object has been detected as a detection result when an animal and a person are detected.
  • the animal detection unit scans the input image based on the feature amount data that reflects the characteristics of the animal stored in the animal feature amount storage unit.
  • the animal detection unit 21 can identify a region that matches the feature amount data of the animal or has a high degree of similarity, the animal detection unit 21 detects the object in the region as an animal.
  • a user wants to know specifically about the situation of a pet (animal) shown in the image data, or when the pet shown in the image data is in a specific situation. There is a request to receive a notification to that effect.
  • an object of the present invention is to provide a pet situation estimation system, a pet camera, a server, a pet situation estimation method, and a program that can easily grasp the pet situation.
  • the pet situation estimation system of one aspect of the present disclosure includes a region detection unit, an information generation unit, and an estimation unit.
  • the area detection unit detects a specific area indicating at least a part of the appearance of the pet as a subject in the image data.
  • the information generation unit generates pet information.
  • the pet information includes a trained model in which the pet's posture is learned in order to recognize an image of the pet's posture, and posture information regarding at least the posture of the pet based on the image data.
  • the estimation unit estimates the pet status regarding at least one of the emotions and behaviors of the pet shown in the specific area.
  • the pet camera of one aspect of the present disclosure includes the above-mentioned pet situation estimation system and an imaging unit for capturing the image data.
  • the server of one aspect of the present disclosure can communicate with the information generation unit and the pet camera provided with the estimation unit in the pet situation estimation system.
  • the server is provided with the area detection unit.
  • the server of one aspect of the present disclosure can communicate with the pet camera provided with the area detection unit in the pet situation estimation system.
  • the server is provided with the information generation unit and the estimation unit.
  • the pet situation estimation method of one aspect of the present disclosure includes a pet detection step, an information generation step, and an estimation step.
  • a specific region showing at least a part of the appearance of the pet as a subject is detected in the image data.
  • Pet information is generated in the information generation step.
  • the pet information includes a trained model in which the pet's posture is learned in order to recognize an image of the pet's posture, and posture information regarding at least the posture of the pet based on the image data.
  • the estimation step based on the pet information, the pet situation regarding at least one of the emotions and behaviors of the pet shown in the specific area is estimated.
  • the program of one aspect of the present disclosure is a program for causing one or more processors to execute the above-mentioned pet situation estimation method.
  • FIG. 1A is a schematic configuration diagram of a pet camera to which the pet situation estimation system according to the embodiment is applied.
  • FIG. 1B is a schematic configuration diagram of a presentation device that communicates with the pet camera of the above.
  • FIG. 2 is a conceptual diagram of the overall configuration of a pet management system including the pet situation estimation system of the above.
  • 3A to 3C are examples of image data to be estimated by the pet situation estimation system of the same.
  • 4A to 4C are other examples of image data to be estimated by the pet situation estimation system of the same.
  • 5A-5C are still another example of the image data to be estimated by the pet situation estimation system of the same.
  • FIG. 6 is still another example of the image data to be estimated by the pet situation estimation system of the above.
  • FIG. 7A and 7B are conceptual diagrams of the presentation device in which the estimation result by the pet situation estimation system of the above is presented on the screen.
  • FIG. 8 is a flowchart for explaining an operation example of the pet situation estimation system of the above.
  • FIG. 9 is a flowchart for explaining an operation example of the pet situation estimation system as described above.
  • FIG. 10 is a schematic configuration diagram of a pet camera to which a modified example of the pet situation estimation system described above is applied.
  • the pet situation estimation system 1 includes a region detection unit 32, an information generation unit 33, and an estimation unit 34.
  • the pet situation estimation system 1 mainly comprises a computer system having one or more processors and one or more memories.
  • all the components (area detection unit 32, information generation unit 33, estimation unit 34, etc.) of the pet situation estimation system 1 are collectively provided in one housing of the pet camera 100. It is explained as.
  • the components of the pet situation estimation system 1 in the present disclosure may be provided in a distributed manner.
  • at least a part of the components of the pet situation estimation system 1 is outside the pet camera 100 (for example, outside the server 7 or the like). It may be installed in the server).
  • the pet camera 100 may be provided with an information generation unit 33 and an estimation unit 34, and the server 7 capable of communicating with the pet camera 100 may be provided with an area detection unit 32.
  • the pet camera 100 may be provided with the area detection unit 32, and the server 7 capable of communicating with the pet camera 100 may be provided with the information generation unit 33 and the estimation unit 34.
  • the "server” referred to here may be composed of one external device (may be a device installed in the house of the user 300), or may be composed of a plurality of external devices.
  • the area detection unit 32 detects a specific area A1 indicating at least a part of the appearance of the pet 5 which is the subject H1 in the image data D1 (FIGS. 3A to 6).
  • the image data D1 is an image (data) imaged (generated) by the image pickup unit 2 (see FIG. 1A) of the pet camera 100.
  • the image data D1 may be a still image or an image of one frame (frame) in a moving image. Further, the image data D1 may be a processed part of the image captured by the image pickup unit 2.
  • the type of "pet” to be estimated by the pet situation estimation system 1 is a dog (animal).
  • the type of "pet” is not particularly limited, and may be a cat or another animal.
  • the specific area A1 is an area surrounded by a rectangular frame in the image data D1 as shown in FIGS. 3A to 6, and is represented by a “bounding box” surrounding the pet 5 of the subject H1.
  • the position of the pet 5 in the image data D1 is defined by, for example, the X-axis coordinates, the Y-axis coordinates, the width of the bounding box, the height of the bounding box, and the like in the upper left corner of the bounding box.
  • the specific region A1 is not limited to being shown by the bounding box, and may be shown by, for example, segmentation that distinguishes the subject H1 from the background on a pixel-by-pixel basis.
  • the "XY coordinates" for specifying the positions of the pet 5 and the specific object 6 other than the pet 5 in the image data D1 in the present disclosure are defined in pixel units as an example.
  • the information generation unit 33 generates pet information.
  • the pet information is at least the posture of the pet 5 based on the trained model (hereinafter, may be referred to as "first model M1") learned about the posture of the pet in order to recognize the image of the posture of the pet and the image data D1. Includes attitude information about.
  • the first model M1 is a model generated by machine learning and is stored in the model storage unit P1 (see FIG. 1A) of the pet camera 100.
  • the area detection unit 32 and the information generation unit 33 constitute a pet detection unit X1 (see FIG. 1A) that detects a dog (pet 5) from the image data D1.
  • a pet detection unit X1 see FIG. 1A
  • at least a part of the function of the information generation unit 33 may be provided outside the pet detection unit X1.
  • the estimation unit 34 estimates the pet status regarding at least one of the emotions and behaviors of the pet 5 shown in the specific area A1 based on the pet information. As an example in this embodiment, the estimation unit 34 estimates the pet situation based on the pet information and the condition information 9 (see FIG. 1A) related to at least one of the specific behaviors and emotions of the pet.
  • the condition information 9 is stored in the condition storage unit P2 (see FIG. 1A) of the pet camera 100.
  • the estimation unit 34 estimates the pet situation regarding at least one of the emotions and behaviors of the pet 5 based on the pet information, and as a result, it is possible to easily grasp the situation of the pet 5. ..
  • the pet situation estimation method includes a pet detection step, an information generation step, and an estimation step.
  • the image data D1 detects a specific region A1 indicating at least a part of the appearance of the pet 5 as the subject H1.
  • Pet information is generated in the information generation step.
  • the pet information includes at least the posture information of the pet 5 based on the trained model M1 learned about the posture of the pet in order to recognize the image of the posture of the pet and the image data D1.
  • the pet status regarding at least one of the emotions and behaviors of the pet 5 shown in the specific area A1 is estimated based on the pet information.
  • the pet situation regarding at least one of the emotions and behaviors of the pet 5 is estimated based on the pet information, and as a result, the situation of the pet 5 can be easily grasped. ..
  • the pet situation estimation method is used on a computer system (pet situation estimation system 1).
  • the pet situation estimation method can also be embodied in a program.
  • the program according to the present embodiment is a program for causing one or more processors to execute the pet situation estimation method according to the present embodiment.
  • pet management system 200 the system to which the pet situation estimation system 1 according to the present embodiment is applied (hereinafter referred to as “pet management system 200”) will be described in detail with reference to FIGS. 1A to 9.
  • the pet management system 200 includes one or a plurality of pet cameras 100, one or a plurality of presentation devices 4, and a server 7.
  • a single user 300 who receives a service for managing (watching over) the pet 5 by using the pet management system 200 will be described.
  • the user 300 is, for example, the owner of the pet 5, but is not particularly limited.
  • the user 300 installs one or more pet cameras 100 at a predetermined position in a facility (for example, a residence where the pet 5 lives together). When a plurality of pet cameras 100 are installed, the user 300 may install one in each room in the house.
  • the pet camera 100 is not limited to being installed indoors, and may be installed outdoors. In the following, for convenience of explanation, one pet camera 100 will be focused on.
  • the presentation device 4 is, for example, an information terminal owned by the user 300.
  • the information terminal is assumed to be a portable information terminal such as a smartphone or a tablet terminal.
  • the presentation device 4 may be a notebook personal computer or a stationary personal computer.
  • the presentation device 4 has a communication unit 41, a processing unit 42, and a display unit 43.
  • the communication unit 41 is a communication interface for enabling communication with each of the pet camera 100 (see FIG. 2) and the server 7 (see FIG. 2).
  • the communication unit 41 may be able to communicate with only one of the pet camera 100 and the server 7.
  • the processing unit 42 can be realized by a computer system including one or more processors (microprocessors) and one or more memories. That is, one or more processors execute one or more programs (applications) stored in one or more memories, thereby functioning as the processing unit 42.
  • the program is recorded in advance in the memory of the processing unit 42 here, it may be recorded and provided through a telecommunication line such as the Internet or on a non-temporary recording medium such as a memory card.
  • the user 300 installs dedicated application software (hereinafter referred to as "pet application”) for presenting a GUI (Graphical User Interface) related to the pet 5 to be watched over, and starts this pet application. It functions as a presentation device 4.
  • GUI Graphic User Interface
  • the display unit 43 constitutes a touch panel type liquid crystal display or an organic EL (Electro-Luminescence) display.
  • a screen for presenting information about the pet 5 is displayed (output) on the display unit 43.
  • the pet management system 200 has a plurality of residents (plurality of residents).
  • a plurality of presentation devices 4 carried by each user 300) will be provided. In the following, for convenience of explanation, one presentation device 4 (smartphone) carried by one user 300 (resident) will be described.
  • the pet camera 100 is, for example, a device having an imaging function for watching over pets.
  • the pet camera 100 includes an image pickup unit 2 (camera device) as shown in FIG. 1A.
  • the area where the pet 5 is in the residence may be outside the residence
  • is mainly the place where the pet can be active for example, the place where the food is placed
  • the pet camera 100 is installed so as to fit in the.
  • the user 300 can watch the situation of the pet 5 through the image captured by the image pickup unit 2, for example, even when he / she is out.
  • FIGS. 3A to 6 image data D1 showing pets 5 of a plurality of dog breeds is exemplified, but these drawings show the “type of posture” of the dog in order to explain the pet situation estimation system 1.
  • the dog breed is not particularly limited as it is merely an example.
  • the pet situation estimation system 1 is configured to recognize, for example, the "posture" of a dog in common to some extent regardless of the breed, but the posture may be recognized individually depending on the breed.
  • the pet camera 100 further includes a communication unit 11 in addition to the image pickup unit 2.
  • the communication unit 11 is a communication interface for enabling communication with each of the presentation device 4 (see FIG. 2) and the server 7 (see FIG. 2).
  • the communication unit 11 may have a function of performing short-range wireless communication with the presentation device 4, for example, in accordance with a standard of BLE (Bluetooth (registered trademark) Low Energy).
  • BLE Bluetooth (registered trademark) Low Energy
  • the communication unit 11 may exchange data by directly communicating with the presentation device 4 by short-range wireless communication.
  • the communication unit 11 is connected to the network NT1 (see FIG. 2) such as the Internet via a router or the like installed in the house.
  • the pet camera 100 can communicate with an external server 7 via the network NT1 to acquire information from the server 7 and output information to the server 7.
  • the presentation device 4 shown in FIG. 2 may be connected to the network NT1 via a mobile phone network (carrier network) or a public wireless LAN (Local Area Network) provided by a telecommunications carrier.
  • the mobile phone network includes, for example, a 3G (third generation) line, an LTE (Long Term Evolution) line, a 4G (fourth generation) line, a 5G (fifth generation) line, and the like.
  • the presenting device 4 can be connected to the network NT1 via the mobile phone network as long as it can be connected to the mobile phone network. For example, when the user 300 carrying the presentation device 4 is outside the house, the pet camera 100 and the server 7 can be communicated with each other by being connected to the network NT1 via a mobile phone network or the like.
  • the communication between the presentation device 4 and the pet camera 100 may be performed via the network NT1 and the server 7.
  • the pet situation estimation system 1 is provided in the pet camera 100 as shown in FIG. 1A.
  • the pet camera 100 further includes a processing unit 3, a model storage unit P1, and a condition storage unit P2, which constitute the pet status estimation system 1.
  • the details of the pet situation estimation system 1 will be described in the next column.
  • the server 7 is connected to the network NT1.
  • the server 7 can communicate with each of the pet camera 100 and the presenting device 4 via the network NT1.
  • the server 7 includes, for example, user information (name, user ID, telephone number, e-mail address, etc.), information on the pet camera 100 and the presentation device 4 owned by the user 300 (identification information, etc.), and the user 300.
  • user information name, user ID, telephone number, e-mail address, etc.
  • information on the pet camera 100 and the presentation device 4 owned by the user 300 identification information, etc.
  • the user 300 manage the information of your pet 5 (dog breed information, etc.).
  • the server 7 collects and stores various image data captured by the plurality of pet cameras 100 and processing results (particularly processing errors and the like).
  • the user 300 may download the pet application by accessing the server 7 through the presentation device 4.
  • the server 7 is assumed to be composed of one server device, but may be composed of a plurality of server devices, and such a server device may construct, for example, a cloud (cloud computing). May be good. Further, at least a part of the functions of the pet situation estimation system 1 may be provided in the server 7.
  • the pet camera 100 has a processing unit 3 and a model storage unit as the pet status estimation system 1 as described above, in addition to the imaging unit 2 and the communication unit 11. It includes P1 and a condition storage unit P2.
  • the pet situation estimation system 1 executes an "estimation process" for estimating the pet situation.
  • the model storage unit P1 is configured to be able to store data including a plurality of trained models.
  • the model storage unit P1 includes a rewritable non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory).
  • the condition storage unit P2 is configured to be able to store data including the condition information 9.
  • the condition storage unit P2 includes a rewritable non-volatile memory such as EEPROM.
  • the model storage unit P1 and the condition storage unit P2 may be composed of the same storage unit (memory). Further, the model storage unit P1 and the condition storage unit P2 may be the built-in memory of the processing unit 3.
  • the processing unit 3 can be realized by, for example, a computer system including one or more processors (microprocessors) and one or more memories. That is, by executing one or more programs (applications) stored in one or more memories by one or more processors, the processing unit 3 functions as each part to be described later.
  • the program is recorded in advance in the memory of the processing unit 3 here, it may be recorded and provided through a telecommunication line such as the Internet or on a non-temporary recording medium such as a memory card.
  • the processing unit 3 has a function as an overall control of the pet camera 100, that is, a control unit that controls the image pickup unit 2, the communication unit 11, the model storage unit P1, the condition storage unit P2, and the like.
  • the processing unit 3 includes an acquisition unit 31, a region detection unit 32, an information generation unit 33, an estimation unit 34, an output unit 35, and an object detection unit 36. ing.
  • the area detection unit 32 and the information generation unit 33 constitute a pet detection unit X1 that detects a dog (pet 5) from the image data D1.
  • the acquisition unit 31 is configured to acquire image data D1 (for example, a still image) from the image pickup unit 2.
  • the acquisition unit 31 may acquire an image of one frame (frame) in the moving image from the image pickup unit 2 as image data D1.
  • the processing unit 3 executes the estimation process.
  • the area detection unit 32 of the pet detection unit X1 is configured to detect a specific area A1 indicating at least a part of the appearance of the pet 5 which is the subject H1 in the image data D1.
  • the region detection unit 32 detects the specific region A1 based on the trained model (hereinafter, may be referred to as “second model M2”).
  • the second model M2 is (machine learning) learning the appearance factor (feature amount) of a predetermined kind of pet (here, "dog”) in order to recognize an image.
  • the second model M2 is stored in the model storage unit P1.
  • the second model M2 may include, for example, a model using a neural network or a model generated by deep learning using a multi-layer neural network.
  • the neural network (including a multi-layer neural network) may include, for example, a CNN (Convolutional Neural Network), a BNN (Bayesian Neural Network), or the like.
  • the second model M2 is realized by mounting a trained neural network on an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).
  • the second model M2 is not limited to the model generated by deep learning.
  • the second model M2 may be a model generated by a support vector machine, a decision tree, or the like.
  • the area detection unit 32 estimates whether or not the dog (pet 5) exists as the subject H1 in the acquired image data D1 by using the second model M2.
  • the region detection unit 32 detects the specific region A1 (see FIGS. 3A to 6) defined by the bounding box surrounding the pet 5.
  • the specific region A1 is not limited to being defined by the bounding box, and may be defined by, for example, segmentation.
  • the region detection unit 32 is a head region A2 (FIGS. 3A to 3) showing the head 50 (see FIG. 2) of the subject H1 based on the trained model (hereinafter, may be referred to as “third model M3”). 6) is detected.
  • the third model M3 learns the appearance factor (feature amount) in order to recognize an image of the head of a predetermined kind of pet (here, "dog"). That is, the region detection unit 32 further has a function as a head detection unit that detects the head region A2 including the face portion by using the third model M3. However, the area detection unit 32 and the head detection unit may be provided separately.
  • the third model M3 is stored in the model storage unit P1.
  • the third model M3 may include, for example, a model using a neural network or a model generated by deep learning using a multi-layer neural network.
  • the third model M3 is not limited to the model generated by deep learning. Further, the third model M3 may be composed of the same model as the second model M2.
  • the region detection unit 32 estimates whether or not the head 50 of the dog (pet 5) is present in the image data D1 by using the third model M3.
  • the region detection unit 32 estimates that the head 50 of the dog (pet 5) exists in the image data D1
  • the region detection unit 32 has a head region A2 defined by a bounding box surrounding the head 50 (see FIGS. 3A to 6). Is detected.
  • the head region A2 is not limited to being defined by the bounding box, and may be defined by, for example, segmentation.
  • the detection of either the specific region A1 or the head region A2 fails ( False detection) may occur.
  • the region detection unit 32 is "dog face (head 50)". Even if the head region A2 can be detected, it may be out of the annotation of "dog (whole appearance)". As a result, the detection of the specific region A1 as a "dog” may fail.
  • the region detection unit 32 detects at least one of the “dog” and the “dog face”, it is estimated that the dog (pet 5) is present in the acquired image data D1. If only the head region A2 is detected, the region detection unit 32 sets a region substantially equal to the head region A2 as the specific region A1. If the detection of the head region A2 fails even though the specific region A1 is detected, the processing unit 3 may end the estimation processing related to the image data D1.
  • the information generation unit 33 of the pet detection unit X1 detects a trained model (first model M1) that has learned about the pet's posture in order to recognize an image of the pet's (here, “dog”) posture, and a specific region A1.
  • Pet information is generated based on the image data D1.
  • the pet information includes posture information regarding at least the posture of the pet 5 shown in the specific area A1.
  • the information generation unit 33 has a posture determination unit 331, an orientation determination unit 332, and a distance determination unit 333.
  • the posture determination unit 331 is configured to determine (estimate) the posture of the dog (pet 5) using the first model M1 and the information regarding the specific area A1.
  • the first model M1 learns the appearance factor (feature amount) in order to recognize an image of the posture of a dog.
  • the first model M1 may include, for example, a model using a neural network or a model generated by deep learning using a multi-layer neural network, similarly to the second model M2 and the third model M3.
  • the first model M1 is not limited to the model generated by deep learning.
  • the first model M1 may be composed of the same model as the second model M2 and the third model M3.
  • 3A to 6A are all examples of image data D1 that can be the target of estimation processing by the pet situation estimation system 1.
  • FIG. 3A is an example of image data D1 showing a state in which the pet 5 stands up on four legs (first posture) and looks at the surroundings.
  • FIG. 3B is an example of image data D1 showing a state in which the pet 5 is lying down on the floor (second posture) and is looking at the surroundings while facing the front.
  • FIG. 3C is an example of image data D1 showing a state in which the pet 5 is in the second posture, facing slightly to the right and looking at the surrounding state, as in FIG. 3B.
  • FIG. 4A is an example of image data D1 showing a state in which the pet 5 is running in a posture in which the front legs are extended forward and the hind legs are extended backward (third posture). In FIG. 4A, the tail of pet 5 is facing up.
  • FIG. 4B is an example of image data D1 showing a state in which the pet 5 is walking with one forefoot and one hindfoot on the floor and the other foot bent away from the floor (fourth posture). be. In FIG. 4B, the tail of pet 5 hangs down.
  • FIG. 4C is an example of image data D1 showing a sleeping state with the eyes closed in a posture in which the pet 5 is lying down (fifth posture).
  • FIG. 5A is an example of image data D1 showing a state in which the pet 5 stands upright only on its hind legs (sixth posture) and is fond of a person (for example, a user 300).
  • FIG. 5B is an example of image data D1 showing a state in which the pet 5 is sitting (seventh posture), facing a person (for example, a user 300) and feeling nostalgic.
  • FIG. 5C is image data D1 showing a state in which the pet 5 is playing with the toy 63 (ball in the illustrated example) in a posture in which one forefoot is separated from the floor and stands up on the other foot (eighth posture). This is an example.
  • FIG. 6 is an example of image data D1 showing a state in which the pet 5 is eating food in the tableware 64 in a posture (9th posture) in which the pet 5 stands up on four legs while lowering the head 50.
  • the above 1st to 9th postures are merely examples of postures that a dog (pet 5) can take, and are not limited thereto.
  • the first model M1 is generated by machine learning about the posture of the dog, which has a high relationship with some behavior of the dog (particularly, a behavior related to some emotion).
  • machine learning is performed so that even finer states can be distinguished for specific postures that are desired to be estimated more accurately.
  • the "specific posture” here is a posture that is closely related to the behavior of the dog and is closely related to the emotion of the dog.
  • the dog's emotions that can be read from the dog's behavior are, for example, “joy”, “anger”, “sorrow”, “comfort”, “fear”, “relaxation”, etc., and the dog's specific posture. Some of the behaviors associated with may be associated with any of these emotions.
  • first posture Even if the pet 5 stands up on four legs (first posture), machine learning can be performed so that the posture can be estimated by distinguishing whether the pet 5 is showing teeth or tongue, and whether the ears are standing or hanging. Will be done.
  • the first posture showing teeth is related to the behavior of "intimidating”.
  • the first posture in which the ears are standing is related to the behavior of "being alert” looking at the surroundings.
  • the first posture in which the ears are drooping is related to the behavior of "being alert”.
  • the behavior of "intimidating” can be set as an behavior related to the dog's emotion of "anger”.
  • the behavior of "being alert” can be set as an behavior related to the dog's emotion of "fear”.
  • the behavior of "being alert” can be set as an behavior related to the dog's emotions such as “sorrow” and “cheap”. Also, even in the posture in which the pet 5 is sleeping (fifth posture), how it is sleeping, specifically, whether the back is curled or stretched, whether the eyes are closed, and whether the tongue is sticking out. Machine learning is performed so that the posture can be estimated by distinguishing the state such as whether or not.
  • a large number of image data are used in the annotation work (determination of teacher data) for tagging the image data (raw data).
  • the training dataset is selected from a large number of image data collected without restrictions on the breed, dog color, orientation of the dog, background in which the dog appears, and the like.
  • the data set for learning may include not only the image data of the actual dog but also the image data of the stuffed dog and the image data of the dog created by CG, and machine learning is performed by mixing these.
  • the posture information of the pet 5 determined by the posture determination unit 331 (including the determination result and the information regarding the specific area A1) is output to the distance determination unit 333.
  • the orientation determination unit 332 is configured to determine (estimate) the direction in which the pet 5 is facing in the image data D1 by using the image data D1 in which the specific region A1 is detected. That is, the pet information further includes the determination result of the orientation determination unit 332. Information regarding the detected specific region A1 and information regarding the head region A2 are input to the orientation determination unit 332 from the region detection unit 32. The orientation determination unit 332 may determine the orientation of the pet 5 which is the subject H1 only from the information regarding the specific region A1 detected by the region detection unit 32, but in the present embodiment, the information regarding the specific region A1 and the head may be determined. The orientation of the pet 5 is determined based on the information about the region A2.
  • the orientation determination unit 332 determines the direction in which the pet 5 is facing, at least from the relative positional relationship of the head region A2 with respect to the specific region A1. Specifically, the orientation determination unit 332 acquires information regarding the position and size of the pet 5 in the image data D1 through the specific area A1 detected by the area detection unit 32. Further, the orientation determination unit 332 acquires information regarding the position and size of the head 50 of the pet 5 in the image data D1 through the head region A2 detected by the region detection unit 32.
  • the orientation determination unit 332 determines that the pet 5 is generally facing to the right because the head region A2 is located in the upper right corner of the specific region A1. Further, in the example of FIG. 3B, in the orientation determination unit 332, since the head region A2 is located in the central portion in the left-right direction of the upper part in the specific region A1, the pet 5 faces the front substantially. judge.
  • the orientation determination unit 332 estimates a position (or a range thereof) of the pet 5's line of sight in the image data D1, and outputs the position information as a determination result.
  • the orientation determination unit 332 includes not only the relative positional relationship of the head region A2 with respect to the specific region A1, but also the ratio of the area occupied by the head region A2 to the specific region A1, the eyes and nose in the head region A2, and the like.
  • the orientation of the pet 5 may be determined in consideration of the position of the mouth and the like. In this case, the reliability of the determination is further improved.
  • the distance determination unit 333 is configured to determine (estimate) the relative distance of the pet 5 to the object region B1 (described later) (hereinafter, may be referred to as "distance between pet objects"). That is, the pet information further includes the determination result (information regarding the distance between pet objects) of the distance determination unit 333. In other words, an object (specific object 6) other than the dog (pet 5) may be reflected in the image data D1 as a part of the subject.
  • the human leg 61 is reflected as the specific object 6.
  • the entire appearance 62 of the person sitting on the floor is reflected as the specific object 6.
  • the dog toy 63 is reflected as the specific object 6.
  • the tableware 64 containing the food of the dog is reflected as the specific object 6.
  • the object detection unit 36 is configured to detect an object region B1 indicating a specific object 6 other than the pet 5 in the image data D1.
  • the object detection unit 36 may refer to a trained model (hereinafter, referred to as "fourth model M4") in which an appearance factor (feature amount) is learned in order to recognize an image of a specific object of a predetermined type. ), The object region B1 is detected.
  • the fourth model M4 may include, for example, a model using a neural network or a model generated by deep learning using a multi-layer neural network, similarly to the first model M1 to the third model M3.
  • the fourth model M4 is not limited to the model generated by deep learning. Further, the fourth model M4 may be composed of the same model as the first model M1, the second model M2, and the third model M3.
  • the fourth model M4 is generated by machine learning about an object that has a high relationship with some behavior of the dog (particularly, an behavior related to some emotion) for a specific object.
  • some behavior of the dog particularly, an behavior related to some emotion
  • the specific object 6 is a part (for example, a leg 61) or all (for example, an overall appearance 62) of a person, it is highly possible that the pet 5 is taking some emotion-related action.
  • the specific object 6 is a toy 63 or tableware 64, there is a high possibility that the action of "playing" or the action of "eating” is taken.
  • a learning data set for generating the fourth model M4 among a large number of image data showing an object other than a dog, an object that the dog is likely to be interested in is shown as a specific object.
  • Image data is selected.
  • the data set for learning may include not only the image data of the actual object but also the image data of the object created by CG, and machine learning is performed by mixing these.
  • the specific object is defined as an object other than a dog, other kinds of animals (cats and the like) may also be included as an object that the dog is likely to be interested in.
  • the object detection unit 36 estimates whether or not the specific object 6 exists in the image data D1 by using the fourth model M4. When it is estimated that the specific object 6 exists in the image data D1, the object detection unit 36 detects the object region B1 (see FIGS. 5A to 6) defined by the bounding box surrounding the specific object 6.
  • the object area B1 is not limited to being defined by the bounding box, and may be defined by, for example, segmentation.
  • the object detection unit 36 considers an object that does not correspond to the specific object 6 as a “background”.
  • the object detection unit 36 outputs information regarding the detected object region B1 (including information regarding the type of the specific object 6) to the distance determination unit 333.
  • the object detection unit 36 outputs to that effect to the distance determination unit 333.
  • the distance determination unit 333 includes information on the head region A2 detected by the region detection unit 32, information on the object region B1 detected by the object detection unit 36, and attitude information of the pet 5 determined by the attitude determination unit 331. Based on, the distance between pet objects is determined.
  • the distance determination unit 333 is based on the distance from the position of the object region B1 (the position of the upper left corner, the position of the center of gravity, etc.) to the position of the pet 5, for example, a three-step distance relationship. It is determined which of the above is applicable.
  • the three-stage distance relationship is a first distance state (a very close distance), a second distance state (a relatively short distance), and a third distance state (a relatively long distance).
  • the first distance state to the third distance state are classified based on, for example, the number of pixels.
  • the fact that the distance relationship has three stages is just an example, and it may be two stages, four stages or more, or stepless (pixel unit).
  • the position of the head region A2 (the position of the upper left corner or the position of the center of gravity, etc.) is defined as the “position of the pet 5", but the position of the specific area A1 (the position of the upper left corner or the position of the center of gravity, etc.) is defined as the "pet”. It may be defined as "position of 5".
  • the distance determination unit 333 further determines the distance between pet objects in consideration of the degree (area) in which the object region B1 and the head region A2 (or the specific region A1) overlap each other.
  • the pet 5 and the specific object 6 may be arranged in the depth direction and may be overlapped in the image data D1. be. If the distance determination unit 333 determines the distance between pet objects based only on the distance from the position of the specific object 6 to the position of the pet 5 in the image data D1, the pet 5 does not take any action related to the specific object 6. However, there is a possibility that it is determined to be the first distance state. Therefore, the distance determination unit 333 further considers the posture information of the pet 5 determined by the attitude determination unit 331, and determines which of the first to third distance states is applicable.
  • the distance determination unit 333 does not have the posture in which the head 50 is lowered.
  • This image data D1 may be regarded as a third distance state.
  • the distance determination unit 333 may consider the image data D1 as an outlier and end the estimation process.
  • the distance determination unit 333 outputs the determination result regarding the distance between pet objects, the information regarding the head region A2, and the posture information to the estimation unit 34.
  • the distance determination unit 333 skips the determination regarding the distance between pet objects and outputs the information regarding the head region A2 and the posture information to the estimation unit 34. do.
  • the pet detection unit X1 executes the above-mentioned detection process of the specific area A1 in the area detection unit 32 and the generation process of generating pet information in the information generation unit 33 in this order.
  • the pet detection unit X1 may execute the detection process and the generation process substantially simultaneously in parallel.
  • the estimation unit 34 is configured to estimate the pet status regarding at least one of the emotions and behaviors of the pet 5 shown in the specific area A1 based on the pet information.
  • the estimation unit 34 estimates the pet situation based on the pet information and the condition information 9.
  • the pet information includes the posture information regarding the posture of the pet 5 determined by the posture determination unit 331, the information regarding the orientation of the pet 5 determined by the orientation determination unit 332, and the pet determined by the distance determination unit 333. Includes information about the distance between objects.
  • Condition information 9 is information related to at least one of a specific behavior and emotion of a pet preset as an extraction target.
  • the correspondence information hereinafter, may be referred to as “pattern” as shown in Tables 1 to 4 below is an example that can be included as the condition information 9, and many such patterns are prepared. It is stored in the condition storage unit P2 as a database.
  • the estimation unit 34 searches the condition information 9 for a pattern of conditions that matches the obtained pet information. At that time, the estimation unit 34 determines whether or not the pet 5 is facing the specific object 6 from the information regarding the orientation of the pet 5 and the information regarding the object region B1, for example, the object region on the line of sight of the pet 5. Whether or not B1 exists is estimated, and the estimation result is taken into consideration to search the condition information 9.
  • the pet information obtained is "first distance state", "standing on four legs and lowering the head", and "facing the tableware". It is assumed that the three results are included.
  • the estimation unit 34 searches the condition information 9 for a pattern of conditions that matches these results.
  • Table 1 there is a pattern of matching conditions, and the "behavior / emotion" of "during meal / delicious” is associated with it. Therefore, the estimation unit 34 estimates that the pet status of the pet 5 in the image data D1 is "during meal / delicious".
  • the estimation unit 34 searches the condition information 9 for a pattern of conditions that matches these results.
  • Table 2 there is a pattern of matching conditions, and the "behavior / emotion" of "nostalgia / joy" is associated with it. Therefore, the estimation unit 34 estimates that the pet status of the pet 5 in the image data D1 is "nostalgia / joy".
  • the obtained pet information includes three results: "third distance state", "standing on four legs and showing teeth", and "turning toward a person”. And.
  • the estimation unit 34 searches the condition information 9 for a pattern of conditions that matches these results.
  • Table 3 there is a pattern of matching conditions, and the "behavior / emotion" of "intimidating / angry” is associated with it. Therefore, the estimation unit 34 estimates that the pet status of the pet 5 in the image data D1 is "intimidating / angry".
  • the estimation unit 34 searches the condition information 9 for a pattern of conditions that matches these results.
  • Table 4 there is a pattern of matching conditions, and "behavior / emotion" of "playing / fun” is associated with it. Therefore, the estimation unit 34 estimates that the pet status of the pet 5 in the image data D1 is "playing / fun”.
  • each pattern both behavior and emotion are associated with the condition, but there are cases where only one of them is associated.
  • the types of conditions for each pattern are not limited to the above three types (distance to an object, pet posture, and pet orientation), at least including the conditions related to "pet posture". For example, a condition regarding an "area" in which the head region A2 and the object region B1 overlap may be included.
  • the condition information 9 of the present embodiment includes a plurality of directions in which the pet 5 is facing (directions of the tableware 64, the person, and the toy 63) and a plurality of pet situations (meal / delicious, nostalgic / joy). , And playing / fun) include orientation information associated with each other.
  • the estimation unit 34 estimates the pet situation based on the determination result of the orientation determination unit 332 and the orientation information. Therefore, the reliability of the estimation regarding the situation of the pet 5 can be improved.
  • condition information 9 in the present embodiment includes a plurality of threshold values (first distance state to third distance) relating to the distance between the plurality of types of the specific object 6 (tableware 64, the person, and the toy 63) and the pet 5 and the specific object 6.
  • the state) and the associated information are included.
  • the estimation unit 34 estimates the pet situation by comparing the determination result of the distance determination unit 333 with the plurality of threshold values. Therefore, the reliability of the estimation regarding the pet situation can be further improved.
  • the distance between the pet objects determined by the distance determination unit 333 is not the first distance state to the third distance state, but the distance between the pet 5 and the specific object 6 is quantified information (for example, the number of pixels). As long as it is a numerical value corresponding to), a plurality of threshold values may be digitized information.
  • the estimation unit 34 when the specific object 6 shown in the object area B1 detected by the object detection unit 36 is the tableware 64 and the distance determined by the distance determination unit 333 is equal to or less than a predetermined threshold value. As for the pet situation, it is estimated that the pet 5 is eating. This estimation is based on the fact that when the specific object 6 is the tableware 64, the pet 5 may be in close contact with the specific object 6 as the tip of the nose is thrust into the tableware 64. Therefore, when the pet 5 of the image data D1 is actually eating, there is a high possibility that the pet situation is also estimated to be "during eating".
  • the estimation unit 34 can estimate the pet situation even when the specific object 6 does not exist in the image data D1 and the object region B1 is not detected.
  • the condition information 9 includes a pattern in which only the posture of the pet and at least one of the specific behaviors and emotions of the pet are associated with each other. Specifically, the pet's posture of "lying with his eyes closed” is associated with "behavior / emotion" of "sleeping / peaceful". As a result, the estimation unit 34 estimates that the pet status of the pet 5 in the image data D1 is "sleeping / peaceful" only by the posture of the pet.
  • the output unit 35 is configured to output the estimation result (estimated pet status) of the estimation unit 34.
  • the output unit 35 outputs the estimation result of the estimation unit 34 in a mode in which the estimation result is associated with the image data D1 in which the specific region A1 which is the basis of the estimation result is detected.
  • the output unit 35 transmits information (hereinafter referred to as “output information”) in which the estimation result (for example, “sleeping / peaceful”) and the image data D1 are associated with the presenting device 4 through the communication unit 11. If the user 300 carrying the presentation device 4 is out of the office, the output information may be transmitted to the presentation device 4 via the server 7. It is preferable that the output information further includes information on the time when the image data D1 which is the basis of the estimation result is imaged by the image pickup unit 2.
  • the output information is stored in a memory or the like built in the pet camera 100.
  • the output information may be transmitted to and stored in the server 7 or other peripheral devices.
  • the presentation device 4 When the presentation device 4 receives the output information from the pet camera 100, the presentation device 4 replaces the pet status included in the output information with, for example, a simple expression (message), and notifies the screen by a push notification or the like with the message. You may (present). When the user 300 opens the push notification, the presentation device 4 may activate the pet application and present a detailed pet status including the image data D1 on the screen (see FIGS. 7A and 7B). Alternatively, the output information may be notified by e-mail via a mail server.
  • a simple expression messages
  • You may (present).
  • the presentation device 4 may activate the pet application and present a detailed pet status including the image data D1 on the screen (see FIGS. 7A and 7B).
  • the output information may be notified by e-mail via a mail server.
  • the presentation device 4 displays the image data D1 (see FIG. 3C: the posture of lying down on the floor) which is the basis for estimating the pet situation on the screen 430 of the display unit 43.
  • the condition information 9 includes a pattern in which the "emotion" of "loneliness” is associated with the two conditions of "there is no specific object” and "the posture of lying on the floor”.
  • “lonely” is estimated as a pet situation.
  • the presenting device 4 displays the character string data obtained by converting the emotion of the pet 5 "lonely” into a familiar expression "sadness” with a balloon superimposed on the image data D1.
  • the presentation device 4 is displayed on the screen 430 of the display unit 43 with the image data D1 (see FIG. 6: standing on four legs and lowering the head) which is the basis for estimating the pet situation. Attitude) is displayed.
  • the condition information 9 is "during meal / delicious” under the three conditions of "first distance state", "standing on four legs and lowering the head", and "facing the tableware". It includes a pattern in which "behavior / emotion" is associated. As a result, it is an example that the pet situation is estimated to be “meal / delicious”.
  • the presentation device 4 superimposes the character string data "during meal” and the character string data obtained by converting the emotion of the pet 5 "delicious” into a familiar expression "delicious” on the image data D1 with a balloon. Let me display it.
  • the presentation device 4 further displays the time (date and time) when the image data D1 is captured on the screen 430 of the display unit 43.
  • the output unit 35 is not limited to transmitting the output information including the image data D1 (raw data) that is the basis of the estimation result, and may be transmitted after processing the image data. Further, the output unit 35 may replace the image data D1 with an icon image corresponding to the estimated pet situation (for example, an icon image of a dog that looks lonely with tears) and then transmits the image data D1. The processing of the data and the replacement with the icon image may be executed on the side of the presentation device 4 or the server 7.
  • the estimation result of the estimation unit 34 is not limited to the screen output, and may be performed by voice output instead of screen output or in addition to screen output.
  • the processing unit 3 executes the estimation process every time the acquisition unit 31 acquires the image data D1. For example, when the image pickup unit 2 captures a still image at a predetermined interval (for example, an interval of several minutes or several tens of minutes), the processing unit 3 can execute the estimation process at the predetermined interval. Alternatively, when the imaging unit 2 captures a moving image at a predetermined frame rate, frames are spaced at regular intervals (for example, intervals of several minutes or tens of minutes) for a plurality of consecutive frame images in the moving image. An image can be acquired as image data D1 and an estimation process can be executed.
  • a predetermined interval for example, an interval of several minutes or several tens of minutes
  • the output unit 35 may transmit the output information to the presenting device 4 each time the estimation unit 34 estimates the pet status for one image data D1, or the output information may be transmitted to some extent to a memory or the like built in the pet camera 100. You may pool them and send them all at once.
  • the output unit 35 of the estimation unit 34 when the estimation result of the estimation unit 34 regarding the plurality of image data D1 indicates that the pet 5 is in a posture facing the same direction consecutively a predetermined number of times (for example, twice), the output unit 35 of the estimation unit 34.
  • the output of the estimation result may be limited. Specifically, the posture and orientation of the pet 5 with respect to a certain image data D1 is "standing on four legs and lowering the head" and “facing the tableware" (that is, "during meal”). It is assumed that the output information is output to the presentation device 4 by presuming that (action).
  • the output unit 35 does not have to output the estimation result. ..
  • the output unit 35 may collectively transmit the output information having the same estimation result a predetermined number of times as one estimation result. The setting regarding the "predetermined number of times" may be appropriately changed by the operation input of the user 300 to the pet camera 100 or the presentation device 4.
  • the pet camera 100 installed in the house of the user 300 captures and monitors a predetermined management area in which the pet 5 can mainly act by the image pickup unit 2.
  • the pet camera 100 may capture the management area as a still image at a predetermined cycle, or may continue to capture the control area as a moving image for a predetermined period.
  • the pet situation estimation system 1 of the pet camera 100 acquires the image data D1 (still image or one frame in the moving image) captured by the image pickup unit 2 (S1), the estimation process is performed. Is executed (S2).
  • the pet situation estimation system 1 determines whether or not the dog (pet 5) is captured as the subject H1 in the image data D1 by using the second model M2 in the area detection unit 32 (S3). If the dog (pet 5) is shown (S3: Yes), the pet situation estimation system 1 detects the specific region A1 indicating the pet 5 (S4: pet detection step), and determines the head 50 (S5). move on.
  • the pet situation estimation system 1 determines whether or not the head 50 of the dog (pet 5) is shown in the image data D1 by using the third model M3 in the area detection unit 32 (S5). If the head 50 is shown (S5: Yes), the pet situation estimation system 1 detects the head region A2 indicating the head 50 (S6). In the present embodiment, if the head 50 is not shown (S5: No), the pet situation estimation system 1 finishes the estimation process for the image data D1 and waits for the acquisition of the next image data D1 (returns to S1). ). However, as long as the specific region A1 is detected, the estimation process may be continued even if the head region A2 is not detected.
  • the process proceeds to the determination of the specific object 6 (S9: see FIG. 9).
  • the pet situation estimation system 1 sets a region substantially equivalent to the head region A2 in the specific region A1 (S7: No). S8), the process proceeds to the determination (S9) of the specific object 6.
  • the pet situation estimation system 1 determines whether or not the specific object 6 is captured in the image data D1 by using the fourth model M4 in the object detection unit 36 (S9). If the specific object 6 is captured (S9: Yes), the pet situation estimation system 1 detects the object region B1 indicating the specific object 6 (S10), and proceeds to the posture determination (S12). On the other hand, if the specific object 6 is not shown (S9: No), the result that the object area B1 is not detected is obtained (S11), and the posture determination (S12) proceeds.
  • the pet situation estimation system 1 determines the posture of the dog (pet 5) by the posture determination unit 331 using the information regarding the first model M1 and the specific area A1 (S12).
  • the pet situation estimation system 1 determines the orientation of the pet 5 by the orientation determination unit 332 based on the information regarding the specific area A1 and the information regarding the head region A2 (S13).
  • the pet situation estimation system 1 determines the distance between pet objects based on the information regarding the head region A2, the information regarding the object region B1, and the posture information by the distance determination unit 333 (S14). If the object area B1 is not detected, the process S14 is skipped.
  • the pet situation estimation system 1 generates pet information from the determination results obtained in the processes S12 to S14 (S15: information generation step).
  • the pet situation estimation system 1 estimates the pet situation based on the pet information and the condition information 9 (S16: estimation step).
  • the pet situation estimation system 1 transmits output information associated with the estimated pet situation and the image data D1 to the presenting device 4, and causes the presenting device 4 to present the output information (S17).
  • the estimation unit 34 estimates the pet situation regarding at least one of the emotions and behaviors of the pet 5 based on the pet information 8, and as a result, it is easy to grasp the pet situation. can do.
  • the orientation determination unit 332 for determining the direction in which the pet 5 is facing in the image data D1 since the orientation determination unit 332 for determining the direction in which the pet 5 is facing in the image data D1 is provided, the reliability of estimation regarding the pet situation can be improved by considering the orientation of the pet 5. Can be improved. Further, since the direction in which the pet 5 is facing is determined from the relative positional relationship of the head region A2 with respect to the specific region A1, the reliability of the determination regarding the direction in which the pet 5 is facing can be further improved.
  • the output unit 35 outputs the estimation result of the estimation unit 34 in the form of associating the estimation result of the estimation unit 34 with the image data D1 in which the specific region A1 which is the basis of the estimation result is detected, the situation of the pet can be better grasped. Can be made easier.
  • the user 300 can easily grasp the behavior / emotion of the pet 5 through the pet situation estimated by the pet situation estimation system 1, and as a result, it becomes easy to communicate with the pet 5. Further, the user 300 can easily grasp the behavior / emotion of the pet 5 in the house by the notification from the presentation device 4 even while going out, and as a result, it is easy to manage (watch over) the pet 5. In particular, for example, when the pet situation is presumed to be an urgent behavior (such as being ill or tired), it is possible to quickly know that fact.
  • the above embodiment is only one of the various embodiments of the present disclosure.
  • the above embodiment can be variously modified according to the design and the like as long as the object of the present disclosure can be achieved.
  • the same function as the pet situation estimation system 1 according to the above embodiment may be embodied by a pet situation estimation method, a computer program, a non-temporary recording medium on which a computer program is recorded, or the like.
  • the pet situation estimation system 1 in the present disclosure includes a computer system.
  • the computer system mainly consists of a processor and a memory as hardware.
  • the processor executes the program recorded in the memory of the computer system, the function as the pet situation estimation system 1 in the present disclosure is realized.
  • the program may be pre-recorded in the memory of the computer system, may be provided through a telecommunications line, and may be recorded on a non-temporary recording medium such as a memory card, optical disk, hard disk drive, etc. that can be read by the computer system. May be provided.
  • the processor of a computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large scale integrated circuit (LSI).
  • IC semiconductor integrated circuit
  • LSI large scale integrated circuit
  • the integrated circuit such as IC or LSI referred to here has a different name depending on the degree of integration, and includes an integrated circuit called a system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Further, an FPGA (Field-Programmable Gate Array) programmed after the LSI is manufactured, or a logical device capable of reconfiguring the junction relationship inside the LSI or reconfiguring the circuit partition inside the LSI should also be adopted as a processor. Can be done.
  • a plurality of electronic circuits may be integrated on one chip, or may be distributed on a plurality of chips. A plurality of chips may be integrated in one device, or may be distributed in a plurality of devices.
  • the computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or a plurality of electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.
  • a plurality of functions in the pet situation estimation system 1 are integrated in one housing.
  • the components of the pet situation estimation system 1 may be distributed in a plurality of housings.
  • at least a part of the trained models of the first model M1 to the fourth model M4 of the pet situation estimation system 1 may be provided outside the pet camera 100 (for example, an external server such as a server 7). ..
  • a plurality of functions in the pet situation estimation system 1 may be integrated in one housing (housing of the pet camera 100) as in the basic example. Further, at least a part of the functions of the pet situation estimation system 1, for example, a part of the functions of the pet situation estimation system 1 may be realized by a cloud (cloud computing) or the like.
  • FIG. 10 shows the pet situation estimation system 1A of this modified example.
  • the same reference numerals may be given and the description thereof may be omitted as appropriate.
  • the pet detection unit X1 has an area detection unit 32 and an information generation unit 33, and after the area detection unit 32 detects the pet 5, the posture determination unit of the information generation unit 33
  • the posture of the pet 5 is determined by 331 and the posture information is generated. That is, first, the presence or absence of the pet 5 in the acquired image data D1 is detected, and then the posture is determined.
  • the pet situation estimation system 1A of this modification is different from the pet situation estimation system 1 of the basic example in that the area detection unit 32 has the function of the posture determination unit 331.
  • the area detection unit 32 identifies the pet 5 in a specific posture based on the first model M1 that has learned about the posture of the pet in order to recognize the image of the pet's posture in the image data D1. Region A1 is detected.
  • the area detection unit 32 uses, for example, the first model M1 to the fourth model M4 to determine whether or not the pet 5 in a specific posture is captured in the image data D1 as the subject H1 and specifies it.
  • the specific area A1 indicating the pet 5 in the posture of is detected.
  • the specific posture is a posture that is closely related to the behavior of the dog and is closely related to the emotion of the dog. Specific postures include, for example, sitting, lying down, sleeping, or standing on four legs.
  • Information about the specific area A1 indicating the pet 5 taking a specific posture is input to the information generation unit 33 and used for the orientation determination in the orientation determination unit 332 and the distance determination in the distance determination unit 333.
  • the pet detection unit X1 of this modification detects the pet 5 in a specific posture rather than determining the posture after detecting the presence of the pet 5.
  • the estimation unit 34 estimates the pet situation based on the pet information and the condition information 9.
  • the estimation unit 34 estimates the pet situation by using the pet information and the trained model (classifier) machine-learned for at least one of the specific behaviors and emotions of the pet instead of the condition information 9. May be good.
  • the classifier classifies the pet information into at least one of the pet's specific behaviors and emotions by inputting the pet information.
  • the number of dogs (pets 5) as the subject H1 in one image data D1 was one.
  • the number of dogs (pets 5) as the subject H1 in one image data D1 is two or more (for example, two dogs, a parent dog and a puppy).
  • the pet situation estimation system 1 When a plurality of specific areas A1 are detected in one image data D1, the pet situation estimation system 1 generates pet information for each specific area A1 and estimates the pet situation.
  • the number of specific objects 6 other than the pet 5 in one image data D1 was zero or one.
  • the number of the specific objects 6 in one image data D1 may be two or more.
  • the pet situation estimation system 1 determines the distance between pet objects with respect to each object area B1. In this case, the pet situation estimation system 1 may estimate the pet situation by selecting the object region B1 having the shortest distance from the pet 5 among the plurality of pet object distances.
  • the pet situation estimation system 1 has a function of determining the orientation of the pet 5 (direction determination unit 332) and a function of determining the distance between pet objects (distance determination unit 333). Is not a required feature and may be omitted.
  • At least a part of the first model M1 to the fourth model M4 in the basic example may be machine-learned by reinforcement learning.
  • the pet situation estimation system (1,1A) includes a region detection unit (32), an information generation unit (33), an estimation unit (34), and the pet status estimation system (1,1A).
  • the area detection unit (32) detects a specific area (A1) indicating at least a part of the appearance of the pet (5), which is the subject (H1), in the image data (D1).
  • the information generation unit (33) generates pet information.
  • the pet information includes at least the posture information of the pet (5) based on the trained model (first model M1) learned about the pet posture in order to recognize the image of the pet posture and the image data (D1). ..
  • the estimation unit (34) estimates the pet situation regarding at least one of the emotions and behaviors of the pet (5) shown in the specific area (A1) based on the pet information. According to the first aspect, the estimation unit (34) estimates the pet situation regarding at least one of the emotions and behaviors of the pet (5) based on the pet information, and as a result, the situation of the pet (5). Can be made easier to understand.
  • the estimation unit (34) has the pet information and the conditional information related to at least one of the specific behavior and emotion of the pet (the pet's specific behavior and emotion).
  • the pet situation is estimated based on 9).
  • the estimation unit (34) simplifies the pet situation estimation system (1,1A) as compared with the case where the pet situation is estimated using, for example, a machine-learned trained model. It is feasible in the configuration.
  • the region detection unit (32) is based on the trained model (second model M2), and the specific region (2nd model M2) is used. A1) is detected.
  • the trained model (second model M2) learns the appearance factors of a predetermined type of pet in order to recognize an image. According to the third aspect, the reliability regarding the detection of the specific region (A1) can be improved, and as a result, the reliability of the estimation regarding the situation of the pet (5) can be improved.
  • the region detection unit (32) is based on the trained model (third model M3).
  • the head region (A2) indicating the head (50) of the subject (H1) is detected.
  • the trained model (third model M3) is a trained model in which the appearance factors of a predetermined type of pet's head are trained in order to recognize an image. According to the fourth aspect, the reliability regarding the detection of the head region (A2) can be improved, and as a result, the reliability of the estimation regarding the situation of the pet (5) can be improved.
  • the information generation unit (33) uses the image data (D1) in which the specific region (A1) is detected to obtain an image. It has an orientation determination unit (332) for determining the direction in which the pet (5) is facing in the data (D1).
  • the pet information further includes the determination result of the orientation determination unit (332). According to the fifth aspect, the reliability of the estimation regarding the situation of the pet (5) can be improved by considering the direction in which the pet (5) is facing.
  • the orientation determination unit (332) has at least a positional relationship of the head region (A2) with respect to the specific region (A1). Therefore, the direction in which the pet (5) is facing is determined. According to the sixth aspect, the reliability of the determination regarding the direction in which the pet (5) is facing can be improved.
  • the estimation unit (34) relates to pet information and at least one of the pet's specific behaviors and emotions.
  • the pet situation is estimated based on the condition information (9).
  • the condition information (9) includes direction information in which a plurality of directions in which the pet (5) is facing and a plurality of pet situations are associated with each other.
  • the estimation unit (34) estimates the pet situation based on the determination result of the orientation determination unit (332) and the orientation information. According to the seventh aspect, the reliability of the estimation regarding the situation of the pet (5) can be improved.
  • the pet situation estimation system (1,1A) further includes an output unit (35) that outputs an estimation result of the estimation unit (34) in any one of the fifth to seventh aspects. ..
  • the output unit (35) is the estimation unit.
  • the output of the estimation result of (34) is limited. According to the eighth aspect, it is possible to suppress continuous output of similar estimation results, and for example, it is possible to reduce the processing load.
  • the pet situation estimation system (1,1A) shows a specific object (6) other than the pet (5) in the image data (D1) in any one of the first to eighth aspects.
  • An object detection unit (36) for detecting an object region (B1) is further provided.
  • the information generation unit (33) has a distance determination unit (333) for determining the relative distance of the pet (5) to the object region (B1).
  • the pet information further includes the determination result of the distance determination unit (333).
  • the estimation unit (34) estimates the pet situation based on the pet information and the condition information (9) related to at least one of the pet's specific behaviors and emotions.
  • the condition information (9) includes information in which a plurality of types of specific objects and a plurality of threshold values relating to the distance between the pet and the specific objects are associated with each other.
  • the estimation unit (34) estimates the pet situation by comparing the determination result of the distance determination unit (333) with a plurality of threshold values. According to the ninth aspect, the reliability of the estimation regarding the situation of the pet (5) can be improved by considering the relative distance of the specific region (A1) to the object region (B1).
  • the object detection unit (36) learns the appearance factor for recognizing an image of a specific object of a predetermined type.
  • the object region (B1) is detected based on the completed model (fourth model M4). According to the tenth aspect, the reliability regarding the detection of the object region (B1) is improved.
  • the estimation unit (34) is in the object region (B1) detected by the object detection unit (36).
  • the specific object (6) shown is tableware (64) and the distance determined by the distance determination unit (333) is equal to or less than a predetermined threshold value, the pet (5) eats as a pet situation. Presumed to be.
  • the eleventh aspect when the pet (5) in the image data (D1) is actually eating, it is highly likely that the pet situation is also presumed to be "eating".
  • the pet situation estimation system (1,1A) further includes an output unit (35) in any one of the first to eleventh aspects.
  • the output unit (35) outputs the estimation result of the estimation unit (34) in a manner associated with the image data (D1) in which the specific region (A1) on which the estimation result is based is detected. According to the twelfth aspect, it is possible to make it easier to grasp the situation of the pet (5).
  • the area detection unit (32) describes the posture of the pet in the image data (D1). Based on the trained model (first model M1) learned about the posture of the pet for image recognition, the specific region (A1) of the pet (5) taking a specific posture is detected. According to the thirteenth aspect, it is possible to make it easier to grasp the situation of the pet (5).
  • the pet camera (100) according to the fourteenth aspect is the pet situation estimation system (1,1A) in any one of the first to thirteenth aspects, and an image pickup unit (2) that captures image data (D1). And. According to the fourteenth aspect, it is possible to provide a pet camera (100) capable of easily grasping the situation of the pet (5).
  • the server (7) according to the fifteenth aspect is provided with an information generation unit (33) and an estimation unit (34) in the pet situation estimation system (1,1A) in any one of the first to thirteenth aspects. It can communicate with the pet camera (100).
  • the server (7) is provided with an area detection unit (32). According to the fifteenth aspect, it is possible to provide a server (7) capable of easily grasping the situation of the pet (5).
  • the server (7) according to the sixteenth aspect is a pet camera (100) provided with an area detection unit (32) in the pet situation estimation system (1,1A) in any one of the first to thirteenth aspects. Can communicate with.
  • the server (7) is provided with an information generation unit (33) and an estimation unit (34). According to the sixteenth aspect, it is possible to provide a server (7) capable of easily grasping the situation of the pet (5).
  • the pet situation estimation method includes a pet detection step, an information generation step, and an estimation step.
  • a specific region (A1) showing at least a part of the appearance of the pet (5) as the subject (H1) is detected in the image data (D1).
  • Pet information is generated in the information generation step.
  • the pet information includes at least the posture information of the pet (5) based on the trained model (first model M1) learned about the pet posture in order to recognize the image of the pet posture and the image data (D1). ..
  • the estimation step the pet status regarding at least one of the emotions and behaviors of the pet (5) shown in the specific area (A1) is estimated based on the pet information.
  • the program according to the eighteenth aspect is a program for causing one or more processors to execute the pet situation estimation method according to the seventeenth aspect. According to the eighteenth aspect, it is possible to provide a function capable of making it easy to grasp the situation of the pet (5).
  • the configurations according to the second to thirteenth aspects are not essential configurations for the pet situation estimation system (1,1A) and can be omitted as appropriate.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Environmental Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Animal Husbandry (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
PCT/JP2021/030654 2020-09-01 2021-08-20 ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム Ceased WO2022050093A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022546235A JP7479019B2 (ja) 2020-09-01 2021-08-20 ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム
US18/022,074 US12446554B2 (en) 2020-09-01 2021-08-20 Pet status assessment system, pet camera, server, pet status assessment method, and program
CN202180050178.5A CN115885313B (zh) 2020-09-01 2021-08-20 宠物状态评估系统、宠物相机、服务器、宠物状态评估方法和程序

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020147129 2020-09-01
JP2020-147129 2020-09-01

Publications (1)

Publication Number Publication Date
WO2022050093A1 true WO2022050093A1 (ja) 2022-03-10

Family

ID=80490861

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/030654 Ceased WO2022050093A1 (ja) 2020-09-01 2021-08-20 ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム

Country Status (4)

Country Link
US (1) US12446554B2 (https=)
JP (1) JP7479019B2 (https=)
CN (1) CN115885313B (https=)
WO (1) WO2022050093A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2023175931A1 (https=) * 2022-03-18 2023-09-21
JP7410607B1 (ja) 2023-07-24 2024-01-10 株式会社Eco‐Pork 飼養管理システム、および飼養管理方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4229546A4 (en) * 2020-10-14 2024-04-10 One Cup Productions Ltd. ANIMAL VISUAL IDENTIFICATION, TRACKING, SURVEILLANCE AND EVALUATION SYSTEMS AND ASSOCIATED METHODS
CN113132632B (zh) * 2021-04-06 2022-08-19 蚂蚁胜信(上海)信息技术有限公司 一种针对宠物的辅助拍摄方法和装置
CN119296129A (zh) * 2024-10-12 2025-01-10 星宠王国(北京)科技有限公司 基于图像处理的宠物监管方法、装置和存储介质
CN120260080B (zh) * 2025-06-04 2026-01-09 深圳市伟文无线通讯技术有限公司 宠物姿态及表情跟踪方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019091233A (ja) * 2017-11-14 2019-06-13 株式会社Ｊｖｃケンウッド 通知制御システム、通知制御方法およびプログラム
JP2019122368A (ja) * 2018-01-12 2019-07-25 デザミス株式会社 牛の健康状態管理システム及び管理方法並びに健康状態管理プログラム
JP2020005558A (ja) * 2018-07-06 2020-01-16 ユニ・チャーム株式会社 動物撮影装置、健康判定システム及びプログラム
JP2020014421A (ja) * 2018-07-26 2020-01-30 日本ユニシス株式会社 家畜出産予測システム
US20200205382A1 (en) * 2018-12-28 2020-07-02 Acer Incorporated Pet monitoring method and pet monitoring system

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4449483B2 (ja) 2004-02-16 2010-04-14 富士ゼロックス株式会社 画像解析装置、および画像解析方法、並びにコンピュータ・プログラム
JP2009289230A (ja) 2008-06-02 2009-12-10 Olympus Corp 画像処理装置、画像処理方法及び画像処理プログラム
JP2013065110A (ja) 2011-09-15 2013-04-11 Omron Corp 検出装置、該検出装置を備えた表示制御装置および撮影制御装置、物体検出方法、制御プログラム、ならびに、記録媒体
US11894143B2 (en) * 2013-08-27 2024-02-06 Whiskers Worldwide, LLC System and methods for integrating animal health records
US9807983B2 (en) 2014-08-22 2017-11-07 Panasonic Intellectual Property Corporation Of America Device control method for estimating a state of an animal and for determining a control detail for an electronic device
US20170196196A1 (en) * 2016-01-08 2017-07-13 Leo Trottier Animal interaction devices, systems and methods
CN106165655B (zh) 2016-06-17 2020-02-07 深圳市沃特沃德股份有限公司 一种检测动物尾巴动作的方法和系统
WO2018185718A1 (en) 2017-04-07 2018-10-11 Smaluet Solutions Private Limited A device and a method of learning a behavior of a pet in response to instructions provided to the pet
JP6914597B2 (ja) 2017-09-26 2021-08-04 東芝情報システム株式会社 ペット・人の友交度測定装置及びペット・人の友交度測定用プログラム
AU2019209465A1 (en) * 2018-01-16 2020-08-27 Habi, Inc. Methods and systems for pet wellness platform
CN108922622B (zh) * 2018-07-10 2023-10-31 平安科技（深圳）有限公司 一种动物健康监测方法、装置及计算机可读存储介质
CN109566451B (zh) 2018-11-14 2021-10-22 绍兴兴科元管业有限公司 一种基于测距传感器的动物行为检测设备
CN110175526B (zh) * 2019-04-28 2024-06-21 平安科技（深圳）有限公司 狗情绪识别模型训练方法、装置、计算机设备及存储介质
CN111597942B (zh) * 2020-05-08 2023-04-18 上海达显智能科技有限公司 一种智能宠物训导、陪伴方法、装置、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019091233A (ja) * 2017-11-14 2019-06-13 株式会社Ｊｖｃケンウッド 通知制御システム、通知制御方法およびプログラム
JP2019122368A (ja) * 2018-01-12 2019-07-25 デザミス株式会社 牛の健康状態管理システム及び管理方法並びに健康状態管理プログラム
JP2020005558A (ja) * 2018-07-06 2020-01-16 ユニ・チャーム株式会社 動物撮影装置、健康判定システム及びプログラム
JP2020014421A (ja) * 2018-07-26 2020-01-30 日本ユニシス株式会社 家畜出産予測システム
US20200205382A1 (en) * 2018-12-28 2020-07-02 Acer Incorporated Pet monitoring method and pet monitoring system

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2023175931A1 (https=) * 2022-03-18 2023-09-21
WO2023175931A1 (ja) * 2022-03-18 2023-09-21 日本電気株式会社 画像分類装置、画像分類方法、及び、記録媒体
JP7729466B2 (ja) 2022-03-18 2025-08-26 日本電気株式会社 画像分類装置、画像分類方法、及び、プログラム
JP7410607B1 (ja) 2023-07-24 2024-01-10 株式会社Eco‐Pork 飼養管理システム、および飼養管理方法
JP2025016839A (ja) * 2023-07-24 2025-02-05 株式会社Eco‐Pork 飼養管理システム、および飼養管理方法

Also Published As

Publication number Publication date
CN115885313A (zh) 2023-03-31
JPWO2022050093A1 (https=) 2022-03-10
JP7479019B2 (ja) 2024-05-08
US12446554B2 (en) 2025-10-21
US20230320328A1 (en) 2023-10-12
CN115885313B (zh) 2025-11-25

Similar Documents

Publication Publication Date Title
JP7479019B2 (ja) ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム
WO2022050092A1 (ja) ペット状況推定システム、ペットカメラ、サーバ、ペット状況推定方法、及びプログラム
US12193414B2 (en) Animal visual identification, tracking, monitoring and assessment systems and methods thereof
CN112655019B (zh) 监控农业围栏中的牲畜
US10231442B1 (en) Apparatuses and methods for smart pet alert, and storage medium thereof
US20140281975A1 (en) System for adaptive selection and presentation of context-based media in communications
US11010609B2 (en) Mobile image pickup system, learning method, mobile image pickup apparatus, information acquisition control apparatus, information acquisition control method and recording medium storing information acquisition control program
CN104932459A (zh) 一种基于物联网的多功能宠物管理监测系统
CN113420708A (zh) 宠物看护方法、装置、电子设备及存储介质
CN110278413A (zh) 图像处理方法、装置、服务器以及存储介质
JP2009265827A (ja) 対象物検出装置及び方法、対象物検出システム、プログラム
US20230186624A1 (en) Method and electronic device for providing augmented reality recommendations
KR20130007079A (ko) 동물 상태 원격 관리 방법 및 시스템
JP2023015924A (ja) 生産管理システム、生産管理方法及びプログラム
WO2022181132A1 (ja) 体重推定システム、及び、体重推定方法
JP2023056040A (ja) 活動検証のためのシステム及び方法
CN114332660A (zh) 一种动物识别方法、装置、存储介质及电子设备
KR102865188B1 (ko) 인공지능 기반의 반려동물 원격케어서버 및 이를 포함하는 시스템
KR102464539B1 (ko) 영역별 개체 모니터링을 위한 유저 인터페이스 장치
WO2024181295A1 (ja) 提示処理装置、情報提示システム、方法、及びプログラム
US20240242414A1 (en) Electronic device and method for creating avatar in virtual space
Black RAID: Real-time Animal Identification and Deterrence
CN117694268A (zh) 基于猫砂盆和投喂双场景下的异常监测方法及装置
CN120255510A (zh) 机器人的巡航控制方法、设备及可读存储介质
CN120689900A (zh) 宠物喂食方法及装置、电子设备、介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21864151

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022546235

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21864151

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18022074

Country of ref document: US