WO2023233583A1

WO2023233583A1 - Electronic device, and information processing system

Info

Publication number: WO2023233583A1
Application number: PCT/JP2022/022311
Authority: WO
Inventors: 祐介大西
Original assignee: 株式会社ソニー・インタラクティブエンタテインメント
Priority date: 2022-06-01
Filing date: 2022-06-01
Publication date: 2023-12-07

Abstract

A power generation unit 30 generates power. A power storage unit 32 stores power obtained in the power generation unit 30. An image sensor 16 captures an image of the circumference of an electronic device. A display unit 12 displays an image corresponding to a subject captured by the image sensor 16. The image sensor 16 and the display unit 12 are driven by the power stored in the power storage unit 32.

Description

Electronic equipment and information processing systems

The present disclosure relates to electronic equipment and information processing technology using electronic equipment.

Patent Document 1 discloses an information processing system that includes an information processing device, a trolley, and a mat. The trolley is equipped with an image sensor (camera) that photographs the array pattern printed on the mat, and acquires positional information and control information on the mat by reading the array pattern. The information processing device acquires position information and control information from the cart and controls the operation of the cart. The cart is equipped with a battery that can be charged by an external power source, and is driven by the power supplied from the battery.

International Publication No. 2018/25467

When electronic devices that use batteries run out of battery power, they need to be replaced or charged. Therefore, if there is no replaceable battery, or if there is no charger or external power source to charge the battery, it will not be possible to secure power for the electronic device and the electronic device will not be able to operate.

The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide technology related to electronic devices that are not equipped with secondary batteries or primary batteries (disposable batteries) that are charged by an external power source. be.

In order to solve the above problems, an electronic device according to an aspect of the present invention includes a power generation unit that generates electric power, a power storage unit that stores the power obtained by the power generation unit, and an image sensor that photographs the surroundings of the electronic device. , and a display unit that displays an image corresponding to the subject photographed by the image sensor. The image sensor and the display section are driven by electric power stored in the power storage section.

An information processing system according to another aspect of the present invention is an information processing system including an electronic device and an information processing device having a camera that photographs the electronic device, the electronic device including a power generation unit that generates electric power, The information processing device includes a power storage unit that stores power obtained by the power generation unit and a first display unit that is driven by the power stored in the power storage unit and displays an image, and the information processing device recognizes the image taken by the camera. and a second display section that displays an image according to the result recognized by the recognition section.

Any combination of the above-mentioned components, conversion of expressions of the present invention between methods, apparatuses, systems, computer programs, recording media readably recording computer programs, data structures, etc. are also aspects of the present invention. It is valid as

FIG. 3 is a diagram showing a state in which the electronic device according to the example is placed on a mat. FIG. 3 is a diagram showing a front view of the electronic device. FIG. 3 is a diagram showing a side view of the electronic device. FIG. 3 is a diagram showing a bottom view of the electronic device. FIG. 3 is a diagram showing functional blocks of an electronic device. It is a figure showing an example of an operation flow of each composition in an electronic device. FIG. 3 is a diagram showing a state in which an object image is displayed on a display unit. 1 is a diagram illustrating an example of an information processing system including an electronic device and an information processing device. FIG. 2 is a diagram showing functional blocks of an information processing device.

FIG. 1 shows a state in which an electronic device 10 according to an embodiment is placed on a mat 2. As shown in FIG. The electronic device 10 in the embodiment may be a toy that a user can hold and play with. The mat 2 is divided into a plurality of grid-like regions, and a code is printed or pasted on each region. In the embodiment, the code may include at least positional information indicating the position of the area on the mat 2 and information regarding the object associated with the area (hereinafter referred to as "object information"). The type of code may be a one-dimensional code, a two-dimensional code, or human readable text as shown. In any case, the code includes information that can be read by the electronic device 10.

The electronic device 10 includes a housing 20 having a polyhedral shape. It is preferable that the casing 20 of the embodiment has a size that allows the user to grasp it with one hand. The casing 20 may have a rectangular parallelepiped shape with a height of about 2 to 3 cm and a side length of about 3 to 5 cm, and the length and width may be the same. The electronic device 10 includes a rectangular display section 12 on its top surface, and an image sensor that photographs the codes attached to each area of the mat 2 on its bottom surface. In the embodiment, when the image sensor photographs a code, the display unit 12 displays an image corresponding to object information included in the photographed code. The displayed image may be an image representing an object.

The electronic device 10 of the embodiment may have a running function. The electronic device 10 runs on the mat 2 and stops within the divided area. For example, if the code included in each area includes destination information indicating the position of the next area, the electronic device 10 autonomously travels to the next area by reading the destination information included in the code. You may do so. Note that the electronic device 10 may run on the mat 2 according to a running instruction from an external control device (not shown).

2 shows a front view of the electronic device 10, FIG. 3 shows a side view of the electronic device 10, and FIG. 4 shows a bottom view of the electronic device 10. The electronic device 10 of the embodiment has two

wheels

14a and 14b (hereinafter referred to as "wheels 14" unless otherwise distinguished) on the bottom surface of the casing 20. A motor is connected to each wheel 14, respectively.

The image sensor 16 is a camera that photographs the surroundings of the electronic device 10, and is arranged on the bottom surface of the housing 20. Therefore, the image sensor 16 photographs the lower part of the housing 20. Note that the image sensor 16 may be provided on the front, right side, back, left side, top surface, etc. of the housing 20. The electronic device 10 of the embodiment has a function of analyzing an image taken by the image sensor 16 to identify a subject, and displaying an image corresponding to the subject on the display unit 12. Specifically, the electronic device 10 identifies the code included in the captured image, and displays on the display unit 12 an image corresponding to the object information included in the code.

The lighting section 18 has an LED that lights up, and is arranged on the bottom surface of the housing 20. The lighting section 18 is provided to protrude downward from the bottom surface of the housing 20. When the electronic device 10 is placed on the mat 2 with the top surface facing upward (the bottom surface facing downward), three points, the

wheels

14a, 14b, and the lighting section 18, are grounded. The electronic device 10 can run on the mat 2 by rotating the

wheels

14a and 14b.

FIG. 5 shows functional blocks of the electronic device 10. The electronic device 10 includes a display section 12, wheels 14, an image sensor 16, a lighting section 18, a power generation section 30, a power storage section 32, an inertial measurement unit (hereinafter referred to as "IMU") 34, a control section 36, and a communication section 42. Be prepared. The IMU 34 may be an attitude sensor including a 3-axis acceleration sensor and a 3-axis gyro sensor. The control section 36 includes a first control section 38 and a second control section 40.

The electronic device 10 includes a computer, and various functions are realized by the computer executing programs. A computer includes a memory for loading a program, one or more processors for executing the loaded program, an auxiliary storage device, other LSI, and the like as hardware. A processor is constituted by a plurality of electronic circuits including a semiconductor integrated circuit and an LSI, and the plurality of electronic circuits may be mounted on one chip or on a plurality of chips. The functional blocks shown in FIG. 5 are realized by cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by only hardware, only software, or a combination thereof. It is understood.

The electronic device 10 of the embodiment does not include a secondary battery or a primary battery that is charged by an external power source, such as a commercial power source. In the electronic device 10 of the embodiment, the power generation unit 30 has a power generation function and generates electric power used in the electronic device 10. The power storage unit 32 stores the power obtained by the power generation unit 30, and the display unit 12, image sensor 16, lighting unit 18, IMU 34, control unit 36, and communication unit 42 are driven by the power stored in the power storage unit 32. be done.

FIG. 6 shows an example of the operation flow of each component in the electronic device 10. In the embodiment, the power generation unit 30 is a motor that generates power in conjunction with the rotation of the wheel 14, and when the user holds the electronic device 10 in his hand and rotates the wheel 14, the motor generates power (S10). When a user moves the electronic device 10 back and forth with the wheels 14 on the floor or table to rotate the wheels 14, the motor connected to the wheels 14 generates electric power. The power storage unit 32 is composed of a capacitor or the like, and stores the generated power (S12).

When power is accumulated in the power storage unit 32, the first control unit 38 is immediately activated (S14) and monitors the amount of power stored in the power storage unit 32 (S16). The first control unit 38 is a microcontroller, and controls activation of the second control unit 40 according to the monitored amount of stored electricity. The first control unit 38 may monitor the amount of power stored in the power storage unit 32 by monitoring the voltage of the power storage unit 32.

If the amount of power stored in the power storage unit 32 is less than the predetermined value (N in S16), the first control unit 38 waits for the user to further rotate the wheels 14 and the amount of power stored increases. When the amount of electricity stored in the electricity storage unit 32 reaches a predetermined value or more (Y in S16), the first control unit 38 turns on the lighting unit 18 for a predetermined period of time (for example, 0.5 seconds) (S18), and the second control unit 40 The second control unit 40 is activated by supplying power from the power storage unit 32 to the second control unit 40 (S20). On the condition that the amount of power stored in the power storage unit 32 reaches a predetermined amount of power storage that allows the second control unit 40 to operate stably, the first control unit 38 controls the power stored in the power storage unit 32 to the second control. Preferably, it is supplied to section 40. The user recognizes that sufficient power has been accumulated in the electronic device 10 by checking that the lighting unit 18 is lit. Note that the lighting control of the lighting section 18 may be performed by the activated second control section 40.

The second control unit 40 can control the driving of the display unit 12, image sensor 16, IMU 34, and communication unit 42 by receiving power from the power storage unit 32. Note that the second control unit 40 may operate in a low power consumption mode immediately after being activated to reduce power consumption. In the low power consumption mode, the second control unit 40 may suppress power consumption by not driving at least the image sensor 16. Furthermore, the second control unit 40 may suppress power consumption by not driving the display unit 12 and the communication unit 42 in the low power consumption mode.

In the low power consumption mode, the second control unit 40 supplies power to the IMU 34, and the IMU 34 provides the second control unit 40 with measurement results at a predetermined period. The second control unit 40 determines whether the electronic device 10 is stationary based on the measurement result of the IMU 34 (S22). If the electronic device 10 is not stationary (N in S22), the user rotates the wheels 14 of the electronic device 10 so that the power generation unit 30 is generating electricity, or the user moves the electronic device 10 to any area on the mat 2. It is possible that they are being moved for placement. In any case, if the electronic device 10 is not stationary, it is unlikely that the electronic device 10 is in a state where it can photograph the code attached to the mat 2. By maintaining the non-supply state and not driving the image sensor 16, unnecessary power consumption is avoided (S24).

On the other hand, if the electronic device 10 is stationary (Y in S22), there is a possibility that the electronic device 10 is placed on the mat 2 and is in a state where the code attached to the mat 2 can be photographed. The control unit 40 supplies power to the image sensor 16 to drive it (S26). The image sensor 16 periodically photographs the surroundings of the electronic device 10, and the second control unit 40 analyzes the image photographed by the image sensor 16 and determines whether the code attached to the mat 2 is included in the photographed image. A search is made to see if there is one (S28). When the second control unit 40 reads the new code (Y in S28), it specifies the object information included in the code and displays the object image corresponding to the object information on the display unit 12 (S30). Note that the code read is new means that the code read this time is different from the code read last time.

FIG. 7 shows a state in which an object image is displayed on the display unit 12. The second control unit 40 identifies object information from the read code, and displays an object image associated with the object information on the display unit 12. The electronic device 10 maintains a database (not shown) in which object images corresponding to object information are recorded, and the second control unit 40 reads the object images from the database and displays them on the display unit 12.

As described above, a code is assigned to each area divided on the mat 2, and therefore, each time the electronic device 10 is placed in a different area, the second control unit 40 is assigned a code corresponding to the area where the electronic device 10 is placed. The obtained object image is displayed on the display unit 12. The display unit 12 may be electronic paper that has image storage properties. Once an image is displayed, electronic paper does not consume power to continue displaying the image, and thus contributes to power saving of the electronic device 10.

In S28, if the code read by the second control unit 40 is the same as the previous code (N in S28), the second control unit 40 does not update the display on the display unit 12. Therefore, if the electronic device 10 continues to be placed in the same area, the display unit 12 continues to display the same object image. If the amount of stored power is not zero (N in S32), the second control unit 40 repeatedly executes the process from S22, and when the amount of stored power becomes zero (Y in S32), this flow ends.

The second control unit 40 may control the imaging cycle of the image sensor 16 according to the amount of power stored in the power storage unit 32. The second control unit 40 sets the imaging period of the image sensor 16 to be relatively long when the amount of stored power is small, and sets the imaging period of the image sensor 16 to be relatively short when the amount of stored power is large. It's fine.

Further, the second control unit 40 may control the display content of the display unit 12 according to the amount of electricity stored in the electricity storage unit 32. The second control unit 40 displays, on the display unit 12, a display content in which the power consumption is relatively low when the amount of stored power is small, and a display content in which the power consumption is relatively increased when the amount of stored power is large. The contents may be displayed on the display section 12. Specifically, the second control unit 40 may display an image with richer colors or animation, and may update the image more frequently as the amount of stored electricity increases. For example, when the database associates a plurality of object images with one piece of object information, the second control unit 40 may determine the frequency of switching object images according to the amount of electrical storage.

Note that the second control unit 40 preferably erases the image on the display unit 12 when the amount of electricity stored in the electricity storage unit 32 becomes less than a predetermined threshold value. Since the display unit 12 of the embodiment is electronic paper, it continues to display the object image even if the amount of stored electricity becomes zero. Therefore, it is preferable that the second control unit 40 erase the display image on the display unit 12 immediately before the amount of stored electricity becomes zero, so that the object image does not continue to be displayed.

In the electronic device 10, the motor constituting the power generation unit 30 is driven by the electric power stored in the power storage unit 32 to rotate the wheels 14 when not generating power. The second control unit 40 has a function of controlling the rotation of the motor when power is not being generated, and may autonomously travel on the mat 2 based on destination information included in the code. By using the motor for both power generation and running in this way, the space in the housing 20 can be effectively utilized.

FIG. 8 shows an example of an information processing system 1 including an electronic device 10 and an information processing device 50. The information processing device 50 is configured to include a camera that photographs the electronic device 10. The information processing device 50 is a mobile terminal device such as a smartphone or a tablet, or smart glasses, and generates an augmented reality (AR) image and displays the AR image on the display unit 58.

FIG. 9 shows functional blocks of the information processing device 50. The information processing device 50 includes a camera 52, an image recognition section 54, a processing section 56, and a display section 58.
The information processing device 50 includes a computer, and various functions are realized by the computer executing programs. A computer includes a memory for loading a program, one or more processors for executing the loaded program, an auxiliary storage device, other LSI, and the like as hardware. A processor is constituted by a plurality of electronic circuits including a semiconductor integrated circuit and an LSI, and the plurality of electronic circuits may be mounted on one chip or on a plurality of chips. The functional blocks shown in FIG. 9 are realized by cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by only hardware, only software, or a combination thereof. It is understood.

The processing unit 56 displays the camera image taken by the camera 52 of the real space on the display unit 58. In the example shown in FIG. 8, the camera 52 is photographing the electronic device 10, and the real space including the electronic device 10 is displayed on the display unit 58. When the image recognition unit 54 recognizes the object image included in the camera image, the image recognition unit 54 provides the recognition result, that is, information identifying the recognized object image (object image ID) to the processing unit 56 . The processing unit 56 maintains a database in which three-dimensional images corresponding to object image IDs are recorded, and reads out the three-dimensional images from the database based on the object image ID specified by the image recognition unit 54.

The processing unit 56 superimposes the read three-dimensional image on the camera image displayed on the display unit 58. Thereby, the display section 58 displays a three-dimensional image according to the result recognized by the image recognition section 54, superimposed on the object image. In this way, the processing unit 56 superimposes the three-dimensional image on the camera image, allowing the user to enjoy AR.

The present invention has been described above based on examples. It will be understood by those skilled in the art that the embodiments are merely illustrative, and that various modifications can be made to the combinations of their constituent elements and processing processes, and that such modifications are also within the scope of the present invention. The electronic device 10 may transmit information regarding the amount of charge from the communication unit 42 to the information processing device 50.

In the embodiment, it has been explained that the power generation unit 30 is a motor connected to the wheels 14, but it may be another type of power generation module. For example, the power generation unit 30 may be a solar power generation module or a vibration power generation module. Further, in the embodiment, it has been explained that the display unit 12 is electronic paper, but it may be other types of displays such as a liquid crystal display or an organic EL display.

A plurality of electronic devices 10 may exist in the information processing system 1. The electronic devices 10 may notify each other of the amount of power stored through the communication unit 42, and the electronic device 10 with the largest amount of charge may display information on the display unit 12 informing the user that the amount of charge is the largest. By knowing the information regarding the amount of charge, the user can obtain an opportunity to generate electricity. Note that the electronic device 10 may have a non-contact power supply function so that power can be supplied from the electronic device 10 with a large amount of charge to the electronic device 10 with a small amount of charge.

In the embodiment, it has been explained that the electronic device 10 is a toy that the user can hold and play with, but the electronic device 10 does not have to be a toy and may have a size that the user can hold in the hand. It doesn't have to be. The electronic device 10 may be a movable device (device) having wheels 14, and may be, for example, a wheeled robot, a trolley, a vehicle, a walker, a wheelchair, a radio-controlled car, an automatic vacuum cleaner, or the like. Further, the electronic device 10 may be a device (device) that does not have wheels 14, and may be, for example, a mobile terminal device such as a smartphone or a tablet, furniture, or an electrical appliance.

In the embodiment, it has been explained that the second control unit 40 identifies the code included in the image captured by the image sensor 16 and causes the display unit 12 to display an image corresponding to the code. An image corresponding to the image may be displayed on the display unit 12. The image sensor 16 may be provided not only on the bottom surface of the housing 20 but also on the front, right side, back, left side, top surface, etc. of the housing 20 to photograph the surrounding environment of the housing 20. analyzes the captured image, identifies the subject, and causes the display unit 12 to display an image corresponding to the subject. For example, the second control unit 40 may specify a subject, such as an object, a person's face, a person's posture or movement, or an animal, included in the photographed image, and cause the display unit 12 to display an image corresponding to the subject.

Further, in the information processing system 1, the information processing device 50 may display information for increasing the efficiency of power generation and information regarding the actual amount of stored electricity on the display unit 58 in AR in order to increase the user's motivation for power generation. good.

In the embodiment, it has been explained that the electronic device 10 is not equipped with a secondary battery or a primary battery that is charged by an external power source, but in a modified example, the electronic device 10 is equipped with a secondary battery and/or a primary battery that is charged by an external power source. Alternatively, a primary battery may be installed as an auxiliary power source. For example, the second control unit 40 may use the electric power of the power storage unit 32 as long as the amount of power stored in the power storage unit 32 does not run out, and may use the auxiliary power source when the amount of power stored in the power storage unit 32 runs out.

The present disclosure can be used in the technical field of electronic devices used by users.

DESCRIPTION OF SYMBOLS 1... Information processing system, 2... Mat, 10... Electronic equipment, 12... Display unit, 14, 14a, 14b... Wheels, 16... Image sensor, 18... Lighting Part, 20... Housing, 30... Power generation unit, 32... Power storage unit, 34... IMU, 36... Control unit, 38... First control unit, 40... th 2 control unit, 42... communication unit, 50... information processing device, 52... camera, 54... image recognition unit, 56... processing unit, 58... display unit.

Claims

An electronic device,
a power generation section that generates electric power;
a power storage unit that stores electric power obtained in the power generation unit;
an image sensor that photographs the area around the electronic device;
a display unit that displays an image corresponding to the subject photographed by the image sensor;
The image sensor and the display section are driven by electric power stored in the power storage section.
An electronic device characterized by:
The display unit displays an image corresponding to a code included in an image captured by the image sensor.
The electronic device according to claim 1, characterized in that:
Equipped with multiple wheels,
The power generation unit includes a motor that generates power in conjunction with rotation of the wheel.
The electronic device according to claim 1, characterized in that:
The motor is driven by the electric power stored in the power storage unit to rotate the wheel when not generating electricity.
The electronic device according to claim 3, characterized in that:
a first control unit that monitors the amount of electricity stored in the electricity storage unit;
a second control unit that controls driving of the image sensor and the display unit,
The first control unit supplies electric power to the second control unit when the amount of electricity stored in the power storage unit reaches a predetermined value or more.
The electronic device according to claim 1, characterized in that:
Equipped with an inertial measurement device,
If the second control unit determines that the electronic device is stationary based on the measurement result of the inertial measurement device, the second control unit drives the image sensor, and if the second control unit does not determine that the electronic device is stationary, not driving the image sensor;
The electronic device according to claim 5, characterized in that:
The second control unit controls the imaging cycle of the image sensor and/or the display content of the display unit according to the amount of electricity stored in the electricity storage unit.
The electronic device according to claim 5, characterized in that:
the display section is electronic paper;
The electronic device according to claim 1, characterized in that:
An information processing system comprising an electronic device and an information processing device having a camera for photographing the electronic device,
The electronic device is
a power generation section that generates electric power;
a power storage unit that stores electric power obtained in the power generation unit;
a first display unit that is driven by electric power stored in the power storage unit and displays an image;
The information processing device includes:
a recognition unit that recognizes the image taken by the camera;
a second display unit that displays an image according to the result recognized by the recognition unit;
An information processing system characterized by: